2023-10-14T10:23:18.430 INFO:root:teuthology version: 0.0.1.dev197+g8cdab07 2023-10-14T10:23:18.432 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --description fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/basic}} --archive /home/teuthworker/archive/rishabh-2023-10-13_18:04:14-fs-rishabh-2023oct13-testing-default-smithi/7425219 --name rishabh-2023-10-13_18:04:14-fs-rishabh-2023oct13-testing-default-smithi --owner scheduled_rishabh@teuthology -- /home/teuthworker/archive/rishabh-2023-10-13_18:04:14-fs-rishabh-2023oct13-testing-default-smithi/7425219/orig.config.yaml 2023-10-14T10:23:18.481 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-14T10:23:18.573 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2023-10-13_18:04:14-fs-rishabh-2023oct13-testing-default-smithi/7425219 branch: rishabh-2023oct13 description: fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/basic}} email: null first_in_suite: false job_id: '7425219' kernel: client: sha1: distro flavor: default kdb: true sha1: 47672119149b103ec7cec2cc65c063ccbb001ac7 ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: rishabh-2023-10-13_18:04:14-fs-rishabh-2023oct13-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: rishabh-2023oct13 ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug finisher: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: 4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 kclient: syntax: v2 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: rishabh-2023oct13 sha1: 4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 owner: scheduled_rishabh@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 6006 sha1: 4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 sleep_before_teardown: 0 subset: 402/512 suite: fs suite_branch: rishabh-2023oct13 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi067.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi072.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - kclient: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_volumes.TestVolumes - tasks.cephfs.test_volumes.TestSubvolumeGroups - tasks.cephfs.test_volumes.TestSubvolumes - tasks.cephfs.test_subvolume.TestSubvolume teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 8cdab074dcca9a68965bc5a50e9c30b691949723 timestamp: 2023-10-13_18:04:14 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2830 2023-10-14T10:23:18.573 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/qa; will attempt to use it 2023-10-14T10:23:18.574 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/qa/tasks 2023-10-14T10:23:18.575 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-10-14T10:23:18.576 INFO:teuthology.task.internal:Checking packages... 2023-10-14T10:23:18.600 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0' 2023-10-14T10:23:18.601 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-10-14T10:23:18.601 INFO:teuthology.packaging:ref: None 2023-10-14T10:23:18.602 INFO:teuthology.packaging:tag: None 2023-10-14T10:23:18.602 INFO:teuthology.packaging:branch: rishabh-2023oct13 2023-10-14T10:23:18.602 INFO:teuthology.packaging:sha1: 4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 2023-10-14T10:23:18.603 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=rishabh-2023oct13 2023-10-14T10:23:18.833 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-6730.g4ba8a190 2023-10-14T10:23:18.834 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-10-14T10:23:18.840 INFO:teuthology.task.internal:no buildpackages task found 2023-10-14T10:23:18.841 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-10-14T10:23:18.847 INFO:teuthology.task.internal:Saving configuration 2023-10-14T10:23:18.869 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-10-14T10:23:18.875 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-14T10:23:18.904 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi067.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-10-13_18:04:14-fs-rishabh-2023oct13-testing-default-smithi/7425219', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-14 10:17:40.368743', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-10-14T10:23:18.927 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi072.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-10-13_18:04:14-fs-rishabh-2023oct13-testing-default-smithi/7425219', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-14 10:17:40.367152', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-10-14T10:23:18.928 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-10-14T10:23:18.934 INFO:teuthology.task.internal:roles: ubuntu@smithi067.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-10-14T10:23:18.934 INFO:teuthology.task.internal:roles: ubuntu@smithi072.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-10-14T10:23:18.935 INFO:teuthology.run_tasks:Running task console_log... 2023-10-14T10:23:19.045 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fa7d1a0eb80>, signals=[15]) 2023-10-14T10:23:19.046 INFO:teuthology.run_tasks:Running task internal.connect... 2023-10-14T10:23:19.052 INFO:teuthology.task.internal:Opening connections... 2023-10-14T10:23:19.052 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi067.front.sepia.ceph.com 2023-10-14T10:23:19.054 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi067.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-14T10:23:19.124 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi072.front.sepia.ceph.com 2023-10-14T10:23:19.125 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-14T10:23:19.189 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-10-14T10:23:19.195 DEBUG:teuthology.orchestra.run.smithi067:> uname -m 2023-10-14T10:23:19.217 INFO:teuthology.orchestra.run.smithi067.stdout:x86_64 2023-10-14T10:23:19.218 DEBUG:teuthology.orchestra.run.smithi067:> cat /etc/os-release 2023-10-14T10:23:19.277 INFO:teuthology.orchestra.run.smithi067.stdout:NAME="Red Hat Enterprise Linux" 2023-10-14T10:23:19.277 INFO:teuthology.orchestra.run.smithi067.stdout:VERSION="8.6 (Ootpa)" 2023-10-14T10:23:19.278 INFO:teuthology.orchestra.run.smithi067.stdout:ID="rhel" 2023-10-14T10:23:19.278 INFO:teuthology.orchestra.run.smithi067.stdout:ID_LIKE="fedora" 2023-10-14T10:23:19.278 INFO:teuthology.orchestra.run.smithi067.stdout:VERSION_ID="8.6" 2023-10-14T10:23:19.278 INFO:teuthology.orchestra.run.smithi067.stdout:PLATFORM_ID="platform:el8" 2023-10-14T10:23:19.278 INFO:teuthology.orchestra.run.smithi067.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-10-14T10:23:19.279 INFO:teuthology.orchestra.run.smithi067.stdout:ANSI_COLOR="0;31" 2023-10-14T10:23:19.279 INFO:teuthology.orchestra.run.smithi067.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-10-14T10:23:19.279 INFO:teuthology.orchestra.run.smithi067.stdout:HOME_URL="https://www.redhat.com/" 2023-10-14T10:23:19.279 INFO:teuthology.orchestra.run.smithi067.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-10-14T10:23:19.279 INFO:teuthology.orchestra.run.smithi067.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-14T10:23:19.280 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-14T10:23:19.280 INFO:teuthology.orchestra.run.smithi067.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-14T10:23:19.280 INFO:teuthology.orchestra.run.smithi067.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-10-14T10:23:19.280 INFO:teuthology.orchestra.run.smithi067.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-10-14T10:23:19.280 INFO:teuthology.orchestra.run.smithi067.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-10-14T10:23:19.281 INFO:teuthology.lock.ops:Updating smithi067.front.sepia.ceph.com on lock server 2023-10-14T10:23:19.303 DEBUG:teuthology.orchestra.run.smithi072:> uname -m 2023-10-14T10:23:19.325 INFO:teuthology.orchestra.run.smithi072.stdout:x86_64 2023-10-14T10:23:19.325 DEBUG:teuthology.orchestra.run.smithi072:> cat /etc/os-release 2023-10-14T10:23:19.384 INFO:teuthology.orchestra.run.smithi072.stdout:NAME="Red Hat Enterprise Linux" 2023-10-14T10:23:19.384 INFO:teuthology.orchestra.run.smithi072.stdout:VERSION="8.6 (Ootpa)" 2023-10-14T10:23:19.384 INFO:teuthology.orchestra.run.smithi072.stdout:ID="rhel" 2023-10-14T10:23:19.384 INFO:teuthology.orchestra.run.smithi072.stdout:ID_LIKE="fedora" 2023-10-14T10:23:19.385 INFO:teuthology.orchestra.run.smithi072.stdout:VERSION_ID="8.6" 2023-10-14T10:23:19.385 INFO:teuthology.orchestra.run.smithi072.stdout:PLATFORM_ID="platform:el8" 2023-10-14T10:23:19.385 INFO:teuthology.orchestra.run.smithi072.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-10-14T10:23:19.385 INFO:teuthology.orchestra.run.smithi072.stdout:ANSI_COLOR="0;31" 2023-10-14T10:23:19.385 INFO:teuthology.orchestra.run.smithi072.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-10-14T10:23:19.386 INFO:teuthology.orchestra.run.smithi072.stdout:HOME_URL="https://www.redhat.com/" 2023-10-14T10:23:19.386 INFO:teuthology.orchestra.run.smithi072.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-10-14T10:23:19.386 INFO:teuthology.orchestra.run.smithi072.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-14T10:23:19.386 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-14T10:23:19.386 INFO:teuthology.orchestra.run.smithi072.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-14T10:23:19.387 INFO:teuthology.orchestra.run.smithi072.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-10-14T10:23:19.387 INFO:teuthology.orchestra.run.smithi072.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-10-14T10:23:19.387 INFO:teuthology.orchestra.run.smithi072.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-10-14T10:23:19.387 INFO:teuthology.lock.ops:Updating smithi072.front.sepia.ceph.com on lock server 2023-10-14T10:23:19.412 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-10-14T10:23:19.420 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-10-14T10:23:19.426 INFO:teuthology.task.internal:Checking for old test directory... 2023-10-14T10:23:19.426 DEBUG:teuthology.orchestra.run.smithi067:> test '!' -e /home/ubuntu/cephtest 2023-10-14T10:23:19.429 DEBUG:teuthology.orchestra.run.smithi072:> test '!' -e /home/ubuntu/cephtest 2023-10-14T10:23:19.448 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-10-14T10:23:19.505 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-10-14T10:23:19.505 DEBUG:teuthology.orchestra.run.smithi067:> test -z $(ls -A /var/lib/ceph) 2023-10-14T10:23:19.508 DEBUG:teuthology.orchestra.run.smithi072:> test -z $(ls -A /var/lib/ceph) 2023-10-14T10:23:19.534 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-10-14T10:23:19.587 INFO:teuthology.run_tasks:Running task kernel... 2023-10-14T10:23:19.601 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': '47672119149b103ec7cec2cc65c063ccbb001ac7'} 2023-10-14T10:23:19.602 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.2': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}, 'client.3': {'sha1': 'distro'}} 2023-10-14T10:23:19.602 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2023-10-14T10:23:19.603 DEBUG:teuthology.orchestra.run.smithi067:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-14T10:23:19.604 DEBUG:teuthology.orchestra.run.smithi072:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-14T10:23:19.623 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-14T10:23:19.623 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-14T10:23:19.624 DEBUG:teuthology.orchestra.run.smithi067:> uname -r 2023-10-14T10:23:19.625 DEBUG:teuthology.orchestra.run.smithi072:> uname -r 2023-10-14T10:23:19.685 INFO:teuthology.orchestra.run.smithi072.stdout:4.18.0-372.9.1.el8.x86_64 2023-10-14T10:23:19.687 INFO:teuthology.task.kernel:Running kernel on smithi072: 4.18.0-372.9.1.el8.x86_64 2023-10-14T10:23:19.687 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum install -y kernel 2023-10-14T10:23:19.688 INFO:teuthology.orchestra.run.smithi067.stdout:4.18.0-372.9.1.el8.x86_64 2023-10-14T10:23:19.689 INFO:teuthology.task.kernel:Running kernel on smithi067: 4.18.0-372.9.1.el8.x86_64 2023-10-14T10:23:19.689 DEBUG:teuthology.orchestra.run.smithi067:> sudo yum install -y kernel 2023-10-14T10:23:20.990 INFO:teuthology.orchestra.run.smithi067.stdout:Updating Subscription Management repositories. 2023-10-14T10:23:20.991 INFO:teuthology.orchestra.run.smithi067.stdout:Unable to read consumer identity 2023-10-14T10:23:21.195 INFO:teuthology.orchestra.run.smithi072.stdout:Updating Subscription Management repositories. 2023-10-14T10:23:21.195 INFO:teuthology.orchestra.run.smithi072.stdout:Unable to read consumer identity 2023-10-14T10:23:36.787 INFO:teuthology.orchestra.run.smithi067.stdout:Copr repo for python3-asyncssh owned by ceph 243 B/s | 3.5 kB 00:14 2023-10-14T10:23:37.454 INFO:teuthology.orchestra.run.smithi067.stdout:Extra Packages for Enterprise Linux 25 MB/s | 16 MB 00:00 2023-10-14T10:23:41.137 INFO:teuthology.orchestra.run.smithi067.stdout:lab-extras 523 kB/s | 24 kB 00:00 2023-10-14T10:23:41.865 INFO:teuthology.orchestra.run.smithi067.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2023-10-14T10:23:41.919 INFO:teuthology.orchestra.run.smithi067.stdout:Dependencies resolved. 2023-10-14T10:23:41.920 INFO:teuthology.orchestra.run.smithi067.stdout:Nothing to do. 2023-10-14T10:23:41.920 INFO:teuthology.orchestra.run.smithi067.stdout:Complete! 2023-10-14T10:23:42.029 DEBUG:teuthology.orchestra.run.smithi067:> echo no | sudo yum reinstall kernel || true 2023-10-14T10:23:42.419 INFO:teuthology.orchestra.run.smithi067.stdout:Updating Subscription Management repositories. 2023-10-14T10:23:42.420 INFO:teuthology.orchestra.run.smithi067.stdout:Unable to read consumer identity 2023-10-14T10:23:42.700 INFO:teuthology.orchestra.run.smithi067.stdout:Last metadata expiration check: 0:00:01 ago on Sat 14 Oct 2023 10:23:41 AM UTC. 2023-10-14T10:23:42.743 INFO:teuthology.orchestra.run.smithi067.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-10-14T10:23:42.781 INFO:teuthology.orchestra.run.smithi067.stderr:Error: No packages marked for reinstall. 2023-10-14T10:23:42.828 DEBUG:teuthology.orchestra.run.smithi067:> sudo yum reinstall -y kernel || true 2023-10-14T10:23:43.176 INFO:teuthology.orchestra.run.smithi067.stdout:Updating Subscription Management repositories. 2023-10-14T10:23:43.176 INFO:teuthology.orchestra.run.smithi067.stdout:Unable to read consumer identity 2023-10-14T10:23:43.449 INFO:teuthology.orchestra.run.smithi067.stdout:Last metadata expiration check: 0:00:02 ago on Sat 14 Oct 2023 10:23:41 AM UTC. 2023-10-14T10:23:43.493 INFO:teuthology.orchestra.run.smithi067.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-10-14T10:23:43.518 INFO:teuthology.orchestra.run.smithi067.stderr:Error: No packages marked for reinstall. 2023-10-14T10:23:43.565 DEBUG:teuthology.orchestra.run.smithi067:> rpm -q kernel | sort -rV | head -n 1 2023-10-14T10:23:43.648 INFO:teuthology.orchestra.run.smithi067.stdout:kernel-4.18.0-372.9.1.el8.x86_64 2023-10-14T10:23:43.649 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2023-10-14T10:23:43.649 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-10-14T10:23:43.649 DEBUG:teuthology.parallel:result is None 2023-10-14T10:23:44.127 INFO:teuthology.orchestra.run.smithi072.stdout:Copr repo for python3-asyncssh owned by ceph 175 B/s | 3.5 kB 00:20 2023-10-14T10:23:44.903 INFO:teuthology.orchestra.run.smithi072.stdout:Extra Packages for Enterprise Linux 21 MB/s | 16 MB 00:00 2023-10-14T10:23:48.364 INFO:teuthology.orchestra.run.smithi072.stdout:lab-extras 398 kB/s | 24 kB 00:00 2023-10-14T10:23:49.085 INFO:teuthology.orchestra.run.smithi072.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2023-10-14T10:23:49.174 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2023-10-14T10:23:49.175 INFO:teuthology.orchestra.run.smithi072.stdout:Nothing to do. 2023-10-14T10:23:49.176 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2023-10-14T10:23:49.332 DEBUG:teuthology.orchestra.run.smithi072:> echo no | sudo yum reinstall kernel || true 2023-10-14T10:23:49.704 INFO:teuthology.orchestra.run.smithi072.stdout:Updating Subscription Management repositories. 2023-10-14T10:23:49.704 INFO:teuthology.orchestra.run.smithi072.stdout:Unable to read consumer identity 2023-10-14T10:23:49.964 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:00:01 ago on Sat 14 Oct 2023 10:23:48 AM UTC. 2023-10-14T10:23:50.015 INFO:teuthology.orchestra.run.smithi072.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-10-14T10:23:50.062 INFO:teuthology.orchestra.run.smithi072.stderr:Error: No packages marked for reinstall. 2023-10-14T10:23:50.109 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum reinstall -y kernel || true 2023-10-14T10:23:50.439 INFO:teuthology.orchestra.run.smithi072.stdout:Updating Subscription Management repositories. 2023-10-14T10:23:50.439 INFO:teuthology.orchestra.run.smithi072.stdout:Unable to read consumer identity 2023-10-14T10:23:50.698 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:00:02 ago on Sat 14 Oct 2023 10:23:48 AM UTC. 2023-10-14T10:23:50.739 INFO:teuthology.orchestra.run.smithi072.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-10-14T10:23:50.775 INFO:teuthology.orchestra.run.smithi072.stderr:Error: No packages marked for reinstall. 2023-10-14T10:23:50.821 DEBUG:teuthology.orchestra.run.smithi072:> rpm -q kernel | sort -rV | head -n 1 2023-10-14T10:23:50.901 INFO:teuthology.orchestra.run.smithi072.stdout:kernel-4.18.0-372.9.1.el8.x86_64 2023-10-14T10:23:50.901 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2023-10-14T10:23:50.902 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-10-14T10:23:50.902 DEBUG:teuthology.parallel:result is None 2023-10-14T10:23:50.903 INFO:teuthology.run_tasks:Running task internal.base... 2023-10-14T10:23:50.910 INFO:teuthology.task.internal:Creating test directory... 2023-10-14T10:23:50.913 DEBUG:teuthology.orchestra.run.smithi067:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-14T10:23:50.916 DEBUG:teuthology.orchestra.run.smithi072:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-14T10:23:50.934 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-10-14T10:23:50.941 INFO:teuthology.run_tasks:Running task internal.archive... 2023-10-14T10:23:50.947 INFO:teuthology.task.internal:Creating archive directory... 2023-10-14T10:23:50.947 DEBUG:teuthology.orchestra.run.smithi067:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-14T10:23:50.977 DEBUG:teuthology.orchestra.run.smithi072:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-14T10:23:51.008 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-10-14T10:23:51.015 INFO:teuthology.task.internal:Enabling coredump saving... 2023-10-14T10:23:51.015 DEBUG:teuthology.orchestra.run.smithi067:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-14T10:23:51.038 DEBUG:teuthology.orchestra.run.smithi072:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-14T10:23:51.074 INFO:teuthology.orchestra.run.smithi067.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-14T10:23:51.080 INFO:teuthology.orchestra.run.smithi072.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-14T10:23:51.091 INFO:teuthology.orchestra.run.smithi072.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-14T10:23:51.092 INFO:teuthology.orchestra.run.smithi067.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-14T10:23:51.094 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-10-14T10:23:51.101 INFO:teuthology.task.internal:Configuring sudo... 2023-10-14T10:23:51.102 DEBUG:teuthology.orchestra.run.smithi067:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-14T10:23:51.136 DEBUG:teuthology.orchestra.run.smithi072:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-14T10:23:51.164 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-10-14T10:23:51.173 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-10-14T10:23:51.173 DEBUG:teuthology.orchestra.run.smithi067:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-14T10:23:51.206 DEBUG:teuthology.orchestra.run.smithi072:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-14T10:23:51.223 DEBUG:teuthology.orchestra.run.smithi067:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-14T10:23:51.307 DEBUG:teuthology.orchestra.run.smithi067:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-14T10:23:51.377 DEBUG:teuthology.orchestra.run.smithi067:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-14T10:23:51.422 DEBUG:teuthology.orchestra.run.smithi067:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-14T10:23:51.466 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2023-10-14T10:23:51.467 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-14T10:23:51.535 DEBUG:teuthology.orchestra.run.smithi072:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-14T10:23:51.579 DEBUG:teuthology.orchestra.run.smithi072:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-14T10:23:51.631 DEBUG:teuthology.orchestra.run.smithi072:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-14T10:23:51.716 DEBUG:teuthology.orchestra.run.smithi072:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-14T10:23:51.756 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-10-14T10:23:51.756 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-14T10:23:51.823 DEBUG:teuthology.orchestra.run.smithi067:> sudo service rsyslog restart 2023-10-14T10:23:51.825 DEBUG:teuthology.orchestra.run.smithi072:> sudo service rsyslog restart 2023-10-14T10:23:51.888 INFO:teuthology.orchestra.run.smithi067.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-14T10:23:51.928 INFO:teuthology.orchestra.run.smithi072.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-14T10:23:52.235 INFO:teuthology.run_tasks:Running task internal.timer... 2023-10-14T10:23:52.243 INFO:teuthology.task.internal:Starting timer... 2023-10-14T10:23:52.244 INFO:teuthology.run_tasks:Running task pcp... 2023-10-14T10:23:52.253 INFO:teuthology.run_tasks:Running task selinux... 2023-10-14T10:23:52.264 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-10-14T10:23:52.265 DEBUG:teuthology.orchestra.run.smithi067:> sudo service auditd rotate 2023-10-14T10:23:52.335 INFO:teuthology.orchestra.run.smithi067.stdout:Rotating logs: 2023-10-14T10:23:52.337 DEBUG:teuthology.orchestra.run.smithi072:> sudo service auditd rotate 2023-10-14T10:23:52.393 INFO:teuthology.orchestra.run.smithi072.stdout:Rotating logs: 2023-10-14T10:23:52.394 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-10-14T10:23:52.395 DEBUG:teuthology.orchestra.run.smithi067:> /usr/sbin/getenforce 2023-10-14T10:23:52.421 INFO:teuthology.orchestra.run.smithi067.stdout:Permissive 2023-10-14T10:23:52.421 DEBUG:teuthology.orchestra.run.smithi072:> /usr/sbin/getenforce 2023-10-14T10:23:52.446 INFO:teuthology.orchestra.run.smithi072.stdout:Permissive 2023-10-14T10:23:52.446 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi067.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi072.front.sepia.ceph.com': 'permissive'} 2023-10-14T10:23:52.447 DEBUG:teuthology.orchestra.run.smithi067:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-14T10:23:52.491 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-14T10:23:52.491 DEBUG:teuthology.orchestra.run.smithi072:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-14T10:23:52.518 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-14T10:23:52.518 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-10-14T10:23:52.519 DEBUG:teuthology.orchestra.run.smithi067:> sudo /usr/sbin/setenforce permissive 2023-10-14T10:23:52.568 DEBUG:teuthology.orchestra.run.smithi072:> sudo /usr/sbin/setenforce permissive 2023-10-14T10:23:52.594 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-10-14T10:23:52.605 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-10-14T10:23:52.611 INFO:teuthology.repo_utils:Fetching main from origin 2023-10-14T10:23:52.734 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-10-14T10:23:52.753 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-10-14T10:23:52.755 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi067.front.sepia.ceph.com,smithi072.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-10-14T10:32:48.938 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi067.front.sepia.ceph.com'), Remote(name='ubuntu@smithi072.front.sepia.ceph.com')] 2023-10-14T10:32:48.939 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi067.front.sepia.ceph.com' 2023-10-14T10:32:48.940 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi067.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-14T10:32:49.009 DEBUG:teuthology.orchestra.run.smithi067:> true 2023-10-14T10:32:49.075 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi067.front.sepia.ceph.com' 2023-10-14T10:32:49.076 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi072.front.sepia.ceph.com' 2023-10-14T10:32:49.076 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-14T10:32:49.144 DEBUG:teuthology.orchestra.run.smithi072:> true 2023-10-14T10:32:49.211 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi072.front.sepia.ceph.com' 2023-10-14T10:32:49.211 INFO:teuthology.run_tasks:Running task clock... 2023-10-14T10:32:49.221 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-10-14T10:32:49.222 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-14T10:32:49.222 DEBUG:teuthology.orchestra.run.smithi067:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-14T10:32:49.224 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-14T10:32:49.225 DEBUG:teuthology.orchestra.run.smithi072:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-14T10:32:49.254 INFO:teuthology.orchestra.run.smithi067.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-14T10:32:49.270 INFO:teuthology.orchestra.run.smithi067.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-14T10:32:49.283 INFO:teuthology.orchestra.run.smithi072.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-14T10:32:49.297 INFO:teuthology.orchestra.run.smithi072.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-14T10:32:49.354 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: ntpd: command not found 2023-10-14T10:32:49.367 INFO:teuthology.orchestra.run.smithi067.stdout:506 Cannot talk to daemon 2023-10-14T10:32:49.381 INFO:teuthology.orchestra.run.smithi067.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-14T10:32:49.386 INFO:teuthology.orchestra.run.smithi072.stderr:sudo: ntpd: command not found 2023-10-14T10:32:49.396 INFO:teuthology.orchestra.run.smithi067.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-14T10:32:49.398 INFO:teuthology.orchestra.run.smithi072.stdout:506 Cannot talk to daemon 2023-10-14T10:32:49.412 INFO:teuthology.orchestra.run.smithi072.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-14T10:32:49.425 INFO:teuthology.orchestra.run.smithi072.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-14T10:32:49.480 INFO:teuthology.orchestra.run.smithi067.stderr:bash: ntpq: command not found 2023-10-14T10:32:49.485 INFO:teuthology.orchestra.run.smithi067.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-14T10:32:49.485 INFO:teuthology.orchestra.run.smithi067.stdout:=============================================================================== 2023-10-14T10:32:49.486 INFO:teuthology.orchestra.run.smithi067.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-14T10:32:49.486 INFO:teuthology.orchestra.run.smithi067.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-14T10:32:49.486 INFO:teuthology.orchestra.run.smithi067.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-14T10:32:49.486 INFO:teuthology.orchestra.run.smithi067.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-14T10:32:49.499 INFO:teuthology.orchestra.run.smithi072.stderr:bash: ntpq: command not found 2023-10-14T10:32:49.503 INFO:teuthology.orchestra.run.smithi072.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-14T10:32:49.504 INFO:teuthology.orchestra.run.smithi072.stdout:=============================================================================== 2023-10-14T10:32:49.504 INFO:teuthology.orchestra.run.smithi072.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-14T10:32:49.504 INFO:teuthology.orchestra.run.smithi072.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-14T10:32:49.504 INFO:teuthology.orchestra.run.smithi072.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-14T10:32:49.504 INFO:teuthology.orchestra.run.smithi072.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-14T10:32:49.505 INFO:teuthology.run_tasks:Running task install... 2023-10-14T10:32:49.513 DEBUG:teuthology.task.install:project ceph 2023-10-14T10:32:49.514 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0'}} 2023-10-14T10:32:49.514 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0'} 2023-10-14T10:32:49.514 INFO:teuthology.task.install:Using flavor: default 2023-10-14T10:32:49.520 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-10-14T10:32:49.520 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-10-14T10:32:49.521 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0', 'tag': None, 'wait_for_package': False} 2023-10-14T10:32:49.521 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 2023-10-14T10:32:49.523 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0', 'tag': None, 'wait_for_package': False} 2023-10-14T10:32:49.523 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 2023-10-14T10:32:49.674 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/centos/8/flavors/default/ 2023-10-14T10:32:49.674 INFO:teuthology.task.install.rpm:Package version is 18.0.0-6730.g4ba8a190 2023-10-14T10:32:49.738 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/centos/8/flavors/default/ 2023-10-14T10:32:49.739 INFO:teuthology.task.install.rpm:Package version is 18.0.0-6730.g4ba8a190 2023-10-14T10:32:49.782 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-14T10:32:49.782 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2023-10-14T10:32:49.783 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-14T10:32:49.812 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-10-14T10:32:49.812 DEBUG:teuthology.orchestra.run.smithi067:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-14T10:32:49.845 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-14T10:32:49.846 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-10-14T10:32:49.846 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-14T10:32:49.875 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-10-14T10:32:49.875 DEBUG:teuthology.orchestra.run.smithi072:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-14T10:32:49.881 DEBUG:teuthology.orchestra.run.smithi067:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-14T10:32:49.942 DEBUG:teuthology.orchestra.run.smithi072:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-14T10:32:49.959 DEBUG:teuthology.orchestra.run.smithi067:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-14T10:32:50.019 DEBUG:teuthology.orchestra.run.smithi072:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-14T10:32:50.025 INFO:teuthology.orchestra.run.smithi067.stdout:check_obsoletes = 1 2023-10-14T10:32:50.027 DEBUG:teuthology.orchestra.run.smithi067:> sudo yum clean all 2023-10-14T10:32:50.085 INFO:teuthology.orchestra.run.smithi072.stdout:check_obsoletes = 1 2023-10-14T10:32:50.086 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum clean all 2023-10-14T10:32:50.416 INFO:teuthology.orchestra.run.smithi067.stdout:Updating Subscription Management repositories. 2023-10-14T10:32:50.457 INFO:teuthology.orchestra.run.smithi072.stdout:Updating Subscription Management repositories. 2023-10-14T10:32:50.673 INFO:teuthology.orchestra.run.smithi067.stdout:44 files removed 2023-10-14T10:32:50.686 INFO:teuthology.orchestra.run.smithi072.stdout:44 files removed 2023-10-14T10:32:50.721 DEBUG:teuthology.orchestra.run.smithi067:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-10-14T10:32:50.732 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-10-14T10:32:51.062 INFO:teuthology.orchestra.run.smithi072.stdout:Updating Subscription Management repositories. 2023-10-14T10:32:51.073 INFO:teuthology.orchestra.run.smithi067.stdout:Updating Subscription Management repositories. 2023-10-14T10:32:51.744 INFO:teuthology.orchestra.run.smithi067.stdout:ceph packages for x86_64 270 kB/s | 81 kB 00:00 2023-10-14T10:32:51.776 INFO:teuthology.orchestra.run.smithi072.stdout:ceph packages for x86_64 224 kB/s | 81 kB 00:00 2023-10-14T10:32:52.112 INFO:teuthology.orchestra.run.smithi067.stdout:ceph noarch packages 28 kB/s | 9.4 kB 00:00 2023-10-14T10:32:52.143 INFO:teuthology.orchestra.run.smithi072.stdout:ceph noarch packages 28 kB/s | 9.4 kB 00:00 2023-10-14T10:32:52.404 INFO:teuthology.orchestra.run.smithi067.stdout:ceph source packages 5.5 kB/s | 1.5 kB 00:00 2023-10-14T10:32:52.495 INFO:teuthology.orchestra.run.smithi072.stdout:ceph source packages 4.5 kB/s | 1.5 kB 00:00 2023-10-14T10:32:52.638 INFO:teuthology.orchestra.run.smithi072.stdout:Copr repo for python3-asyncssh owned by ceph 28 kB/s | 3.5 kB 00:00 2023-10-14T10:32:52.738 INFO:teuthology.orchestra.run.smithi067.stdout:Copr repo for python3-asyncssh owned by ceph 11 kB/s | 3.5 kB 00:00 2023-10-14T10:32:53.582 INFO:teuthology.orchestra.run.smithi067.stdout:Extra Packages for Enterprise Linux 19 MB/s | 16 MB 00:00 2023-10-14T10:32:54.226 INFO:teuthology.orchestra.run.smithi072.stdout:Extra Packages for Enterprise Linux 10 MB/s | 16 MB 00:01 2023-10-14T10:32:57.356 INFO:teuthology.orchestra.run.smithi067.stdout:lab-extras 380 kB/s | 24 kB 00:00 2023-10-14T10:32:57.751 INFO:teuthology.orchestra.run.smithi072.stdout:lab-extras 486 kB/s | 24 kB 00:00 2023-10-14T10:32:58.316 INFO:teuthology.orchestra.run.smithi067.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 54 MB/s | 47 MB 00:00 2023-10-14T10:32:58.995 INFO:teuthology.orchestra.run.smithi072.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 45 MB/s | 53 MB 00:01 2023-10-14T10:33:06.900 INFO:teuthology.orchestra.run.smithi067.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 59 MB/s | 53 MB 00:00 2023-10-14T10:33:07.448 INFO:teuthology.orchestra.run.smithi072.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 42 MB/s | 47 MB 00:01 2023-10-14T10:33:18.333 INFO:teuthology.orchestra.run.smithi072.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-14T10:33:18.334 INFO:teuthology.orchestra.run.smithi072.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-14T10:33:18.348 INFO:teuthology.orchestra.run.smithi072.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-10-14T10:33:18.349 INFO:teuthology.orchestra.run.smithi072.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-10-14T10:33:18.351 INFO:teuthology.orchestra.run.smithi072.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-10-14T10:33:18.352 INFO:teuthology.orchestra.run.smithi072.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-10-14T10:33:18.354 INFO:teuthology.orchestra.run.smithi072.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-10-14T10:33:18.357 INFO:teuthology.orchestra.run.smithi072.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-10-14T10:33:18.359 INFO:teuthology.orchestra.run.smithi072.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-10-14T10:33:18.489 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2023-10-14T10:33:18.501 INFO:teuthology.orchestra.run.smithi072.stdout:=========================================================================================================================== 2023-10-14T10:33:18.501 INFO:teuthology.orchestra.run.smithi072.stdout: Package Arch Version Repository Size 2023-10-14T10:33:18.501 INFO:teuthology.orchestra.run.smithi072.stdout:=========================================================================================================================== 2023-10-14T10:33:18.501 INFO:teuthology.orchestra.run.smithi072.stdout:Installing: 2023-10-14T10:33:18.502 INFO:teuthology.orchestra.run.smithi072.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-10-14T10:33:18.502 INFO:teuthology.orchestra.run.smithi072.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-10-14T10:33:18.502 INFO:teuthology.orchestra.run.smithi072.stdout: ceph x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 6.4 k 2023-10-14T10:33:18.502 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-base x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 5.4 M 2023-10-14T10:33:18.502 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-fuse x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 918 k 2023-10-14T10:33:18.503 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 167 k 2023-10-14T10:33:18.503 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 1.6 M 2023-10-14T10:33:18.503 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-cephadm noarch 2:18.0.0-6730.g4ba8a190.el8 ceph-noarch 132 k 2023-10-14T10:33:18.503 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-dashboard noarch 2:18.0.0-6730.g4ba8a190.el8 ceph-noarch 1.8 M 2023-10-14T10:33:18.503 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-6730.g4ba8a190.el8 ceph-noarch 7.4 M 2023-10-14T10:33:18.504 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-rook noarch 2:18.0.0-6730.g4ba8a190.el8 ceph-noarch 47 k 2023-10-14T10:33:18.504 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-radosgw x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 14 M 2023-10-14T10:33:18.504 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-test x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 49 M 2023-10-14T10:33:18.504 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-volume noarch 2:18.0.0-6730.g4ba8a190.el8 ceph-noarch 261 k 2023-10-14T10:33:18.504 INFO:teuthology.orchestra.run.smithi072.stdout: cephadm noarch 2:18.0.0-6730.g4ba8a190.el8 ceph-noarch 192 k 2023-10-14T10:33:18.505 INFO:teuthology.orchestra.run.smithi072.stdout: cephfs-mirror x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 261 k 2023-10-14T10:33:18.505 INFO:teuthology.orchestra.run.smithi072.stdout: cephfs-top noarch 2:18.0.0-6730.g4ba8a190.el8 ceph-noarch 18 k 2023-10-14T10:33:18.505 INFO:teuthology.orchestra.run.smithi072.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-10-14T10:33:18.505 INFO:teuthology.orchestra.run.smithi072.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-10-14T10:33:18.505 INFO:teuthology.orchestra.run.smithi072.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-10-14T10:33:18.506 INFO:teuthology.orchestra.run.smithi072.stdout: libcephfs-devel x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 31 k 2023-10-14T10:33:18.506 INFO:teuthology.orchestra.run.smithi072.stdout: libcephfs2 x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 839 k 2023-10-14T10:33:18.506 INFO:teuthology.orchestra.run.smithi072.stdout: librados-devel x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 139 k 2023-10-14T10:33:18.506 INFO:teuthology.orchestra.run.smithi072.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-10-14T10:33:18.506 INFO:teuthology.orchestra.run.smithi072.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-10-14T10:33:18.507 INFO:teuthology.orchestra.run.smithi072.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-10-14T10:33:18.507 INFO:teuthology.orchestra.run.smithi072.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-10-14T10:33:18.507 INFO:teuthology.orchestra.run.smithi072.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-10-14T10:33:18.507 INFO:teuthology.orchestra.run.smithi072.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-10-14T10:33:18.507 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cephfs x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 197 k 2023-10-14T10:33:18.508 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rados x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 371 k 2023-10-14T10:33:18.508 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rbd x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 365 k 2023-10-14T10:33:18.508 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rgw x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 107 k 2023-10-14T10:33:18.508 INFO:teuthology.orchestra.run.smithi072.stdout: rbd-fuse x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 94 k 2023-10-14T10:33:18.509 INFO:teuthology.orchestra.run.smithi072.stdout: rbd-mirror x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 3.6 M 2023-10-14T10:33:18.509 INFO:teuthology.orchestra.run.smithi072.stdout: rbd-nbd x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 179 k 2023-10-14T10:33:18.509 INFO:teuthology.orchestra.run.smithi072.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-10-14T10:33:18.509 INFO:teuthology.orchestra.run.smithi072.stdout:Upgrading: 2023-10-14T10:33:18.509 INFO:teuthology.orchestra.run.smithi072.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-10-14T10:33:18.509 INFO:teuthology.orchestra.run.smithi072.stdout: librados2 x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 3.7 M 2023-10-14T10:33:18.510 INFO:teuthology.orchestra.run.smithi072.stdout: librbd1 x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 3.7 M 2023-10-14T10:33:18.510 INFO:teuthology.orchestra.run.smithi072.stdout:Installing dependencies: 2023-10-14T10:33:18.510 INFO:teuthology.orchestra.run.smithi072.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-10-14T10:33:18.510 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-common x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 23 M 2023-10-14T10:33:18.511 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-grafana-dashboards noarch 2:18.0.0-6730.g4ba8a190.el8 ceph-noarch 23 k 2023-10-14T10:33:18.511 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mds x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 2.3 M 2023-10-14T10:33:18.511 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-modules-core noarch 2:18.0.0-6730.g4ba8a190.el8 ceph-noarch 242 k 2023-10-14T10:33:18.511 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mon x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 4.6 M 2023-10-14T10:33:18.511 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-osd x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 18 M 2023-10-14T10:33:18.512 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-prometheus-alerts noarch 2:18.0.0-6730.g4ba8a190.el8 ceph-noarch 15 k 2023-10-14T10:33:18.512 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-selinux x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 24 k 2023-10-14T10:33:18.512 INFO:teuthology.orchestra.run.smithi072.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-10-14T10:33:18.512 INFO:teuthology.orchestra.run.smithi072.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-10-14T10:33:18.512 INFO:teuthology.orchestra.run.smithi072.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-10-14T10:33:18.513 INFO:teuthology.orchestra.run.smithi072.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-10-14T10:33:18.513 INFO:teuthology.orchestra.run.smithi072.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-10-14T10:33:18.513 INFO:teuthology.orchestra.run.smithi072.stdout: libcephsqlite x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 184 k 2023-10-14T10:33:18.513 INFO:teuthology.orchestra.run.smithi072.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-10-14T10:33:18.513 INFO:teuthology.orchestra.run.smithi072.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-10-14T10:33:18.514 INFO:teuthology.orchestra.run.smithi072.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-10-14T10:33:18.514 INFO:teuthology.orchestra.run.smithi072.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-10-14T10:33:18.514 INFO:teuthology.orchestra.run.smithi072.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-10-14T10:33:18.514 INFO:teuthology.orchestra.run.smithi072.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-10-14T10:33:18.515 INFO:teuthology.orchestra.run.smithi072.stdout: libradosstriper1 x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 518 k 2023-10-14T10:33:18.515 INFO:teuthology.orchestra.run.smithi072.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-10-14T10:33:18.515 INFO:teuthology.orchestra.run.smithi072.stdout: librgw2 x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 6.9 M 2023-10-14T10:33:18.515 INFO:teuthology.orchestra.run.smithi072.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-10-14T10:33:18.515 INFO:teuthology.orchestra.run.smithi072.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-10-14T10:33:18.516 INFO:teuthology.orchestra.run.smithi072.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-10-14T10:33:18.516 INFO:teuthology.orchestra.run.smithi072.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-10-14T10:33:18.516 INFO:teuthology.orchestra.run.smithi072.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-10-14T10:33:18.516 INFO:teuthology.orchestra.run.smithi072.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-10-14T10:33:18.516 INFO:teuthology.orchestra.run.smithi072.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-10-14T10:33:18.517 INFO:teuthology.orchestra.run.smithi072.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-10-14T10:33:18.517 INFO:teuthology.orchestra.run.smithi072.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-10-14T10:33:18.517 INFO:teuthology.orchestra.run.smithi072.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-10-14T10:33:18.517 INFO:teuthology.orchestra.run.smithi072.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-10-14T10:33:18.517 INFO:teuthology.orchestra.run.smithi072.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-10-14T10:33:18.518 INFO:teuthology.orchestra.run.smithi072.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-10-14T10:33:18.518 INFO:teuthology.orchestra.run.smithi072.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-10-14T10:33:18.518 INFO:teuthology.orchestra.run.smithi072.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-10-14T10:33:18.518 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-10-14T10:33:18.519 INFO:teuthology.orchestra.run.smithi072.stdout: python3-ceph-argparse x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 45 k 2023-10-14T10:33:18.519 INFO:teuthology.orchestra.run.smithi072.stdout: python3-ceph-common x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 126 k 2023-10-14T10:33:18.519 INFO:teuthology.orchestra.run.smithi072.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-10-14T10:33:18.519 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-10-14T10:33:18.519 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-10-14T10:33:18.520 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-10-14T10:33:18.520 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-10-14T10:33:18.520 INFO:teuthology.orchestra.run.smithi072.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-10-14T10:33:18.520 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-10-14T10:33:18.520 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-10-14T10:33:18.521 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-10-14T10:33:18.521 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-10-14T10:33:18.521 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-10-14T10:33:18.521 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-14T10:33:18.521 INFO:teuthology.orchestra.run.smithi072.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-10-14T10:33:18.522 INFO:teuthology.orchestra.run.smithi072.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-10-14T10:33:18.522 INFO:teuthology.orchestra.run.smithi072.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-10-14T10:33:18.522 INFO:teuthology.orchestra.run.smithi072.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-10-14T10:33:18.522 INFO:teuthology.orchestra.run.smithi072.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-10-14T10:33:18.522 INFO:teuthology.orchestra.run.smithi072.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-10-14T10:33:18.523 INFO:teuthology.orchestra.run.smithi072.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-10-14T10:33:18.523 INFO:teuthology.orchestra.run.smithi072.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-10-14T10:33:18.523 INFO:teuthology.orchestra.run.smithi072.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-10-14T10:33:18.523 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-10-14T10:33:18.523 INFO:teuthology.orchestra.run.smithi072.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-10-14T10:33:18.524 INFO:teuthology.orchestra.run.smithi072.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-10-14T10:33:18.524 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-10-14T10:33:18.524 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-10-14T10:33:18.524 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-10-14T10:33:18.524 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-10-14T10:33:18.524 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-10-14T10:33:18.525 INFO:teuthology.orchestra.run.smithi072.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-10-14T10:33:18.525 INFO:teuthology.orchestra.run.smithi072.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-14T10:33:18.525 INFO:teuthology.orchestra.run.smithi072.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-10-14T10:33:18.525 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-10-14T10:33:18.525 INFO:teuthology.orchestra.run.smithi072.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-10-14T10:33:18.526 INFO:teuthology.orchestra.run.smithi072.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-10-14T10:33:18.526 INFO:teuthology.orchestra.run.smithi072.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-10-14T10:33:18.526 INFO:teuthology.orchestra.run.smithi072.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-10-14T10:33:18.526 INFO:teuthology.orchestra.run.smithi072.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-10-14T10:33:18.526 INFO:teuthology.orchestra.run.smithi072.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-10-14T10:33:18.526 INFO:teuthology.orchestra.run.smithi072.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-10-14T10:33:18.527 INFO:teuthology.orchestra.run.smithi072.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-10-14T10:33:18.527 INFO:teuthology.orchestra.run.smithi072.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-10-14T10:33:18.527 INFO:teuthology.orchestra.run.smithi072.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-10-14T10:33:18.527 INFO:teuthology.orchestra.run.smithi072.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-10-14T10:33:18.528 INFO:teuthology.orchestra.run.smithi072.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-10-14T10:33:18.528 INFO:teuthology.orchestra.run.smithi072.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-10-14T10:33:18.528 INFO:teuthology.orchestra.run.smithi072.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-10-14T10:33:18.528 INFO:teuthology.orchestra.run.smithi072.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-10-14T10:33:18.528 INFO:teuthology.orchestra.run.smithi072.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-10-14T10:33:18.529 INFO:teuthology.orchestra.run.smithi072.stdout:Enabling module streams: 2023-10-14T10:33:18.529 INFO:teuthology.orchestra.run.smithi072.stdout: postgresql 10 2023-10-14T10:33:18.529 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-14T10:33:18.529 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction Summary 2023-10-14T10:33:18.529 INFO:teuthology.orchestra.run.smithi072.stdout:=========================================================================================================================== 2023-10-14T10:33:18.529 INFO:teuthology.orchestra.run.smithi072.stdout:Install 128 Packages 2023-10-14T10:33:18.530 INFO:teuthology.orchestra.run.smithi072.stdout:Upgrade 3 Packages 2023-10-14T10:33:18.530 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-14T10:33:18.530 INFO:teuthology.orchestra.run.smithi072.stdout:Total download size: 209 M 2023-10-14T10:33:18.531 INFO:teuthology.orchestra.run.smithi072.stdout:Downloading Packages: 2023-10-14T10:33:18.701 INFO:teuthology.orchestra.run.smithi072.stdout:(1/131): ceph-18.0.0-6730.g4ba8a190.el8.x86_64. 41 kB/s | 6.4 kB 00:00 2023-10-14T10:33:18.891 INFO:teuthology.orchestra.run.smithi067.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-14T10:33:18.892 INFO:teuthology.orchestra.run.smithi067.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-14T10:33:18.893 INFO:teuthology.orchestra.run.smithi072.stdout:(2/131): ceph-fuse-18.0.0-6730.g4ba8a190.el8.x8 4.7 MB/s | 918 kB 00:00 2023-10-14T10:33:18.908 INFO:teuthology.orchestra.run.smithi067.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-10-14T10:33:18.909 INFO:teuthology.orchestra.run.smithi067.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-10-14T10:33:18.910 INFO:teuthology.orchestra.run.smithi067.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-10-14T10:33:18.914 INFO:teuthology.orchestra.run.smithi067.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-10-14T10:33:18.914 INFO:teuthology.orchestra.run.smithi067.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-10-14T10:33:18.917 INFO:teuthology.orchestra.run.smithi067.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-10-14T10:33:18.918 INFO:teuthology.orchestra.run.smithi067.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-10-14T10:33:18.953 INFO:teuthology.orchestra.run.smithi072.stdout:(3/131): ceph-immutable-object-cache-18.0.0-673 2.8 MB/s | 167 kB 00:00 2023-10-14T10:33:19.063 INFO:teuthology.orchestra.run.smithi067.stdout:Dependencies resolved. 2023-10-14T10:33:19.076 INFO:teuthology.orchestra.run.smithi067.stdout:=========================================================================================================================== 2023-10-14T10:33:19.077 INFO:teuthology.orchestra.run.smithi067.stdout: Package Arch Version Repository Size 2023-10-14T10:33:19.077 INFO:teuthology.orchestra.run.smithi067.stdout:=========================================================================================================================== 2023-10-14T10:33:19.077 INFO:teuthology.orchestra.run.smithi067.stdout:Installing: 2023-10-14T10:33:19.077 INFO:teuthology.orchestra.run.smithi067.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-10-14T10:33:19.077 INFO:teuthology.orchestra.run.smithi067.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-10-14T10:33:19.078 INFO:teuthology.orchestra.run.smithi067.stdout: ceph x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 6.4 k 2023-10-14T10:33:19.078 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-base x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 5.4 M 2023-10-14T10:33:19.078 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-fuse x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 918 k 2023-10-14T10:33:19.078 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 167 k 2023-10-14T10:33:19.078 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 1.6 M 2023-10-14T10:33:19.079 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-cephadm noarch 2:18.0.0-6730.g4ba8a190.el8 ceph-noarch 132 k 2023-10-14T10:33:19.079 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-dashboard noarch 2:18.0.0-6730.g4ba8a190.el8 ceph-noarch 1.8 M 2023-10-14T10:33:19.079 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-6730.g4ba8a190.el8 ceph-noarch 7.4 M 2023-10-14T10:33:19.079 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-rook noarch 2:18.0.0-6730.g4ba8a190.el8 ceph-noarch 47 k 2023-10-14T10:33:19.079 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-radosgw x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 14 M 2023-10-14T10:33:19.080 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-test x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 49 M 2023-10-14T10:33:19.080 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-volume noarch 2:18.0.0-6730.g4ba8a190.el8 ceph-noarch 261 k 2023-10-14T10:33:19.080 INFO:teuthology.orchestra.run.smithi067.stdout: cephadm noarch 2:18.0.0-6730.g4ba8a190.el8 ceph-noarch 192 k 2023-10-14T10:33:19.080 INFO:teuthology.orchestra.run.smithi067.stdout: cephfs-mirror x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 261 k 2023-10-14T10:33:19.080 INFO:teuthology.orchestra.run.smithi067.stdout: cephfs-top noarch 2:18.0.0-6730.g4ba8a190.el8 ceph-noarch 18 k 2023-10-14T10:33:19.081 INFO:teuthology.orchestra.run.smithi067.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-10-14T10:33:19.081 INFO:teuthology.orchestra.run.smithi067.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-10-14T10:33:19.081 INFO:teuthology.orchestra.run.smithi067.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-10-14T10:33:19.081 INFO:teuthology.orchestra.run.smithi067.stdout: libcephfs-devel x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 31 k 2023-10-14T10:33:19.081 INFO:teuthology.orchestra.run.smithi067.stdout: libcephfs2 x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 839 k 2023-10-14T10:33:19.082 INFO:teuthology.orchestra.run.smithi067.stdout: librados-devel x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 139 k 2023-10-14T10:33:19.082 INFO:teuthology.orchestra.run.smithi067.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-10-14T10:33:19.082 INFO:teuthology.orchestra.run.smithi067.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-10-14T10:33:19.082 INFO:teuthology.orchestra.run.smithi067.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-10-14T10:33:19.082 INFO:teuthology.orchestra.run.smithi067.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-10-14T10:33:19.083 INFO:teuthology.orchestra.run.smithi067.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-10-14T10:33:19.083 INFO:teuthology.orchestra.run.smithi067.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-10-14T10:33:19.083 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cephfs x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 197 k 2023-10-14T10:33:19.083 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rados x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 371 k 2023-10-14T10:33:19.083 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rbd x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 365 k 2023-10-14T10:33:19.084 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rgw x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 107 k 2023-10-14T10:33:19.084 INFO:teuthology.orchestra.run.smithi067.stdout: rbd-fuse x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 94 k 2023-10-14T10:33:19.084 INFO:teuthology.orchestra.run.smithi067.stdout: rbd-mirror x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 3.6 M 2023-10-14T10:33:19.084 INFO:teuthology.orchestra.run.smithi067.stdout: rbd-nbd x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 179 k 2023-10-14T10:33:19.084 INFO:teuthology.orchestra.run.smithi067.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-10-14T10:33:19.085 INFO:teuthology.orchestra.run.smithi067.stdout:Upgrading: 2023-10-14T10:33:19.085 INFO:teuthology.orchestra.run.smithi067.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-10-14T10:33:19.085 INFO:teuthology.orchestra.run.smithi067.stdout: librados2 x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 3.7 M 2023-10-14T10:33:19.085 INFO:teuthology.orchestra.run.smithi067.stdout: librbd1 x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 3.7 M 2023-10-14T10:33:19.085 INFO:teuthology.orchestra.run.smithi067.stdout:Installing dependencies: 2023-10-14T10:33:19.086 INFO:teuthology.orchestra.run.smithi067.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-10-14T10:33:19.086 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-common x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 23 M 2023-10-14T10:33:19.086 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-grafana-dashboards noarch 2:18.0.0-6730.g4ba8a190.el8 ceph-noarch 23 k 2023-10-14T10:33:19.086 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mds x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 2.3 M 2023-10-14T10:33:19.086 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-modules-core noarch 2:18.0.0-6730.g4ba8a190.el8 ceph-noarch 242 k 2023-10-14T10:33:19.087 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mon x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 4.6 M 2023-10-14T10:33:19.087 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-osd x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 18 M 2023-10-14T10:33:19.087 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-prometheus-alerts noarch 2:18.0.0-6730.g4ba8a190.el8 ceph-noarch 15 k 2023-10-14T10:33:19.087 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-selinux x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 24 k 2023-10-14T10:33:19.087 INFO:teuthology.orchestra.run.smithi067.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-10-14T10:33:19.088 INFO:teuthology.orchestra.run.smithi067.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-10-14T10:33:19.088 INFO:teuthology.orchestra.run.smithi067.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-10-14T10:33:19.088 INFO:teuthology.orchestra.run.smithi067.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-10-14T10:33:19.088 INFO:teuthology.orchestra.run.smithi067.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-10-14T10:33:19.088 INFO:teuthology.orchestra.run.smithi067.stdout: libcephsqlite x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 184 k 2023-10-14T10:33:19.089 INFO:teuthology.orchestra.run.smithi067.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-10-14T10:33:19.089 INFO:teuthology.orchestra.run.smithi067.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-10-14T10:33:19.089 INFO:teuthology.orchestra.run.smithi067.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-10-14T10:33:19.089 INFO:teuthology.orchestra.run.smithi067.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-10-14T10:33:19.089 INFO:teuthology.orchestra.run.smithi067.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-10-14T10:33:19.089 INFO:teuthology.orchestra.run.smithi067.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-10-14T10:33:19.090 INFO:teuthology.orchestra.run.smithi067.stdout: libradosstriper1 x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 518 k 2023-10-14T10:33:19.090 INFO:teuthology.orchestra.run.smithi067.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-10-14T10:33:19.090 INFO:teuthology.orchestra.run.smithi067.stdout: librgw2 x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 6.9 M 2023-10-14T10:33:19.090 INFO:teuthology.orchestra.run.smithi067.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-10-14T10:33:19.090 INFO:teuthology.orchestra.run.smithi067.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-10-14T10:33:19.091 INFO:teuthology.orchestra.run.smithi067.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-10-14T10:33:19.091 INFO:teuthology.orchestra.run.smithi067.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-10-14T10:33:19.091 INFO:teuthology.orchestra.run.smithi067.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-10-14T10:33:19.091 INFO:teuthology.orchestra.run.smithi067.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-10-14T10:33:19.091 INFO:teuthology.orchestra.run.smithi067.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-10-14T10:33:19.092 INFO:teuthology.orchestra.run.smithi067.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-10-14T10:33:19.092 INFO:teuthology.orchestra.run.smithi067.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-10-14T10:33:19.092 INFO:teuthology.orchestra.run.smithi067.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-10-14T10:33:19.092 INFO:teuthology.orchestra.run.smithi067.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-10-14T10:33:19.092 INFO:teuthology.orchestra.run.smithi067.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-10-14T10:33:19.093 INFO:teuthology.orchestra.run.smithi067.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-10-14T10:33:19.093 INFO:teuthology.orchestra.run.smithi067.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-10-14T10:33:19.093 INFO:teuthology.orchestra.run.smithi067.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-10-14T10:33:19.093 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-10-14T10:33:19.093 INFO:teuthology.orchestra.run.smithi067.stdout: python3-ceph-argparse x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 45 k 2023-10-14T10:33:19.094 INFO:teuthology.orchestra.run.smithi067.stdout: python3-ceph-common x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 126 k 2023-10-14T10:33:19.094 INFO:teuthology.orchestra.run.smithi067.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-10-14T10:33:19.094 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-10-14T10:33:19.094 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-10-14T10:33:19.094 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-10-14T10:33:19.095 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-10-14T10:33:19.095 INFO:teuthology.orchestra.run.smithi067.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-10-14T10:33:19.095 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-10-14T10:33:19.095 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-10-14T10:33:19.095 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-10-14T10:33:19.096 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-10-14T10:33:19.096 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-10-14T10:33:19.096 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-14T10:33:19.096 INFO:teuthology.orchestra.run.smithi067.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-10-14T10:33:19.096 INFO:teuthology.orchestra.run.smithi067.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-10-14T10:33:19.096 INFO:teuthology.orchestra.run.smithi067.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-10-14T10:33:19.097 INFO:teuthology.orchestra.run.smithi067.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-10-14T10:33:19.097 INFO:teuthology.orchestra.run.smithi067.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-10-14T10:33:19.097 INFO:teuthology.orchestra.run.smithi067.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-10-14T10:33:19.097 INFO:teuthology.orchestra.run.smithi067.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-10-14T10:33:19.097 INFO:teuthology.orchestra.run.smithi067.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-10-14T10:33:19.098 INFO:teuthology.orchestra.run.smithi067.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-10-14T10:33:19.098 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-10-14T10:33:19.098 INFO:teuthology.orchestra.run.smithi067.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-10-14T10:33:19.098 INFO:teuthology.orchestra.run.smithi067.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-10-14T10:33:19.099 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-10-14T10:33:19.099 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-10-14T10:33:19.099 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-10-14T10:33:19.099 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-10-14T10:33:19.099 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-10-14T10:33:19.100 INFO:teuthology.orchestra.run.smithi067.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-10-14T10:33:19.100 INFO:teuthology.orchestra.run.smithi067.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-14T10:33:19.100 INFO:teuthology.orchestra.run.smithi067.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-10-14T10:33:19.100 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-10-14T10:33:19.100 INFO:teuthology.orchestra.run.smithi067.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-10-14T10:33:19.100 INFO:teuthology.orchestra.run.smithi067.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-10-14T10:33:19.101 INFO:teuthology.orchestra.run.smithi067.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-10-14T10:33:19.101 INFO:teuthology.orchestra.run.smithi067.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-10-14T10:33:19.101 INFO:teuthology.orchestra.run.smithi067.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-10-14T10:33:19.101 INFO:teuthology.orchestra.run.smithi067.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-10-14T10:33:19.101 INFO:teuthology.orchestra.run.smithi067.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-10-14T10:33:19.102 INFO:teuthology.orchestra.run.smithi067.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-10-14T10:33:19.102 INFO:teuthology.orchestra.run.smithi067.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-10-14T10:33:19.102 INFO:teuthology.orchestra.run.smithi067.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-10-14T10:33:19.102 INFO:teuthology.orchestra.run.smithi067.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-10-14T10:33:19.102 INFO:teuthology.orchestra.run.smithi067.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-10-14T10:33:19.103 INFO:teuthology.orchestra.run.smithi067.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-10-14T10:33:19.103 INFO:teuthology.orchestra.run.smithi067.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-10-14T10:33:19.103 INFO:teuthology.orchestra.run.smithi067.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-10-14T10:33:19.103 INFO:teuthology.orchestra.run.smithi067.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-10-14T10:33:19.103 INFO:teuthology.orchestra.run.smithi067.stdout:Enabling module streams: 2023-10-14T10:33:19.104 INFO:teuthology.orchestra.run.smithi067.stdout: postgresql 10 2023-10-14T10:33:19.104 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-14T10:33:19.104 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction Summary 2023-10-14T10:33:19.104 INFO:teuthology.orchestra.run.smithi067.stdout:=========================================================================================================================== 2023-10-14T10:33:19.104 INFO:teuthology.orchestra.run.smithi067.stdout:Install 128 Packages 2023-10-14T10:33:19.105 INFO:teuthology.orchestra.run.smithi067.stdout:Upgrade 3 Packages 2023-10-14T10:33:19.105 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-14T10:33:19.105 INFO:teuthology.orchestra.run.smithi067.stdout:Total download size: 209 M 2023-10-14T10:33:19.106 INFO:teuthology.orchestra.run.smithi067.stdout:Downloading Packages: 2023-10-14T10:33:19.236 INFO:teuthology.orchestra.run.smithi072.stdout:(4/131): ceph-base-18.0.0-6730.g4ba8a190.el8.x8 7.8 MB/s | 5.4 MB 00:00 2023-10-14T10:33:19.295 INFO:teuthology.orchestra.run.smithi072.stdout:(5/131): ceph-mds-18.0.0-6730.g4ba8a190.el8.x86 6.8 MB/s | 2.3 MB 00:00 2023-10-14T10:33:19.376 INFO:teuthology.orchestra.run.smithi067.stdout:(1/131): ceph-18.0.0-6730.g4ba8a190.el8.x86_64. 44 kB/s | 6.4 kB 00:00 2023-10-14T10:33:19.429 INFO:teuthology.orchestra.run.smithi072.stdout:(6/131): ceph-mgr-18.0.0-6730.g4ba8a190.el8.x86 8.4 MB/s | 1.6 MB 00:00 2023-10-14T10:33:19.976 INFO:teuthology.orchestra.run.smithi067.stdout:(2/131): ceph-fuse-18.0.0-6730.g4ba8a190.el8.x8 1.5 MB/s | 918 kB 00:00 2023-10-14T10:33:20.124 INFO:teuthology.orchestra.run.smithi072.stdout:(7/131): ceph-mon-18.0.0-6730.g4ba8a190.el8.x86 5.6 MB/s | 4.6 MB 00:00 2023-10-14T10:33:20.252 INFO:teuthology.orchestra.run.smithi067.stdout:(3/131): ceph-immutable-object-cache-18.0.0-673 606 kB/s | 167 kB 00:00 2023-10-14T10:33:20.502 INFO:teuthology.orchestra.run.smithi067.stdout:(4/131): ceph-base-18.0.0-6730.g4ba8a190.el8.x8 4.3 MB/s | 5.4 MB 00:01 2023-10-14T10:33:20.827 INFO:teuthology.orchestra.run.smithi067.stdout:(5/131): ceph-mgr-18.0.0-6730.g4ba8a190.el8.x86 5.0 MB/s | 1.6 MB 00:00 2023-10-14T10:33:21.127 INFO:teuthology.orchestra.run.smithi067.stdout:(6/131): ceph-mds-18.0.0-6730.g4ba8a190.el8.x86 2.6 MB/s | 2.3 MB 00:00 2023-10-14T10:33:21.428 INFO:teuthology.orchestra.run.smithi067.stdout:(7/131): ceph-mon-18.0.0-6730.g4ba8a190.el8.x86 7.7 MB/s | 4.6 MB 00:00 2023-10-14T10:33:21.856 INFO:teuthology.orchestra.run.smithi072.stdout:(8/131): ceph-osd-18.0.0-6730.g4ba8a190.el8.x86 7.4 MB/s | 18 MB 00:02 2023-10-14T10:33:21.956 INFO:teuthology.orchestra.run.smithi072.stdout:(9/131): ceph-selinux-18.0.0-6730.g4ba8a190.el8 243 kB/s | 24 kB 00:00 2023-10-14T10:33:22.586 INFO:teuthology.orchestra.run.smithi067.stdout:(8/131): ceph-radosgw-18.0.0-6730.g4ba8a190.el8 12 MB/s | 14 MB 00:01 2023-10-14T10:33:22.628 INFO:teuthology.orchestra.run.smithi067.stdout:(9/131): ceph-selinux-18.0.0-6730.g4ba8a190.el8 589 kB/s | 24 kB 00:00 2023-10-14T10:33:22.709 INFO:teuthology.orchestra.run.smithi072.stdout:(10/131): ceph-radosgw-18.0.0-6730.g4ba8a190.el 5.4 MB/s | 14 MB 00:02 2023-10-14T10:33:22.859 INFO:teuthology.orchestra.run.smithi072.stdout:(11/131): cephfs-mirror-18.0.0-6730.g4ba8a190.e 1.7 MB/s | 261 kB 00:00 2023-10-14T10:33:22.943 INFO:teuthology.orchestra.run.smithi072.stdout:(12/131): libcephfs-devel-18.0.0-6730.g4ba8a190 371 kB/s | 31 kB 00:00 2023-10-14T10:33:23.168 INFO:teuthology.orchestra.run.smithi072.stdout:(13/131): libcephfs2-18.0.0-6730.g4ba8a190.el8. 3.6 MB/s | 839 kB 00:00 2023-10-14T10:33:23.261 INFO:teuthology.orchestra.run.smithi072.stdout:(14/131): libcephsqlite-18.0.0-6730.g4ba8a190.e 2.0 MB/s | 184 kB 00:00 2023-10-14T10:33:23.345 INFO:teuthology.orchestra.run.smithi072.stdout:(15/131): librados-devel-18.0.0-6730.g4ba8a190. 1.6 MB/s | 139 kB 00:00 2023-10-14T10:33:23.787 INFO:teuthology.orchestra.run.smithi072.stdout:(16/131): ceph-common-18.0.0-6730.g4ba8a190.el8 4.3 MB/s | 23 MB 00:05 2023-10-14T10:33:23.821 INFO:teuthology.orchestra.run.smithi072.stdout:(17/131): libradosstriper1-18.0.0-6730.g4ba8a19 1.1 MB/s | 518 kB 00:00 2023-10-14T10:33:23.905 INFO:teuthology.orchestra.run.smithi072.stdout:(18/131): python3-ceph-argparse-18.0.0-6730.g4b 535 kB/s | 45 kB 00:00 2023-10-14T10:33:23.988 INFO:teuthology.orchestra.run.smithi072.stdout:(19/131): python3-ceph-common-18.0.0-6730.g4ba8 1.5 MB/s | 126 kB 00:00 2023-10-14T10:33:24.081 INFO:teuthology.orchestra.run.smithi072.stdout:(20/131): python3-cephfs-18.0.0-6730.g4ba8a190. 2.1 MB/s | 197 kB 00:00 2023-10-14T10:33:24.240 INFO:teuthology.orchestra.run.smithi072.stdout:(21/131): python3-rados-18.0.0-6730.g4ba8a190.e 2.3 MB/s | 371 kB 00:00 2023-10-14T10:33:24.407 INFO:teuthology.orchestra.run.smithi072.stdout:(22/131): python3-rbd-18.0.0-6730.g4ba8a190.el8 2.1 MB/s | 365 kB 00:00 2023-10-14T10:33:24.491 INFO:teuthology.orchestra.run.smithi072.stdout:(23/131): python3-rgw-18.0.0-6730.g4ba8a190.el8 1.2 MB/s | 107 kB 00:00 2023-10-14T10:33:24.584 INFO:teuthology.orchestra.run.smithi072.stdout:(24/131): rbd-fuse-18.0.0-6730.g4ba8a190.el8.x8 1.0 MB/s | 94 kB 00:00 2023-10-14T10:33:25.435 INFO:teuthology.orchestra.run.smithi072.stdout:(25/131): rbd-mirror-18.0.0-6730.g4ba8a190.el8. 4.2 MB/s | 3.6 MB 00:00 2023-10-14T10:33:25.527 INFO:teuthology.orchestra.run.smithi072.stdout:(26/131): rbd-nbd-18.0.0-6730.g4ba8a190.el8.x86 1.9 MB/s | 179 kB 00:00 2023-10-14T10:33:25.603 INFO:teuthology.orchestra.run.smithi072.stdout:(27/131): ceph-grafana-dashboards-18.0.0-6730.g 306 kB/s | 23 kB 00:00 2023-10-14T10:33:25.828 INFO:teuthology.orchestra.run.smithi072.stdout:(28/131): ceph-mgr-cephadm-18.0.0-6730.g4ba8a19 588 kB/s | 132 kB 00:00 2023-10-14T10:33:25.988 INFO:teuthology.orchestra.run.smithi072.stdout:(29/131): librgw2-18.0.0-6730.g4ba8a190.el8.x86 3.1 MB/s | 6.9 MB 00:02 2023-10-14T10:33:26.062 INFO:teuthology.orchestra.run.smithi067.stdout:(10/131): ceph-test-18.0.0-6730.g4ba8a190.el8.x 14 MB/s | 49 MB 00:03 2023-10-14T10:33:26.063 INFO:teuthology.orchestra.run.smithi072.stdout:(30/131): ceph-mgr-dashboard-18.0.0-6730.g4ba8a 7.5 MB/s | 1.8 MB 00:00 2023-10-14T10:33:26.120 INFO:teuthology.orchestra.run.smithi067.stdout:(11/131): cephfs-mirror-18.0.0-6730.g4ba8a190.e 4.4 MB/s | 261 kB 00:00 2023-10-14T10:33:26.139 INFO:teuthology.orchestra.run.smithi072.stdout:(31/131): ceph-mgr-modules-core-18.0.0-6730.g4b 3.2 MB/s | 242 kB 00:00 2023-10-14T10:33:26.162 INFO:teuthology.orchestra.run.smithi067.stdout:(12/131): libcephfs-devel-18.0.0-6730.g4ba8a190 749 kB/s | 31 kB 00:00 2023-10-14T10:33:26.190 INFO:teuthology.orchestra.run.smithi072.stdout:(32/131): ceph-mgr-rook-18.0.0-6730.g4ba8a190.e 946 kB/s | 47 kB 00:00 2023-10-14T10:33:26.257 INFO:teuthology.orchestra.run.smithi072.stdout:(33/131): ceph-prometheus-alerts-18.0.0-6730.g4 224 kB/s | 15 kB 00:00 2023-10-14T10:33:26.316 INFO:teuthology.orchestra.run.smithi072.stdout:(34/131): ceph-volume-18.0.0-6730.g4ba8a190.el8 4.4 MB/s | 261 kB 00:00 2023-10-14T10:33:26.383 INFO:teuthology.orchestra.run.smithi072.stdout:(35/131): cephadm-18.0.0-6730.g4ba8a190.el8.noa 2.8 MB/s | 192 kB 00:00 2023-10-14T10:33:26.387 INFO:teuthology.orchestra.run.smithi067.stdout:(13/131): ceph-osd-18.0.0-6730.g4ba8a190.el8.x8 3.4 MB/s | 18 MB 00:05 2023-10-14T10:33:26.421 INFO:teuthology.orchestra.run.smithi067.stdout:(14/131): libcephfs2-18.0.0-6730.g4ba8a190.el8. 3.2 MB/s | 839 kB 00:00 2023-10-14T10:33:26.433 INFO:teuthology.orchestra.run.smithi072.stdout:(36/131): cephfs-top-18.0.0-6730.g4ba8a190.el8. 361 kB/s | 18 kB 00:00 2023-10-14T10:33:26.474 INFO:teuthology.orchestra.run.smithi067.stdout:(15/131): libcephsqlite-18.0.0-6730.g4ba8a190.e 2.1 MB/s | 184 kB 00:00 2023-10-14T10:33:26.496 INFO:teuthology.orchestra.run.smithi067.stdout:(16/131): librados-devel-18.0.0-6730.g4ba8a190. 1.8 MB/s | 139 kB 00:00 2023-10-14T10:33:26.630 INFO:teuthology.orchestra.run.smithi067.stdout:(17/131): libradosstriper1-18.0.0-6730.g4ba8a19 3.3 MB/s | 518 kB 00:00 2023-10-14T10:33:26.697 INFO:teuthology.orchestra.run.smithi067.stdout:(18/131): python3-ceph-argparse-18.0.0-6730.g4b 670 kB/s | 45 kB 00:00 2023-10-14T10:33:26.773 INFO:teuthology.orchestra.run.smithi067.stdout:(19/131): python3-ceph-common-18.0.0-6730.g4ba8 1.6 MB/s | 126 kB 00:00 2023-10-14T10:33:26.856 INFO:teuthology.orchestra.run.smithi067.stdout:(20/131): python3-cephfs-18.0.0-6730.g4ba8a190. 2.3 MB/s | 197 kB 00:00 2023-10-14T10:33:27.040 INFO:teuthology.orchestra.run.smithi067.stdout:(21/131): librgw2-18.0.0-6730.g4ba8a190.el8.x86 13 MB/s | 6.9 MB 00:00 2023-10-14T10:33:27.065 INFO:teuthology.orchestra.run.smithi067.stdout:(22/131): python3-rados-18.0.0-6730.g4ba8a190.e 1.7 MB/s | 371 kB 00:00 2023-10-14T10:33:27.090 INFO:teuthology.orchestra.run.smithi067.stdout:(23/131): python3-rbd-18.0.0-6730.g4ba8a190.el8 7.1 MB/s | 365 kB 00:00 2023-10-14T10:33:27.116 INFO:teuthology.orchestra.run.smithi067.stdout:(24/131): python3-rgw-18.0.0-6730.g4ba8a190.el8 2.1 MB/s | 107 kB 00:00 2023-10-14T10:33:27.141 INFO:teuthology.orchestra.run.smithi067.stdout:(25/131): rbd-fuse-18.0.0-6730.g4ba8a190.el8.x8 1.8 MB/s | 94 kB 00:00 2023-10-14T10:33:27.200 INFO:teuthology.orchestra.run.smithi067.stdout:(26/131): rbd-nbd-18.0.0-6730.g4ba8a190.el8.x86 3.0 MB/s | 179 kB 00:00 2023-10-14T10:33:27.209 INFO:teuthology.orchestra.run.smithi072.stdout:(37/131): gperftools-libs-2.7-9.el8.x86_64.rpm 395 kB/s | 306 kB 00:00 2023-10-14T10:33:27.250 INFO:teuthology.orchestra.run.smithi067.stdout:(27/131): ceph-grafana-dashboards-18.0.0-6730.g 463 kB/s | 23 kB 00:00 2023-10-14T10:33:27.308 INFO:teuthology.orchestra.run.smithi067.stdout:(28/131): ceph-mgr-cephadm-18.0.0-6730.g4ba8a19 2.2 MB/s | 132 kB 00:00 2023-10-14T10:33:27.344 INFO:teuthology.orchestra.run.smithi072.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 436 kB/s | 59 kB 00:00 2023-10-14T10:33:27.479 INFO:teuthology.orchestra.run.smithi072.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 562 kB/s | 75 kB 00:00 2023-10-14T10:33:27.492 INFO:teuthology.orchestra.run.smithi067.stdout:(29/131): ceph-mgr-dashboard-18.0.0-6730.g4ba8a 9.6 MB/s | 1.8 MB 00:00 2023-10-14T10:33:27.788 INFO:teuthology.orchestra.run.smithi072.stdout:(40/131): ceph-mgr-diskprediction-local-18.0.0- 4.1 MB/s | 7.4 MB 00:01 2023-10-14T10:33:27.817 INFO:teuthology.orchestra.run.smithi067.stdout:(30/131): rbd-mirror-18.0.0-6730.g4ba8a190.el8. 5.1 MB/s | 3.6 MB 00:00 2023-10-14T10:33:27.821 INFO:teuthology.orchestra.run.smithi072.stdout:(41/131): python3-asyncssh-2.7.0-2.el8.noarch.r 1.4 MB/s | 476 kB 00:00 2023-10-14T10:33:27.926 INFO:teuthology.orchestra.run.smithi067.stdout:(31/131): ceph-mgr-modules-core-18.0.0-6730.g4b 2.2 MB/s | 242 kB 00:00 2023-10-14T10:33:27.964 INFO:teuthology.orchestra.run.smithi072.stdout:(42/131): python3-beautifulsoup4-4.6.3-2.el8.1. 1.3 MB/s | 185 kB 00:00 2023-10-14T10:33:28.010 INFO:teuthology.orchestra.run.smithi067.stdout:(32/131): ceph-mgr-diskprediction-local-18.0.0- 14 MB/s | 7.4 MB 00:00 2023-10-14T10:33:28.026 INFO:teuthology.orchestra.run.smithi067.stdout:(33/131): ceph-mgr-rook-18.0.0-6730.g4ba8a190.e 473 kB/s | 47 kB 00:00 2023-10-14T10:33:28.043 INFO:teuthology.orchestra.run.smithi067.stdout:(34/131): ceph-prometheus-alerts-18.0.0-6730.g4 449 kB/s | 15 kB 00:00 2023-10-14T10:33:28.089 INFO:teuthology.orchestra.run.smithi072.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 263 kB/s | 33 kB 00:00 2023-10-14T10:33:28.094 INFO:teuthology.orchestra.run.smithi067.stdout:(35/131): cephadm-18.0.0-6730.g4ba8a190.el8.noa 3.8 MB/s | 192 kB 00:00 2023-10-14T10:33:28.119 INFO:teuthology.orchestra.run.smithi067.stdout:(36/131): ceph-volume-18.0.0-6730.g4ba8a190.el8 2.8 MB/s | 261 kB 00:00 2023-10-14T10:33:28.136 INFO:teuthology.orchestra.run.smithi067.stdout:(37/131): cephfs-top-18.0.0-6730.g4ba8a190.el8. 434 kB/s | 18 kB 00:00 2023-10-14T10:33:28.215 INFO:teuthology.orchestra.run.smithi072.stdout:(44/131): python3-certifi-2018.10.15-7.el8.noar 130 kB/s | 16 kB 00:00 2023-10-14T10:33:28.244 INFO:teuthology.orchestra.run.smithi067.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 543 kB/s | 59 kB 00:00 2023-10-14T10:33:28.282 INFO:teuthology.orchestra.run.smithi072.stdout:(45/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 88 kB/s | 44 kB 00:00 2023-10-14T10:33:28.295 INFO:teuthology.orchestra.run.smithi067.stdout:(39/131): gperftools-libs-2.7-9.el8.x86_64.rpm 1.7 MB/s | 306 kB 00:00 2023-10-14T10:33:28.328 INFO:teuthology.orchestra.run.smithi067.stdout:(40/131): libunwind-1.3.1-3.el8.x86_64.rpm 898 kB/s | 75 kB 00:00 2023-10-14T10:33:28.349 INFO:teuthology.orchestra.run.smithi072.stdout:(46/131): python3-cheroot-8.5.2-1.el8.noarch.rp 1.3 MB/s | 173 kB 00:00 2023-10-14T10:33:28.370 INFO:teuthology.orchestra.run.smithi067.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 1.0 MB/s | 44 kB 00:00 2023-10-14T10:33:28.404 INFO:teuthology.orchestra.run.smithi067.stdout:(42/131): python3-asyncssh-2.7.0-2.el8.noarch.r 4.3 MB/s | 476 kB 00:00 2023-10-14T10:33:28.429 INFO:teuthology.orchestra.run.smithi067.stdout:(43/131): python3-beautifulsoup4-4.6.3-2.el8.1. 3.1 MB/s | 185 kB 00:00 2023-10-14T10:33:28.454 INFO:teuthology.orchestra.run.smithi067.stdout:(44/131): python3-cachetools-3.1.1-4.el8.noarch 655 kB/s | 33 kB 00:00 2023-10-14T10:33:28.475 INFO:teuthology.orchestra.run.smithi072.stdout:(47/131): python3-google-auth-1.1.1-10.el8.noar 648 kB/s | 81 kB 00:00 2023-10-14T10:33:28.480 INFO:teuthology.orchestra.run.smithi067.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 325 kB/s | 16 kB 00:00 2023-10-14T10:33:28.530 INFO:teuthology.orchestra.run.smithi067.stdout:(46/131): python3-cherrypy-18.4.0-1.el8.noarch. 7.5 MB/s | 384 kB 00:00 2023-10-14T10:33:28.556 INFO:teuthology.orchestra.run.smithi067.stdout:(47/131): python3-cheroot-8.5.2-1.el8.noarch.rp 1.7 MB/s | 173 kB 00:00 2023-10-14T10:33:28.581 INFO:teuthology.orchestra.run.smithi067.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 1.6 MB/s | 81 kB 00:00 2023-10-14T10:33:28.600 INFO:teuthology.orchestra.run.smithi072.stdout:(48/131): python3-jaraco-6.2-6.el8.noarch.rpm 89 kB/s | 11 kB 00:00 2023-10-14T10:33:28.606 INFO:teuthology.orchestra.run.smithi067.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 221 kB/s | 11 kB 00:00 2023-10-14T10:33:28.656 INFO:teuthology.orchestra.run.smithi067.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 234 kB/s | 18 kB 00:00 2023-10-14T10:33:28.698 INFO:teuthology.orchestra.run.smithi067.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 1.1 MB/s | 49 kB 00:00 2023-10-14T10:33:28.717 INFO:teuthology.orchestra.run.smithi072.stdout:(49/131): python3-jaraco-functools-2.0-4.el8.no 150 kB/s | 18 kB 00:00 2023-10-14T10:33:28.733 INFO:teuthology.orchestra.run.smithi067.stdout:(52/131): python3-kubernetes-11.0.0-6.el8.noarc 9.3 MB/s | 1.2 MB 00:00 2023-10-14T10:33:28.751 INFO:teuthology.orchestra.run.smithi072.stdout:(50/131): python3-cherrypy-18.4.0-1.el8.noarch. 820 kB/s | 384 kB 00:00 2023-10-14T10:33:28.758 INFO:teuthology.orchestra.run.smithi067.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 1.0 MB/s | 59 kB 00:00 2023-10-14T10:33:28.791 INFO:teuthology.orchestra.run.smithi067.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 1.0 MB/s | 60 kB 00:00 2023-10-14T10:33:28.816 INFO:teuthology.orchestra.run.smithi067.stdout:(55/131): python3-pecan-1.3.2-9.el8.noarch.rpm 4.7 MB/s | 283 kB 00:00 2023-10-14T10:33:28.842 INFO:teuthology.orchestra.run.smithi067.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 310 kB/s | 16 kB 00:00 2023-10-14T10:33:28.867 INFO:teuthology.orchestra.run.smithi067.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 670 kB/s | 34 kB 00:00 2023-10-14T10:33:28.877 INFO:teuthology.orchestra.run.smithi072.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 389 kB/s | 49 kB 00:00 2023-10-14T10:33:28.892 INFO:teuthology.orchestra.run.smithi067.stdout:(58/131): python3-routes-2.4.1-12.el8.noarch.rp 3.8 MB/s | 196 kB 00:00 2023-10-14T10:33:28.917 INFO:teuthology.orchestra.run.smithi067.stdout:(59/131): python3-rsa-4.9-2.el8.noarch.rpm 1.3 MB/s | 64 kB 00:00 2023-10-14T10:33:28.943 INFO:teuthology.orchestra.run.smithi067.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 372 kB/s | 19 kB 00:00 2023-10-14T10:33:28.954 INFO:teuthology.orchestra.run.smithi072.stdout:(52/131): python3-kubernetes-11.0.0-6.el8.noarc 4.9 MB/s | 1.2 MB 00:00 2023-10-14T10:33:28.968 INFO:teuthology.orchestra.run.smithi067.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 477 kB/s | 24 kB 00:00 2023-10-14T10:33:28.994 INFO:teuthology.orchestra.run.smithi067.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 584 kB/s | 29 kB 00:00 2023-10-14T10:33:29.013 INFO:teuthology.orchestra.run.smithi072.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 432 kB/s | 59 kB 00:00 2023-10-14T10:33:29.018 INFO:teuthology.orchestra.run.smithi067.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 549 kB/s | 27 kB 00:00 2023-10-14T10:33:29.075 INFO:teuthology.orchestra.run.smithi067.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 3.1 MB/s | 255 kB 00:00 2023-10-14T10:33:29.080 INFO:teuthology.orchestra.run.smithi072.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 479 kB/s | 60 kB 00:00 2023-10-14T10:33:29.111 INFO:teuthology.orchestra.run.smithi067.stdout:(65/131): python3-webob-1.8.5-1.el8.1.noarch.rp 2.7 MB/s | 251 kB 00:00 2023-10-14T10:33:29.136 INFO:teuthology.orchestra.run.smithi067.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 1.0 MB/s | 61 kB 00:00 2023-10-14T10:33:29.162 INFO:teuthology.orchestra.run.smithi067.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 1.7 MB/s | 85 kB 00:00 2023-10-14T10:33:29.187 INFO:teuthology.orchestra.run.smithi067.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 456 kB/s | 23 kB 00:00 2023-10-14T10:33:29.189 INFO:teuthology.orchestra.run.smithi072.stdout:(55/131): python3-pecan-1.3.2-9.el8.noarch.rpm 1.6 MB/s | 283 kB 00:00 2023-10-14T10:33:29.212 INFO:teuthology.orchestra.run.smithi067.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.8 MB/s | 70 kB 00:00 2023-10-14T10:33:29.214 INFO:teuthology.orchestra.run.smithi072.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 116 kB/s | 16 kB 00:00 2023-10-14T10:33:29.279 INFO:teuthology.orchestra.run.smithi067.stdout:(70/131): thrift-0.13.0-2.el8.x86_64.rpm 15 MB/s | 1.7 MB 00:00 2023-10-14T10:33:29.346 INFO:teuthology.orchestra.run.smithi067.stdout:(71/131): python3-pyasn1-modules-0.3.7-6.el8.no 1.6 MB/s | 110 kB 00:00 2023-10-14T10:33:29.357 INFO:teuthology.orchestra.run.smithi072.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 201 kB/s | 34 kB 00:00 2023-10-14T10:33:29.382 INFO:teuthology.orchestra.run.smithi072.stdout:(58/131): python3-routes-2.4.1-12.el8.noarch.rp 1.1 MB/s | 196 kB 00:00 2023-10-14T10:33:29.396 INFO:teuthology.orchestra.run.smithi067.stdout:(72/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.1 MB/s | 54 kB 00:00 2023-10-14T10:33:29.446 INFO:teuthology.orchestra.run.smithi067.stdout:(73/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 5.1 MB/s | 259 kB 00:00 2023-10-14T10:33:29.483 INFO:teuthology.orchestra.run.smithi072.stdout:(59/131): python3-rsa-4.9-2.el8.noarch.rpm 511 kB/s | 64 kB 00:00 2023-10-14T10:33:29.509 INFO:teuthology.orchestra.run.smithi072.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 149 kB/s | 19 kB 00:00 2023-10-14T10:33:29.522 INFO:teuthology.orchestra.run.smithi067.stdout:(74/131): socat-1.7.4.1-1.el8.x86_64.rpm 4.2 MB/s | 323 kB 00:00 2023-10-14T10:33:29.564 INFO:teuthology.orchestra.run.smithi067.stdout:(75/131): python3-markupsafe-0.23-19.el8.x86_64 933 kB/s | 39 kB 00:00 2023-10-14T10:33:29.609 INFO:teuthology.orchestra.run.smithi072.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 190 kB/s | 24 kB 00:00 2023-10-14T10:33:29.635 INFO:teuthology.orchestra.run.smithi072.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 232 kB/s | 29 kB 00:00 2023-10-14T10:33:29.714 INFO:teuthology.orchestra.run.smithi067.stdout:(76/131): python3-scipy-1.0.0-21.module+el8.5.0 27 MB/s | 14 MB 00:00 2023-10-14T10:33:29.735 INFO:teuthology.orchestra.run.smithi072.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 219 kB/s | 27 kB 00:00 2023-10-14T10:33:29.748 INFO:teuthology.orchestra.run.smithi067.stdout:(77/131): bison-3.0.4-10.el8.x86_64.rpm 3.7 MB/s | 688 kB 00:00 2023-10-14T10:33:29.782 INFO:teuthology.orchestra.run.smithi067.stdout:(78/131): python3-jinja2-2.10.1-3.el8.noarch.rp 7.9 MB/s | 538 kB 00:00 2023-10-14T10:33:29.798 INFO:teuthology.orchestra.run.smithi067.stdout:(79/131): python3-mako-1.0.6-13.el8.noarch.rpm 3.1 MB/s | 157 kB 00:00 2023-10-14T10:33:29.811 INFO:teuthology.orchestra.run.smithi072.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 1.4 MB/s | 255 kB 00:00 2023-10-14T10:33:29.832 INFO:teuthology.orchestra.run.smithi067.stdout:(80/131): oniguruma-6.8.2-2.el8.x86_64.rpm 3.6 MB/s | 187 kB 00:00 2023-10-14T10:33:29.874 INFO:teuthology.orchestra.run.smithi067.stdout:(81/131): automake-1.16.1-7.el8.noarch.rpm 9.3 MB/s | 713 kB 00:00 2023-10-14T10:33:29.878 INFO:teuthology.orchestra.run.smithi072.stdout:(65/131): python3-webob-1.8.5-1.el8.1.noarch.rp 1.7 MB/s | 251 kB 00:00 2023-10-14T10:33:29.949 INFO:teuthology.orchestra.run.smithi067.stdout:(82/131): python3-jsonpatch-1.21-2.el8.noarch.r 355 kB/s | 27 kB 00:00 2023-10-14T10:33:30.041 INFO:teuthology.orchestra.run.smithi067.stdout:(83/131): python3-babel-2.5.1-7.el8.noarch.rpm 23 MB/s | 4.8 MB 00:00 2023-10-14T10:33:30.067 INFO:teuthology.orchestra.run.smithi067.stdout:(84/131): flex-2.6.1-9.el8.x86_64.rpm 2.7 MB/s | 320 kB 00:00 2023-10-14T10:33:30.092 INFO:teuthology.orchestra.run.smithi067.stdout:(85/131): python3-prettytable-0.7.2-14.el8.noar 873 kB/s | 44 kB 00:00 2023-10-14T10:33:30.142 INFO:teuthology.orchestra.run.smithi067.stdout:(86/131): python3-werkzeug-0.12.2-4.el8.noarch. 9.0 MB/s | 457 kB 00:00 2023-10-14T10:33:30.209 INFO:teuthology.orchestra.run.smithi067.stdout:(87/131): python3-numpy-1.14.3-10.el8.x86_64.rp 26 MB/s | 3.7 MB 00:00 2023-10-14T10:33:30.234 INFO:teuthology.orchestra.run.smithi067.stdout:(88/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 2.4 MB/s | 225 kB 00:00 2023-10-14T10:33:30.260 INFO:teuthology.orchestra.run.smithi067.stdout:(89/131): uuid-1.6.2-43.el8.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2023-10-14T10:33:30.543 INFO:teuthology.orchestra.run.smithi067.stdout:(90/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 409 kB/s | 126 kB 00:00 2023-10-14T10:33:30.569 INFO:teuthology.orchestra.run.smithi067.stdout:(91/131): python3-jsonpointer-1.10-11.el8.noarc 65 kB/s | 20 kB 00:00 2023-10-14T10:33:30.594 INFO:teuthology.orchestra.run.smithi067.stdout:(92/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 2.0 MB/s | 103 kB 00:00 2023-10-14T10:33:30.636 INFO:teuthology.orchestra.run.smithi067.stdout:(93/131): libtool-2.4.6-25.el8.x86_64.rpm 10 MB/s | 709 kB 00:00 2023-10-14T10:33:30.654 INFO:teuthology.orchestra.run.smithi072.stdout:(66/131): ceph-test-18.0.0-6730.g4ba8a190.el8.x 5.6 MB/s | 49 MB 00:08 2023-10-14T10:33:30.686 INFO:teuthology.orchestra.run.smithi067.stdout:(94/131): libpq-13.5-1.el8.x86_64.rpm 3.9 MB/s | 198 kB 00:00 2023-10-14T10:33:30.713 INFO:teuthology.orchestra.run.smithi072.stdout:(67/131): python3-websocket-client-0.56.0-5.el8 68 kB/s | 61 kB 00:00 2023-10-14T10:33:30.787 INFO:teuthology.orchestra.run.smithi067.stdout:(95/131): openblas-threads-0.3.15-3.el8.x86_64. 25 MB/s | 4.9 MB 00:00 2023-10-14T10:33:30.845 INFO:teuthology.orchestra.run.smithi067.stdout:(96/131): jq-1.6-3.el8.x86_64.rpm 3.4 MB/s | 202 kB 00:00 2023-10-14T10:33:30.887 INFO:teuthology.orchestra.run.smithi072.stdout:(68/131): python3-webtest-2.0.33-1.el8.noarch.r 85 kB/s | 85 kB 00:01 2023-10-14T10:33:30.920 INFO:teuthology.orchestra.run.smithi067.stdout:(97/131): openblas-0.3.15-3.el8.x86_64.rpm 20 MB/s | 4.7 MB 00:00 2023-10-14T10:33:30.954 INFO:teuthology.orchestra.run.smithi067.stdout:(98/131): librdkafka-0.11.4-3.el8.x86_64.rpm 3.2 MB/s | 354 kB 00:00 2023-10-14T10:33:31.013 INFO:teuthology.orchestra.run.smithi067.stdout:(99/131): postgresql-contrib-10.21-2.module+el8 14 MB/s | 808 kB 00:00 2023-10-14T10:33:31.082 INFO:teuthology.orchestra.run.smithi072.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 365 kB/s | 70 kB 00:00 2023-10-14T10:33:31.105 INFO:teuthology.orchestra.run.smithi067.stdout:(100/131): postgresql-server-10.21-2.module+el8 28 MB/s | 5.1 MB 00:00 2023-10-14T10:33:31.132 INFO:teuthology.orchestra.run.smithi072.stdout:(70/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 947 kB/s | 47 kB 00:00 2023-10-14T10:33:31.138 INFO:teuthology.orchestra.run.smithi067.stdout:(101/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2023-10-14T10:33:31.183 INFO:teuthology.orchestra.run.smithi072.stdout:(71/131): thrift-0.13.0-2.el8.x86_64.rpm 3.7 MB/s | 1.7 MB 00:00 2023-10-14T10:33:31.208 INFO:teuthology.orchestra.run.smithi072.stdout:(72/131): libselinux-devel-2.9-5.el8.x86_64.rpm 2.6 MB/s | 200 kB 00:00 2023-10-14T10:33:31.251 INFO:teuthology.orchestra.run.smithi072.stdout:(73/131): python3-jwt-1.6.1-2.el8.noarch.rpm 633 kB/s | 43 kB 00:00 2023-10-14T10:33:31.285 INFO:teuthology.orchestra.run.smithi072.stdout:(74/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 3.1 MB/s | 220 kB 00:00 2023-10-14T10:33:31.310 INFO:teuthology.orchestra.run.smithi072.stdout:(75/131): python3-zc-lockfile-2.0-2.el8.noarch. 35 kB/s | 23 kB 00:00 2023-10-14T10:33:31.336 INFO:teuthology.orchestra.run.smithi072.stdout:(76/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 570 kB/s | 48 kB 00:00 2023-10-14T10:33:31.361 INFO:teuthology.orchestra.run.smithi072.stdout:(77/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.1 MB/s | 87 kB 00:00 2023-10-14T10:33:31.381 INFO:teuthology.orchestra.run.smithi067.stdout:(102/131): ceph-common-18.0.0-6730.g4ba8a190.el 1.9 MB/s | 23 MB 00:12 2023-10-14T10:33:31.420 INFO:teuthology.orchestra.run.smithi072.stdout:(78/131): python3-requests-oauthlib-1.0.0-1.el8 391 kB/s | 43 kB 00:00 2023-10-14T10:33:31.422 INFO:teuthology.orchestra.run.smithi067.stdout:(103/131): postgresql-10.21-2.module+el8.6.0+15 3.7 MB/s | 1.5 MB 00:00 2023-10-14T10:33:31.445 INFO:teuthology.orchestra.run.smithi072.stdout:(79/131): python3-pycparser-2.14-14.el8.noarch. 1.0 MB/s | 109 kB 00:00 2023-10-14T10:33:31.456 INFO:teuthology.orchestra.run.smithi067.stdout:(104/131): libselinux-devel-2.9-5.el8.x86_64.rp 630 kB/s | 200 kB 00:00 2023-10-14T10:33:31.473 INFO:teuthology.orchestra.run.smithi067.stdout:(105/131): python3-jwt-1.6.1-2.el8.noarch.rpm 462 kB/s | 43 kB 00:00 2023-10-14T10:33:31.479 INFO:teuthology.orchestra.run.smithi072.stdout:(80/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.9 MB/s | 229 kB 00:00 2023-10-14T10:33:31.490 INFO:teuthology.orchestra.run.smithi067.stdout:(106/131): keyutils-libs-devel-1.5.10-9.el8.x86 1.4 MB/s | 48 kB 00:00 2023-10-14T10:33:31.505 INFO:teuthology.orchestra.run.smithi072.stdout:(81/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 316 kB/s | 19 kB 00:00 2023-10-14T10:33:31.507 INFO:teuthology.orchestra.run.smithi067.stdout:(107/131): libsepol-devel-2.9-3.el8.x86_64.rpm 2.5 MB/s | 87 kB 00:00 2023-10-14T10:33:31.532 INFO:teuthology.orchestra.run.smithi067.stdout:(108/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 2.1 MB/s | 220 kB 00:00 2023-10-14T10:33:31.532 INFO:teuthology.orchestra.run.smithi072.stdout:(82/131): python3-oauthlib-2.1.0-1.el8.noarch.r 2.9 MB/s | 155 kB 00:00 2023-10-14T10:33:31.549 INFO:teuthology.orchestra.run.smithi067.stdout:(109/131): python3-requests-oauthlib-1.0.0-1.el 727 kB/s | 43 kB 00:00 2023-10-14T10:33:31.566 INFO:teuthology.orchestra.run.smithi072.stdout:(83/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 3.7 MB/s | 560 kB 00:00 2023-10-14T10:33:31.574 INFO:teuthology.orchestra.run.smithi067.stdout:(110/131): python3-pycparser-2.14-14.el8.noarch 1.6 MB/s | 109 kB 00:00 2023-10-14T10:33:31.583 INFO:teuthology.orchestra.run.smithi072.stdout:(84/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 353 kB/s | 18 kB 00:00 2023-10-14T10:33:31.601 INFO:teuthology.orchestra.run.smithi067.stdout:(111/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 3.3 MB/s | 229 kB 00:00 2023-10-14T10:33:31.617 INFO:teuthology.orchestra.run.smithi067.stdout:(112/131): libaio-devel-0.3.112-1.el8.x86_64.rp 448 kB/s | 19 kB 00:00 2023-10-14T10:33:31.617 INFO:teuthology.orchestra.run.smithi072.stdout:(85/131): python3-cryptography-3.2.1-5.el8.x86_ 4.9 MB/s | 559 kB 00:00 2023-10-14T10:33:31.642 INFO:teuthology.orchestra.run.smithi067.stdout:(113/131): python3-oauthlib-2.1.0-1.el8.noarch. 3.6 MB/s | 155 kB 00:00 2023-10-14T10:33:31.651 INFO:teuthology.orchestra.run.smithi072.stdout:(86/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 3.5 MB/s | 238 kB 00:00 2023-10-14T10:33:31.675 INFO:teuthology.orchestra.run.smithi067.stdout:(114/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 4.3 MB/s | 560 kB 00:00 2023-10-14T10:33:31.685 INFO:teuthology.orchestra.run.smithi072.stdout:(87/131): pcre2-devel-10.32-2.el8.x86_64.rpm 5.0 MB/s | 605 kB 00:00 2023-10-14T10:33:31.701 INFO:teuthology.orchestra.run.smithi067.stdout:(115/131): libverto-devel-0.3.0-5.el8.x86_64.rp 306 kB/s | 18 kB 00:00 2023-10-14T10:33:31.710 INFO:teuthology.orchestra.run.smithi072.stdout:(88/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 1.2 MB/s | 72 kB 00:00 2023-10-14T10:33:31.734 INFO:teuthology.orchestra.run.smithi067.stdout:(116/131): python3-cryptography-3.2.1-5.el8.x86 4.7 MB/s | 559 kB 00:00 2023-10-14T10:33:31.736 INFO:teuthology.orchestra.run.smithi072.stdout:(89/131): fuse-2.9.7-15.el8.x86_64.rpm 1.6 MB/s | 83 kB 00:00 2023-10-14T10:33:31.759 INFO:teuthology.orchestra.run.smithi067.stdout:(117/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 4.0 MB/s | 238 kB 00:00 2023-10-14T10:33:31.761 INFO:teuthology.orchestra.run.smithi072.stdout:(90/131): elfutils-libelf-devel-0.186-1.el8.x86 1.2 MB/s | 60 kB 00:00 2023-10-14T10:33:31.787 INFO:teuthology.orchestra.run.smithi072.stdout:(91/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 1.9 MB/s | 98 kB 00:00 2023-10-14T10:33:31.812 INFO:teuthology.orchestra.run.smithi072.stdout:(92/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2023-10-14T10:33:31.818 INFO:teuthology.orchestra.run.smithi067.stdout:(118/131): pcre2-devel-10.32-2.el8.x86_64.rpm 4.1 MB/s | 605 kB 00:00 2023-10-14T10:33:31.838 INFO:teuthology.orchestra.run.smithi072.stdout:(93/131): libcom_err-devel-1.45.6-4.el8.x86_64. 762 kB/s | 39 kB 00:00 2023-10-14T10:33:31.843 INFO:teuthology.orchestra.run.smithi067.stdout:(119/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 865 kB/s | 72 kB 00:00 2023-10-14T10:33:31.869 INFO:teuthology.orchestra.run.smithi067.stdout:(120/131): fuse-2.9.7-15.el8.x86_64.rpm 1.6 MB/s | 83 kB 00:00 2023-10-14T10:33:31.880 INFO:teuthology.orchestra.run.smithi072.stdout:(94/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 9.3 MB/s | 644 kB 00:00 2023-10-14T10:33:31.894 INFO:teuthology.orchestra.run.smithi067.stdout:(121/131): elfutils-libelf-devel-0.186-1.el8.x8 1.2 MB/s | 60 kB 00:00 2023-10-14T10:33:31.914 INFO:teuthology.orchestra.run.smithi072.stdout:(95/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 2.2 MB/s | 171 kB 00:00 2023-10-14T10:33:31.919 INFO:teuthology.orchestra.run.smithi067.stdout:(122/131): libuuid-devel-2.32.1-35.el8.x86_64.r 1.9 MB/s | 98 kB 00:00 2023-10-14T10:33:31.944 INFO:teuthology.orchestra.run.smithi067.stdout:(123/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 1.1 MB/s | 58 kB 00:00 2023-10-14T10:33:31.948 INFO:teuthology.orchestra.run.smithi072.stdout:(96/131): NetworkManager-initscripts-updown-1.3 2.0 MB/s | 138 kB 00:00 2023-10-14T10:33:31.970 INFO:teuthology.orchestra.run.smithi067.stdout:(124/131): libcom_err-devel-1.45.6-4.el8.x86_64 772 kB/s | 39 kB 00:00 2023-10-14T10:33:31.992 INFO:teuthology.orchestra.run.smithi072.stdout:(97/131): python3-pyasn1-modules-0.3.7-6.el8.no 2.5 MB/s | 110 kB 00:00 2023-10-14T10:33:32.012 INFO:teuthology.orchestra.run.smithi067.stdout:(125/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 9.4 MB/s | 644 kB 00:00 2023-10-14T10:33:32.037 INFO:teuthology.orchestra.run.smithi067.stdout:(126/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 2.5 MB/s | 171 kB 00:00 2023-10-14T10:33:32.050 INFO:teuthology.orchestra.run.smithi072.stdout:(98/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 5.4 MB/s | 2.3 MB 00:00 2023-10-14T10:33:32.062 INFO:teuthology.orchestra.run.smithi067.stdout:(127/131): NetworkManager-initscripts-updown-1. 2.7 MB/s | 138 kB 00:00 2023-10-14T10:33:32.076 INFO:teuthology.orchestra.run.smithi072.stdout:(99/131): python3-pytz-2017.2-9.el8.noarch.rpm 644 kB/s | 54 kB 00:00 2023-10-14T10:33:32.121 INFO:teuthology.orchestra.run.smithi067.stdout:(128/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 6.0 MB/s | 2.3 MB 00:00 2023-10-14T10:33:32.135 INFO:teuthology.orchestra.run.smithi072.stdout:(100/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 3.0 MB/s | 259 kB 00:00 2023-10-14T10:33:32.169 INFO:teuthology.orchestra.run.smithi072.stdout:(101/131): socat-1.7.4.1-1.el8.x86_64.rpm 3.4 MB/s | 323 kB 00:00 2023-10-14T10:33:32.171 INFO:teuthology.orchestra.run.smithi067.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 16 MB/s | 798 kB 00:00 2023-10-14T10:33:32.194 INFO:teuthology.orchestra.run.smithi072.stdout:(102/131): python3-markupsafe-0.23-19.el8.x86_6 659 kB/s | 39 kB 00:00 2023-10-14T10:33:32.236 INFO:teuthology.orchestra.run.smithi072.stdout:(103/131): bison-3.0.4-10.el8.x86_64.rpm 10 MB/s | 688 kB 00:00 2023-10-14T10:33:32.270 INFO:teuthology.orchestra.run.smithi072.stdout:(104/131): python3-jinja2-2.10.1-3.el8.noarch.r 7.0 MB/s | 538 kB 00:00 2023-10-14T10:33:32.304 INFO:teuthology.orchestra.run.smithi072.stdout:(105/131): python3-mako-1.0.6-13.el8.noarch.rpm 2.3 MB/s | 157 kB 00:00 2023-10-14T10:33:32.330 INFO:teuthology.orchestra.run.smithi072.stdout:(106/131): oniguruma-6.8.2-2.el8.x86_64.rpm 3.1 MB/s | 187 kB 00:00 2023-10-14T10:33:32.380 INFO:teuthology.orchestra.run.smithi072.stdout:(107/131): automake-1.16.1-7.el8.noarch.rpm 9.2 MB/s | 713 kB 00:00 2023-10-14T10:33:32.414 INFO:teuthology.orchestra.run.smithi072.stdout:(108/131): python3-jsonpatch-1.21-2.el8.noarch. 796 kB/s | 27 kB 00:00 2023-10-14T10:33:32.447 INFO:teuthology.orchestra.run.smithi067.stdout:(130/131): librbd1-18.0.0-6730.g4ba8a190.el8.x8 9.7 MB/s | 3.7 MB 00:00 2023-10-14T10:33:32.465 INFO:teuthology.orchestra.run.smithi072.stdout:(109/131): flex-2.6.1-9.el8.x86_64.rpm 6.2 MB/s | 320 kB 00:00 2023-10-14T10:33:32.498 INFO:teuthology.orchestra.run.smithi072.stdout:(110/131): python3-prettytable-0.7.2-14.el8.noa 1.3 MB/s | 44 kB 00:00 2023-10-14T10:33:32.605 INFO:teuthology.orchestra.run.smithi067.stdout:(131/131): librados2-18.0.0-6730.g4ba8a190.el8. 6.6 MB/s | 3.7 MB 00:00 2023-10-14T10:33:32.608 INFO:teuthology.orchestra.run.smithi067.stdout:-------------------------------------------------------------------------------- 2023-10-14T10:33:32.609 INFO:teuthology.orchestra.run.smithi067.stdout:Total 15 MB/s | 209 MB 00:13 2023-10-14T10:33:32.625 INFO:teuthology.orchestra.run.smithi072.stdout:(111/131): python3-babel-2.5.1-7.el8.noarch.rpm 16 MB/s | 4.8 MB 00:00 2023-10-14T10:33:32.675 INFO:teuthology.orchestra.run.smithi072.stdout:(112/131): python3-werkzeug-0.12.2-4.el8.noarch 8.9 MB/s | 457 kB 00:00 2023-10-14T10:33:32.718 INFO:teuthology.orchestra.run.smithi072.stdout:(113/131): python3-numpy-f2py-1.14.3-10.el8.x86 5.1 MB/s | 225 kB 00:00 2023-10-14T10:33:32.840 INFO:teuthology.orchestra.run.smithi072.stdout:(114/131): python3-numpy-1.14.3-10.el8.x86_64.r 11 MB/s | 3.7 MB 00:00 2023-10-14T10:33:32.869 INFO:teuthology.orchestra.run.smithi072.stdout:(115/131): uuid-1.6.2-43.el8.x86_64.rpm 426 kB/s | 64 kB 00:00 2023-10-14T10:33:32.903 INFO:teuthology.orchestra.run.smithi072.stdout:(116/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 2.0 MB/s | 126 kB 00:00 2023-10-14T10:33:32.929 INFO:teuthology.orchestra.run.smithi072.stdout:(117/131): python3-jsonpointer-1.10-11.el8.noar 339 kB/s | 20 kB 00:00 2023-10-14T10:33:32.954 INFO:teuthology.orchestra.run.smithi072.stdout:(118/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 2.0 MB/s | 103 kB 00:00 2023-10-14T10:33:32.996 INFO:teuthology.orchestra.run.smithi072.stdout:(119/131): libtool-2.4.6-25.el8.x86_64.rpm 10 MB/s | 709 kB 00:00 2023-10-14T10:33:33.039 INFO:teuthology.orchestra.run.smithi072.stdout:(120/131): libpq-13.5-1.el8.x86_64.rpm 4.6 MB/s | 198 kB 00:00 2023-10-14T10:33:33.264 INFO:teuthology.orchestra.run.smithi072.stdout:(121/131): python3-scipy-1.0.0-21.module+el8.5. 10 MB/s | 14 MB 00:01 2023-10-14T10:33:33.314 INFO:teuthology.orchestra.run.smithi072.stdout:(122/131): jq-1.6-3.el8.x86_64.rpm 4.0 MB/s | 202 kB 00:00 2023-10-14T10:33:33.357 INFO:teuthology.orchestra.run.smithi072.stdout:(123/131): librdkafka-0.11.4-3.el8.x86_64.rpm 8.3 MB/s | 354 kB 00:00 2023-10-14T10:33:33.465 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction check 2023-10-14T10:33:33.482 INFO:teuthology.orchestra.run.smithi072.stdout:(124/131): openblas-threads-0.3.15-3.el8.x86_64 9.2 MB/s | 4.9 MB 00:00 2023-10-14T10:33:33.596 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction check succeeded. 2023-10-14T10:33:33.597 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction test 2023-10-14T10:33:33.599 INFO:teuthology.orchestra.run.smithi072.stdout:(125/131): openblas-0.3.15-3.el8.x86_64.rpm 8.4 MB/s | 4.7 MB 00:00 2023-10-14T10:33:33.642 INFO:teuthology.orchestra.run.smithi072.stdout:(126/131): postgresql-contrib-10.21-2.module+el 5.0 MB/s | 808 kB 00:00 2023-10-14T10:33:33.709 INFO:teuthology.orchestra.run.smithi072.stdout:(127/131): postgresql-10.21-2.module+el8.6.0+15 14 MB/s | 1.5 MB 00:00 2023-10-14T10:33:33.829 INFO:teuthology.orchestra.run.smithi072.stdout:(128/131): postgresql-server-10.21-2.module+el8 11 MB/s | 5.1 MB 00:00 2023-10-14T10:33:33.887 INFO:teuthology.orchestra.run.smithi072.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 14 MB/s | 798 kB 00:00 2023-10-14T10:33:34.129 INFO:teuthology.orchestra.run.smithi072.stdout:(130/131): librados2-18.0.0-6730.g4ba8a190.el8. 7.7 MB/s | 3.7 MB 00:00 2023-10-14T10:33:34.296 INFO:teuthology.orchestra.run.smithi072.stdout:(131/131): librbd1-18.0.0-6730.g4ba8a190.el8.x8 6.3 MB/s | 3.7 MB 00:00 2023-10-14T10:33:34.299 INFO:teuthology.orchestra.run.smithi072.stdout:-------------------------------------------------------------------------------- 2023-10-14T10:33:34.299 INFO:teuthology.orchestra.run.smithi072.stdout:Total 13 MB/s | 209 MB 00:15 2023-10-14T10:33:34.790 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction test succeeded. 2023-10-14T10:33:34.795 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction 2023-10-14T10:33:35.129 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction check 2023-10-14T10:33:35.250 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction check succeeded. 2023-10-14T10:33:35.250 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction test 2023-10-14T10:33:36.398 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction test succeeded. 2023-10-14T10:33:36.406 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction 2023-10-14T10:33:40.173 INFO:teuthology.orchestra.run.smithi067.stdout: Preparing : 1/1 2023-10-14T10:33:40.338 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-10-14T10:33:40.519 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-10-14T10:33:40.732 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-10-14T10:33:40.873 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-10-14T10:33:41.009 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-10-14T10:33:41.199 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-10-14T10:33:41.631 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-10-14T10:33:41.835 INFO:teuthology.orchestra.run.smithi072.stdout: Preparing : 1/1 2023-10-14T10:33:41.840 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : librados2-2:18.0.0-6730.g4ba8a190.el8.x86_64 6/134 2023-10-14T10:33:42.192 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-10-14T10:33:42.276 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: librados2-2:18.0.0-6730.g4ba8a190.el8.x86_64 6/134 2023-10-14T10:33:42.381 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-10-14T10:33:42.448 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : librbd1-2:18.0.0-6730.g4ba8a190.el8.x86_64 7/134 2023-10-14T10:33:42.558 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-10-14T10:33:42.565 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: librbd1-2:18.0.0-6730.g4ba8a190.el8.x86_64 7/134 2023-10-14T10:33:42.730 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-10-14T10:33:42.782 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libcephfs2-2:18.0.0-6730.g4ba8a190.el8.x86_64 8/134 2023-10-14T10:33:42.880 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: libcephfs2-2:18.0.0-6730.g4ba8a190.el8.x86_64 8/134 2023-10-14T10:33:42.925 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-10-14T10:33:43.075 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-rados-2:18.0.0-6730.g4ba8a190.el8.x86_64 9/134 2023-10-14T10:33:43.315 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-10-14T10:33:43.330 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-10-14T10:33:43.535 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-10-14T10:33:43.608 INFO:teuthology.orchestra.run.smithi072.stdout: Upgrading : librados2-2:18.0.0-6730.g4ba8a190.el8.x86_64 5/134 2023-10-14T10:33:43.940 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-10-14T10:33:44.016 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: librados2-2:18.0.0-6730.g4ba8a190.el8.x86_64 5/134 2023-10-14T10:33:44.181 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-10-14T10:33:44.249 INFO:teuthology.orchestra.run.smithi072.stdout: Upgrading : librbd1-2:18.0.0-6730.g4ba8a190.el8.x86_64 6/134 2023-10-14T10:33:44.386 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: librbd1-2:18.0.0-6730.g4ba8a190.el8.x86_64 6/134 2023-10-14T10:33:44.398 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libcephsqlite-2:18.0.0-6730.g4ba8a190.el8.x86_64 14/134 2023-10-14T10:33:44.495 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: libcephsqlite-2:18.0.0-6730.g4ba8a190.el8.x86_64 14/134 2023-10-14T10:33:44.592 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libcephfs2-2:18.0.0-6730.g4ba8a190.el8.x86_64 7/134 2023-10-14T10:33:44.671 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: libcephfs2-2:18.0.0-6730.g4ba8a190.el8.x86_64 7/134 2023-10-14T10:33:44.674 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libradosstriper1-2:18.0.0-6730.g4ba8a190.el8.x86 15/134 2023-10-14T10:33:44.757 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: libradosstriper1-2:18.0.0-6730.g4ba8a190.el8.x86 15/134 2023-10-14T10:33:44.900 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-10-14T10:33:44.901 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-rados-2:18.0.0-6730.g4ba8a190.el8.x86_64 8/134 2023-10-14T10:33:45.047 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-10-14T10:33:45.097 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-10-14T10:33:45.181 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-10-14T10:33:45.354 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-10-14T10:33:45.594 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-10-14T10:33:45.869 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-10-14T10:33:46.006 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-10-14T10:33:46.015 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-10-14T10:33:46.306 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-10-14T10:33:46.491 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libcephsqlite-2:18.0.0-6730.g4ba8a190.el8.x86_64 14/134 2023-10-14T10:33:46.599 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: libcephsqlite-2:18.0.0-6730.g4ba8a190.el8.x86_64 14/134 2023-10-14T10:33:46.643 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-10-14T10:33:46.791 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libradosstriper1-2:18.0.0-6730.g4ba8a190.el8.x86 15/134 2023-10-14T10:33:46.940 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: libradosstriper1-2:18.0.0-6730.g4ba8a190.el8.x86 15/134 2023-10-14T10:33:47.039 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-10-14T10:33:47.279 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-10-14T10:33:47.800 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-10-14T10:33:47.866 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-10-14T10:33:48.042 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-10-14T10:33:48.089 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-10-14T10:33:48.182 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-10-14T10:33:48.330 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-10-14T10:33:48.330 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-10-14T10:33:48.465 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-10-14T10:33:48.544 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-10-14T10:33:48.755 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-10-14T10:33:48.950 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-10-14T10:33:49.089 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-10-14T10:33:49.145 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-10-14T10:33:49.296 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-ceph-common-2:18.0.0-6730.g4ba8a190.el8. 26/134 2023-10-14T10:33:49.299 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-10-14T10:33:49.426 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-ceph-argparse-2:18.0.0-6730.g4ba8a190.el 27/134 2023-10-14T10:33:49.886 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-10-14T10:33:50.107 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-10-14T10:33:50.402 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-cephfs-2:18.0.0-6730.g4ba8a190.el8.x86_6 28/134 2023-10-14T10:33:50.756 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-10-14T10:33:50.832 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-10-14T10:33:51.055 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-10-14T10:33:51.141 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-10-14T10:33:51.300 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-10-14T10:33:51.322 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-10-14T10:33:51.514 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-10-14T10:33:51.526 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-10-14T10:33:51.688 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-ceph-common-2:18.0.0-6730.g4ba8a190.el8. 26/134 2023-10-14T10:33:51.859 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-ceph-argparse-2:18.0.0-6730.g4ba8a190.el 27/134 2023-10-14T10:33:52.113 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-cephfs-2:18.0.0-6730.g4ba8a190.el8.x86_6 28/134 2023-10-14T10:33:52.307 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-10-14T10:33:53.406 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-10-14T10:33:53.734 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-10-14T10:33:53.928 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-10-14T10:33:54.111 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-10-14T10:33:54.126 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-10-14T10:33:54.287 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-rbd-2:18.0.0-6730.g4ba8a190.el8.x86_64 36/134 2023-10-14T10:33:54.468 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : librados-devel-2:18.0.0-6730.g4ba8a190.el8.x86_6 37/134 2023-10-14T10:33:54.606 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-10-14T10:33:54.765 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-10-14T10:33:54.933 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-10-14T10:33:55.077 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-10-14T10:33:55.160 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-10-14T10:33:55.306 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-10-14T10:33:55.515 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-10-14T10:33:55.529 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-10-14T10:33:55.782 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-10-14T10:33:55.909 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-10-14T10:33:55.970 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-10-14T10:33:56.193 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-10-14T10:33:56.219 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-10-14T10:33:56.409 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-10-14T10:33:56.425 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-10-14T10:33:56.623 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-rbd-2:18.0.0-6730.g4ba8a190.el8.x86_64 36/134 2023-10-14T10:33:56.652 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-10-14T10:33:56.821 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : librados-devel-2:18.0.0-6730.g4ba8a190.el8.x86_6 37/134 2023-10-14T10:33:56.879 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-10-14T10:33:56.972 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-10-14T10:33:57.134 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-10-14T10:33:57.245 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-10-14T10:33:57.339 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-10-14T10:33:57.410 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-10-14T10:33:57.522 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-10-14T10:33:57.684 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-10-14T10:33:57.704 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-10-14T10:33:57.903 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-10-14T10:33:57.929 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-10-14T10:33:58.073 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-10-14T10:33:58.078 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-10-14T10:33:58.182 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-10-14T10:33:58.251 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-10-14T10:33:58.404 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-10-14T10:33:58.577 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-10-14T10:33:58.578 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-10-14T10:33:58.749 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-10-14T10:33:58.869 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-10-14T10:33:58.985 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-10-14T10:33:59.105 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-10-14T10:33:59.159 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-10-14T10:33:59.293 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-10-14T10:33:59.345 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-10-14T10:33:59.504 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-10-14T10:33:59.530 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-10-14T10:33:59.682 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-10-14T10:33:59.779 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-10-14T10:33:59.821 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-10-14T10:33:59.898 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-10-14T10:34:00.137 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-10-14T10:34:00.167 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-10-14T10:34:00.359 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-10-14T10:34:00.417 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-10-14T10:34:00.547 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-10-14T10:34:00.627 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-10-14T10:34:00.634 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-10-14T10:34:00.848 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-10-14T10:34:00.893 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-10-14T10:34:01.033 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-10-14T10:34:01.181 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-10-14T10:34:01.185 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-10-14T10:34:01.345 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-10-14T10:34:01.513 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-10-14T10:34:01.619 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-10-14T10:34:01.747 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-10-14T10:34:01.854 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-10-14T10:34:01.944 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-10-14T10:34:02.001 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-10-14T10:34:02.172 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-10-14T10:34:02.184 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-10-14T10:34:02.402 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-10-14T10:34:02.435 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-10-14T10:34:02.595 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-10-14T10:34:02.645 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-10-14T10:34:02.877 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-10-14T10:34:02.903 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-10-14T10:34:03.033 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-10-14T10:34:03.211 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-10-14T10:34:03.221 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-10-14T10:34:03.423 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-10-14T10:34:03.459 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-10-14T10:34:03.668 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-10-14T10:34:03.699 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-10-14T10:34:03.852 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-10-14T10:34:03.888 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-10-14T10:34:04.067 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-10-14T10:34:04.143 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-10-14T10:34:04.350 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-10-14T10:34:04.371 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-10-14T10:34:04.580 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-10-14T10:34:04.612 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-10-14T10:34:04.793 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-10-14T10:34:04.806 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-10-14T10:34:04.959 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-10-14T10:34:05.150 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-10-14T10:34:05.376 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-10-14T10:34:05.567 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-10-14T10:34:05.733 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-10-14T10:34:05.892 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-10-14T10:34:05.966 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-10-14T10:34:06.160 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-10-14T10:34:06.227 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-10-14T10:34:06.376 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-10-14T10:34:06.456 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-10-14T10:34:06.561 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-10-14T10:34:06.766 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-10-14T10:34:06.885 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-10-14T10:34:07.032 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-10-14T10:34:07.113 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-10-14T10:34:07.241 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-10-14T10:34:07.338 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-10-14T10:34:07.563 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-10-14T10:34:08.078 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-10-14T10:34:08.228 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : librgw2-2:18.0.0-6730.g4ba8a190.el8.x86_64 92/134 2023-10-14T10:34:08.302 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: librgw2-2:18.0.0-6730.g4ba8a190.el8.x86_64 92/134 2023-10-14T10:34:08.487 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-rgw-2:18.0.0-6730.g4ba8a190.el8.x86_64 93/134 2023-10-14T10:34:08.648 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-10-14T10:34:08.959 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-10-14T10:34:09.204 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-10-14T10:34:09.555 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-10-14T10:34:09.853 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-10-14T10:34:10.063 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-10-14T10:34:10.945 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-10-14T10:34:11.129 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : librgw2-2:18.0.0-6730.g4ba8a190.el8.x86_64 92/134 2023-10-14T10:34:11.203 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: librgw2-2:18.0.0-6730.g4ba8a190.el8.x86_64 92/134 2023-10-14T10:34:11.388 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-rgw-2:18.0.0-6730.g4ba8a190.el8.x86_64 93/134 2023-10-14T10:34:12.466 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-common-2:18.0.0-6730.g4ba8a190.el8.x86_64 94/134 2023-10-14T10:34:12.654 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-common-2:18.0.0-6730.g4ba8a190.el8.x86_64 94/134 2023-10-14T10:34:13.335 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-common-2:18.0.0-6730.g4ba8a190.el8.x86_64 94/134 2023-10-14T10:34:13.622 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-base-2:18.0.0-6730.g4ba8a190.el8.x86_64 95/134 2023-10-14T10:34:13.704 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-base-2:18.0.0-6730.g4ba8a190.el8.x86_64 95/134 2023-10-14T10:34:13.836 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-selinux-2:18.0.0-6730.g4ba8a190.el8.x86_64 96/134 2023-10-14T10:34:15.338 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-common-2:18.0.0-6730.g4ba8a190.el8.x86_64 94/134 2023-10-14T10:34:15.579 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-common-2:18.0.0-6730.g4ba8a190.el8.x86_64 94/134 2023-10-14T10:34:16.193 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-common-2:18.0.0-6730.g4ba8a190.el8.x86_64 94/134 2023-10-14T10:34:16.542 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-base-2:18.0.0-6730.g4ba8a190.el8.x86_64 95/134 2023-10-14T10:34:16.628 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-base-2:18.0.0-6730.g4ba8a190.el8.x86_64 95/134 2023-10-14T10:34:16.795 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-selinux-2:18.0.0-6730.g4ba8a190.el8.x86_64 96/134 2023-10-14T10:34:33.208 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-selinux-2:18.0.0-6730.g4ba8a190.el8.x86_64 96/134 2023-10-14T10:34:33.209 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /sys 2023-10-14T10:34:33.209 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /proc 2023-10-14T10:34:33.209 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /dev 2023-10-14T10:34:33.210 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /run 2023-10-14T10:34:33.210 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /mnt 2023-10-14T10:34:33.210 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /var/tmp 2023-10-14T10:34:33.211 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /home 2023-10-14T10:34:33.211 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /tmp 2023-10-14T10:34:33.211 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /dev 2023-10-14T10:34:33.211 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-14T10:34:33.584 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-volume-2:18.0.0-6730.g4ba8a190.el8.noarch 97/134 2023-10-14T10:34:35.277 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-volume-2:18.0.0-6730.g4ba8a190.el8.noarch 97/134 2023-10-14T10:34:35.460 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-osd-2:18.0.0-6730.g4ba8a190.el8.x86_64 98/134 2023-10-14T10:34:35.694 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-osd-2:18.0.0-6730.g4ba8a190.el8.x86_64 98/134 2023-10-14T10:34:35.850 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mds-2:18.0.0-6730.g4ba8a190.el8.x86_64 99/134 2023-10-14T10:34:36.347 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-mds-2:18.0.0-6730.g4ba8a190.el8.x86_64 99/134 2023-10-14T10:34:36.517 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mon-2:18.0.0-6730.g4ba8a190.el8.x86_64 100/134 2023-10-14T10:34:36.535 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-mon-2:18.0.0-6730.g4ba8a190.el8.x86_64 100/134 2023-10-14T10:34:37.664 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-selinux-2:18.0.0-6730.g4ba8a190.el8.x86_64 96/134 2023-10-14T10:34:37.664 INFO:teuthology.orchestra.run.smithi072.stdout:skipping the directory /sys 2023-10-14T10:34:37.665 INFO:teuthology.orchestra.run.smithi072.stdout:skipping the directory /proc 2023-10-14T10:34:37.665 INFO:teuthology.orchestra.run.smithi072.stdout:skipping the directory /dev 2023-10-14T10:34:37.665 INFO:teuthology.orchestra.run.smithi072.stdout:skipping the directory /run 2023-10-14T10:34:37.665 INFO:teuthology.orchestra.run.smithi072.stdout:skipping the directory /mnt 2023-10-14T10:34:37.665 INFO:teuthology.orchestra.run.smithi072.stdout:skipping the directory /var/tmp 2023-10-14T10:34:37.666 INFO:teuthology.orchestra.run.smithi072.stdout:skipping the directory /home 2023-10-14T10:34:37.666 INFO:teuthology.orchestra.run.smithi072.stdout:skipping the directory /tmp 2023-10-14T10:34:37.666 INFO:teuthology.orchestra.run.smithi072.stdout:skipping the directory /dev 2023-10-14T10:34:37.666 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-14T10:34:38.010 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-volume-2:18.0.0-6730.g4ba8a190.el8.noarch 97/134 2023-10-14T10:34:38.408 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: cephadm-2:18.0.0-6730.g4ba8a190.el8.noarch 101/134 2023-10-14T10:34:38.588 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : cephadm-2:18.0.0-6730.g4ba8a190.el8.noarch 101/134 2023-10-14T10:34:38.748 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-6730.g4ba8a190.e 102/134 2023-10-14T10:34:38.943 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-6730.g4ba8a190. 103/134 2023-10-14T10:34:39.116 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-6730.g4ba8a190.el8.noa 104/134 2023-10-14T10:34:39.349 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-6730.g4ba8a190.el8.noa 104/134 2023-10-14T10:34:39.598 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-volume-2:18.0.0-6730.g4ba8a190.el8.noarch 97/134 2023-10-14T10:34:39.618 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-6730.g4ba8a190.el8.n 105/134 2023-10-14T10:34:39.772 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-osd-2:18.0.0-6730.g4ba8a190.el8.x86_64 98/134 2023-10-14T10:34:39.992 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-osd-2:18.0.0-6730.g4ba8a190.el8.x86_64 98/134 2023-10-14T10:34:40.151 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-mds-2:18.0.0-6730.g4ba8a190.el8.x86_64 99/134 2023-10-14T10:34:40.618 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-mds-2:18.0.0-6730.g4ba8a190.el8.x86_64 99/134 2023-10-14T10:34:40.643 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-6730.g4ba8a190.el8.n 105/134 2023-10-14T10:34:40.808 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-6730.g4ba 106/134 2023-10-14T10:34:40.818 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-mon-2:18.0.0-6730.g4ba8a190.el8.x86_64 100/134 2023-10-14T10:34:40.834 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-mon-2:18.0.0-6730.g4ba8a190.el8.x86_64 100/134 2023-10-14T10:34:40.907 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-6730.g4ba 106/134 2023-10-14T10:34:41.274 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-6730.g4ba8a190.el 107/134 2023-10-14T10:34:41.468 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mgr-2:18.0.0-6730.g4ba8a190.el8.x86_64 108/134 2023-10-14T10:34:41.505 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-mgr-2:18.0.0-6730.g4ba8a190.el8.x86_64 108/134 2023-10-14T10:34:41.684 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mgr-rook-2:18.0.0-6730.g4ba8a190.el8.noarch 109/134 2023-10-14T10:34:41.695 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-6730.g4ba8a190.el8.noarch 109/134 2023-10-14T10:34:42.558 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: cephadm-2:18.0.0-6730.g4ba8a190.el8.noarch 101/134 2023-10-14T10:34:42.783 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : cephadm-2:18.0.0-6730.g4ba8a190.el8.noarch 101/134 2023-10-14T10:34:42.898 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-6730.g4ba8a190.e 102/134 2023-10-14T10:34:43.136 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-2:18.0.0-6730.g4ba8a190.el8.x86_64 110/134 2023-10-14T10:34:43.149 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-6730.g4ba8a190. 103/134 2023-10-14T10:34:43.334 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-6730.g4ba8a190.el8.noa 104/134 2023-10-14T10:34:43.340 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-radosgw-2:18.0.0-6730.g4ba8a190.el8.x86_64 111/134 2023-10-14T10:34:43.375 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-6730.g4ba8a190.el8.x86_64 111/134 2023-10-14T10:34:43.533 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-6730.g4ba8a 112/134 2023-10-14T10:34:43.554 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-6730.g4ba8a190.el8.noa 104/134 2023-10-14T10:34:43.577 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-6730.g4ba8a 112/134 2023-10-14T10:34:43.725 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : cephfs-mirror-2:18.0.0-6730.g4ba8a190.el8.x86_64 113/134 2023-10-14T10:34:43.863 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-6730.g4ba8a190.el8.n 105/134 2023-10-14T10:34:44.116 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-6730.g4ba8a190.el8.x86_64 113/134 2023-10-14T10:34:44.250 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : rbd-mirror-2:18.0.0-6730.g4ba8a190.el8.x86_64 114/134 2023-10-14T10:34:44.823 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-6730.g4ba8a190.el8.n 105/134 2023-10-14T10:34:45.000 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-6730.g4ba 106/134 2023-10-14T10:34:45.091 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-6730.g4ba 106/134 2023-10-14T10:34:45.486 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-6730.g4ba8a190.el 107/134 2023-10-14T10:34:45.689 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-mgr-2:18.0.0-6730.g4ba8a190.el8.x86_64 108/134 2023-10-14T10:34:45.724 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-mgr-2:18.0.0-6730.g4ba8a190.el8.x86_64 108/134 2023-10-14T10:34:45.958 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-mgr-rook-2:18.0.0-6730.g4ba8a190.el8.noarch 109/134 2023-10-14T10:34:45.969 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-6730.g4ba8a190.el8.noarch 109/134 2023-10-14T10:34:47.315 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-2:18.0.0-6730.g4ba8a190.el8.x86_64 110/134 2023-10-14T10:34:47.537 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-radosgw-2:18.0.0-6730.g4ba8a190.el8.x86_64 111/134 2023-10-14T10:34:47.571 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-6730.g4ba8a190.el8.x86_64 111/134 2023-10-14T10:34:47.734 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-6730.g4ba8a 112/134 2023-10-14T10:34:47.776 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-6730.g4ba8a 112/134 2023-10-14T10:34:47.917 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : cephfs-mirror-2:18.0.0-6730.g4ba8a190.el8.x86_64 113/134 2023-10-14T10:34:48.290 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-6730.g4ba8a190.el8.x86_64 113/134 2023-10-14T10:34:48.450 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : rbd-mirror-2:18.0.0-6730.g4ba8a190.el8.x86_64 114/134 2023-10-14T10:34:49.208 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: rbd-mirror-2:18.0.0-6730.g4ba8a190.el8.x86_64 114/134 2023-10-14T10:34:49.507 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-test-2:18.0.0-6730.g4ba8a190.el8.x86_64 115/134 2023-10-14T10:34:49.673 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-10-14T10:34:49.892 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-10-14T10:34:50.863 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-10-14T10:34:51.191 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-10-14T10:34:51.401 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-fuse-2:18.0.0-6730.g4ba8a190.el8.x86_64 119/134 2023-10-14T10:34:51.578 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-10-14T10:34:51.761 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libcephfs-devel-2:18.0.0-6730.g4ba8a190.el8.x86_ 121/134 2023-10-14T10:34:51.908 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-10-14T10:34:53.103 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: rbd-mirror-2:18.0.0-6730.g4ba8a190.el8.x86_64 114/134 2023-10-14T10:34:53.987 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-test-2:18.0.0-6730.g4ba8a190.el8.x86_64 115/134 2023-10-14T10:34:54.254 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-14T10:34:54.328 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-10-14T10:34:54.507 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-14T10:34:54.529 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-14T10:34:54.537 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-fuse-2:18.0.0-6730.g4ba8a190.el8.x86_64 117/134 2023-10-14T10:34:54.705 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : cephfs-top-2:18.0.0-6730.g4ba8a190.el8.noarch 124/134 2023-10-14T10:34:54.790 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-10-14T10:34:54.949 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-10-14T10:34:54.982 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : rbd-fuse-2:18.0.0-6730.g4ba8a190.el8.x86_64 125/134 2023-10-14T10:34:55.175 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-10-14T10:34:55.292 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : rbd-nbd-2:18.0.0-6730.g4ba8a190.el8.x86_64 126/134 2023-10-14T10:34:55.412 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-10-14T10:34:55.583 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-10-14T10:34:55.594 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libcephfs-devel-2:18.0.0-6730.g4ba8a190.el8.x86_ 121/134 2023-10-14T10:34:55.773 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-10-14T10:34:55.782 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-10-14T10:34:55.792 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-10-14T10:34:55.966 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-10-14T10:34:56.156 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-10-14T10:34:56.241 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-10-14T10:34:56.423 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-10-14T10:34:56.436 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-10-14T10:34:56.436 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-14T10:34:56.659 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-14T10:34:56.660 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-10-14T10:34:56.878 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-10-14T10:34:57.080 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-10-14T10:34:57.610 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-10-14T10:34:57.610 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-2:18.0.0-6730.g4ba8a190.el8.x86_64 1/134 2023-10-14T10:34:57.610 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-base-2:18.0.0-6730.g4ba8a190.el8.x86_64 2/134 2023-10-14T10:34:57.611 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-common-2:18.0.0-6730.g4ba8a190.el8.x86_64 3/134 2023-10-14T10:34:57.611 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-fuse-2:18.0.0-6730.g4ba8a190.el8.x86_64 4/134 2023-10-14T10:34:57.611 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-6730.g4ba8a 5/134 2023-10-14T10:34:57.612 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mds-2:18.0.0-6730.g4ba8a190.el8.x86_64 6/134 2023-10-14T10:34:57.612 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mgr-2:18.0.0-6730.g4ba8a190.el8.x86_64 7/134 2023-10-14T10:34:57.612 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mon-2:18.0.0-6730.g4ba8a190.el8.x86_64 8/134 2023-10-14T10:34:57.612 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-osd-2:18.0.0-6730.g4ba8a190.el8.x86_64 9/134 2023-10-14T10:34:57.612 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-radosgw-2:18.0.0-6730.g4ba8a190.el8.x86_64 10/134 2023-10-14T10:34:57.613 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-selinux-2:18.0.0-6730.g4ba8a190.el8.x86_64 11/134 2023-10-14T10:34:57.613 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-test-2:18.0.0-6730.g4ba8a190.el8.x86_64 12/134 2023-10-14T10:34:57.613 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : cephfs-mirror-2:18.0.0-6730.g4ba8a190.el8.x86_64 13/134 2023-10-14T10:34:57.614 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libcephfs-devel-2:18.0.0-6730.g4ba8a190.el8.x86_ 14/134 2023-10-14T10:34:57.614 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libcephfs2-2:18.0.0-6730.g4ba8a190.el8.x86_64 15/134 2023-10-14T10:34:57.614 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libcephsqlite-2:18.0.0-6730.g4ba8a190.el8.x86_64 16/134 2023-10-14T10:34:57.615 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librados-devel-2:18.0.0-6730.g4ba8a190.el8.x86_6 17/134 2023-10-14T10:34:57.615 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libradosstriper1-2:18.0.0-6730.g4ba8a190.el8.x86 18/134 2023-10-14T10:34:57.615 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librgw2-2:18.0.0-6730.g4ba8a190.el8.x86_64 19/134 2023-10-14T10:34:57.615 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-ceph-argparse-2:18.0.0-6730.g4ba8a190.el 20/134 2023-10-14T10:34:57.616 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-ceph-common-2:18.0.0-6730.g4ba8a190.el8. 21/134 2023-10-14T10:34:57.616 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-cephfs-2:18.0.0-6730.g4ba8a190.el8.x86_6 22/134 2023-10-14T10:34:57.616 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-rados-2:18.0.0-6730.g4ba8a190.el8.x86_64 23/134 2023-10-14T10:34:57.617 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-rbd-2:18.0.0-6730.g4ba8a190.el8.x86_64 24/134 2023-10-14T10:34:57.617 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-rgw-2:18.0.0-6730.g4ba8a190.el8.x86_64 25/134 2023-10-14T10:34:57.617 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : rbd-fuse-2:18.0.0-6730.g4ba8a190.el8.x86_64 26/134 2023-10-14T10:34:57.617 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : rbd-mirror-2:18.0.0-6730.g4ba8a190.el8.x86_64 27/134 2023-10-14T10:34:57.618 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : rbd-nbd-2:18.0.0-6730.g4ba8a190.el8.x86_64 28/134 2023-10-14T10:34:57.618 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-6730.g4ba8a190. 29/134 2023-10-14T10:34:57.618 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-6730.g4ba8a190.el8.noa 30/134 2023-10-14T10:34:57.618 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-6730.g4ba8a190.el8.n 31/134 2023-10-14T10:34:57.619 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-6730.g4ba 32/134 2023-10-14T10:34:57.619 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-6730.g4ba8a190.el 33/134 2023-10-14T10:34:57.619 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mgr-rook-2:18.0.0-6730.g4ba8a190.el8.noarch 34/134 2023-10-14T10:34:57.619 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-6730.g4ba8a190.e 35/134 2023-10-14T10:34:57.620 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-volume-2:18.0.0-6730.g4ba8a190.el8.noarch 36/134 2023-10-14T10:34:57.620 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : cephadm-2:18.0.0-6730.g4ba8a190.el8.noarch 37/134 2023-10-14T10:34:57.620 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : cephfs-top-2:18.0.0-6730.g4ba8a190.el8.noarch 38/134 2023-10-14T10:34:57.621 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-10-14T10:34:57.621 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-10-14T10:34:57.621 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-10-14T10:34:57.622 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-10-14T10:34:57.622 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-10-14T10:34:57.622 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-10-14T10:34:57.622 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-10-14T10:34:57.623 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-10-14T10:34:57.623 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-10-14T10:34:57.623 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-10-14T10:34:57.623 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-10-14T10:34:57.624 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-10-14T10:34:57.624 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-10-14T10:34:57.624 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-10-14T10:34:57.625 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-10-14T10:34:57.625 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-10-14T10:34:57.625 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-10-14T10:34:57.625 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-10-14T10:34:57.626 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-10-14T10:34:57.626 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-10-14T10:34:57.626 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-10-14T10:34:57.627 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-10-14T10:34:57.627 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-10-14T10:34:57.627 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-10-14T10:34:57.628 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-10-14T10:34:57.628 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-10-14T10:34:57.628 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-10-14T10:34:57.628 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-10-14T10:34:57.629 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-10-14T10:34:57.629 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-10-14T10:34:57.629 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-10-14T10:34:57.630 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-10-14T10:34:57.630 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-10-14T10:34:57.630 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-10-14T10:34:57.631 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-10-14T10:34:57.631 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-10-14T10:34:57.631 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-10-14T10:34:57.631 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-10-14T10:34:57.632 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-10-14T10:34:57.632 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-10-14T10:34:57.632 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-10-14T10:34:57.632 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-10-14T10:34:57.633 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-10-14T10:34:57.633 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-10-14T10:34:57.633 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-10-14T10:34:57.633 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-10-14T10:34:57.634 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-10-14T10:34:57.634 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-10-14T10:34:57.634 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-10-14T10:34:57.634 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-10-14T10:34:57.635 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-10-14T10:34:57.635 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-10-14T10:34:57.635 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-10-14T10:34:57.635 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-10-14T10:34:57.636 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-10-14T10:34:57.636 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-10-14T10:34:57.636 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-10-14T10:34:57.636 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-10-14T10:34:57.636 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-10-14T10:34:57.637 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-10-14T10:34:57.637 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-10-14T10:34:57.637 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-10-14T10:34:57.637 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-10-14T10:34:57.637 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-10-14T10:34:57.638 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-10-14T10:34:57.638 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-10-14T10:34:57.638 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-10-14T10:34:57.638 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-10-14T10:34:57.639 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-10-14T10:34:57.639 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-10-14T10:34:57.639 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-10-14T10:34:57.639 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-10-14T10:34:57.639 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-10-14T10:34:57.640 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-10-14T10:34:57.640 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-10-14T10:34:57.640 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-10-14T10:34:57.640 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-10-14T10:34:57.641 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-10-14T10:34:57.641 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-10-14T10:34:57.641 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-10-14T10:34:57.641 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-10-14T10:34:57.641 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-10-14T10:34:57.642 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-10-14T10:34:57.642 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-10-14T10:34:57.642 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-10-14T10:34:57.642 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-10-14T10:34:57.642 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-10-14T10:34:57.643 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-10-14T10:34:57.643 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-10-14T10:34:57.643 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-10-14T10:34:57.644 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librados2-2:18.0.0-6730.g4ba8a190.el8.x86_64 129/134 2023-10-14T10:34:57.644 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-10-14T10:34:57.644 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librbd1-2:18.0.0-6730.g4ba8a190.el8.x86_64 131/134 2023-10-14T10:34:57.644 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-14T10:34:57.645 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-10-14T10:34:58.034 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-14T10:34:58.299 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-14T10:34:58.318 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-14T10:34:58.524 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : cephfs-top-2:18.0.0-6730.g4ba8a190.el8.noarch 124/134 2023-10-14T10:34:58.807 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : rbd-fuse-2:18.0.0-6730.g4ba8a190.el8.x86_64 125/134 2023-10-14T10:34:59.092 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : rbd-nbd-2:18.0.0-6730.g4ba8a190.el8.x86_64 126/134 2023-10-14T10:34:59.461 INFO:teuthology.orchestra.run.smithi072.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-10-14T10:34:59.632 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-10-14T10:34:59.713 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-10-14T10:34:59.915 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-10-14T10:34:59.928 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-10-14T10:35:00.107 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-10-14T10:35:00.115 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-10-14T10:35:00.259 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-10-14T10:35:00.259 INFO:teuthology.orchestra.run.smithi072.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-14T10:35:00.323 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-10-14T10:35:00.323 INFO:teuthology.orchestra.run.smithi067.stdout:Last metadata expiration check: 0:01:54 ago on Sat 14 Oct 2023 10:33:06 AM UTC. 2023-10-14T10:35:00.452 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-14T10:35:00.452 INFO:teuthology.orchestra.run.smithi072.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-10-14T10:35:00.759 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-10-14T10:35:00.939 INFO:teuthology.orchestra.run.smithi072.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-10-14T10:35:01.445 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-10-14T10:35:01.445 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-2:18.0.0-6730.g4ba8a190.el8.x86_64 1/134 2023-10-14T10:35:01.445 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-base-2:18.0.0-6730.g4ba8a190.el8.x86_64 2/134 2023-10-14T10:35:01.446 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-common-2:18.0.0-6730.g4ba8a190.el8.x86_64 3/134 2023-10-14T10:35:01.446 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-fuse-2:18.0.0-6730.g4ba8a190.el8.x86_64 4/134 2023-10-14T10:35:01.446 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-6730.g4ba8a 5/134 2023-10-14T10:35:01.446 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-mds-2:18.0.0-6730.g4ba8a190.el8.x86_64 6/134 2023-10-14T10:35:01.446 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-mgr-2:18.0.0-6730.g4ba8a190.el8.x86_64 7/134 2023-10-14T10:35:01.447 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-mon-2:18.0.0-6730.g4ba8a190.el8.x86_64 8/134 2023-10-14T10:35:01.447 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-osd-2:18.0.0-6730.g4ba8a190.el8.x86_64 9/134 2023-10-14T10:35:01.447 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-radosgw-2:18.0.0-6730.g4ba8a190.el8.x86_64 10/134 2023-10-14T10:35:01.447 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-selinux-2:18.0.0-6730.g4ba8a190.el8.x86_64 11/134 2023-10-14T10:35:01.448 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-test-2:18.0.0-6730.g4ba8a190.el8.x86_64 12/134 2023-10-14T10:35:01.448 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : cephfs-mirror-2:18.0.0-6730.g4ba8a190.el8.x86_64 13/134 2023-10-14T10:35:01.448 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libcephfs-devel-2:18.0.0-6730.g4ba8a190.el8.x86_ 14/134 2023-10-14T10:35:01.448 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libcephfs2-2:18.0.0-6730.g4ba8a190.el8.x86_64 15/134 2023-10-14T10:35:01.449 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libcephsqlite-2:18.0.0-6730.g4ba8a190.el8.x86_64 16/134 2023-10-14T10:35:01.449 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : librados-devel-2:18.0.0-6730.g4ba8a190.el8.x86_6 17/134 2023-10-14T10:35:01.449 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libradosstriper1-2:18.0.0-6730.g4ba8a190.el8.x86 18/134 2023-10-14T10:35:01.449 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : librgw2-2:18.0.0-6730.g4ba8a190.el8.x86_64 19/134 2023-10-14T10:35:01.449 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-ceph-argparse-2:18.0.0-6730.g4ba8a190.el 20/134 2023-10-14T10:35:01.450 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-ceph-common-2:18.0.0-6730.g4ba8a190.el8. 21/134 2023-10-14T10:35:01.450 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-cephfs-2:18.0.0-6730.g4ba8a190.el8.x86_6 22/134 2023-10-14T10:35:01.450 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-rados-2:18.0.0-6730.g4ba8a190.el8.x86_64 23/134 2023-10-14T10:35:01.450 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-rbd-2:18.0.0-6730.g4ba8a190.el8.x86_64 24/134 2023-10-14T10:35:01.451 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-rgw-2:18.0.0-6730.g4ba8a190.el8.x86_64 25/134 2023-10-14T10:35:01.451 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : rbd-fuse-2:18.0.0-6730.g4ba8a190.el8.x86_64 26/134 2023-10-14T10:35:01.451 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : rbd-mirror-2:18.0.0-6730.g4ba8a190.el8.x86_64 27/134 2023-10-14T10:35:01.452 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : rbd-nbd-2:18.0.0-6730.g4ba8a190.el8.x86_64 28/134 2023-10-14T10:35:01.452 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-6730.g4ba8a190. 29/134 2023-10-14T10:35:01.452 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-6730.g4ba8a190.el8.noa 30/134 2023-10-14T10:35:01.462 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-6730.g4ba8a190.el8.n 31/134 2023-10-14T10:35:01.463 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-6730.g4ba 32/134 2023-10-14T10:35:01.463 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-6730.g4ba8a190.el 33/134 2023-10-14T10:35:01.463 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-mgr-rook-2:18.0.0-6730.g4ba8a190.el8.noarch 34/134 2023-10-14T10:35:01.463 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-6730.g4ba8a190.e 35/134 2023-10-14T10:35:01.464 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-volume-2:18.0.0-6730.g4ba8a190.el8.noarch 36/134 2023-10-14T10:35:01.464 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : cephadm-2:18.0.0-6730.g4ba8a190.el8.noarch 37/134 2023-10-14T10:35:01.464 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : cephfs-top-2:18.0.0-6730.g4ba8a190.el8.noarch 38/134 2023-10-14T10:35:01.464 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-10-14T10:35:01.464 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-10-14T10:35:01.465 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-10-14T10:35:01.465 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-10-14T10:35:01.465 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-10-14T10:35:01.465 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-10-14T10:35:01.465 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-10-14T10:35:01.466 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-10-14T10:35:01.466 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-10-14T10:35:01.466 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-10-14T10:35:01.466 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-10-14T10:35:01.466 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-10-14T10:35:01.467 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-10-14T10:35:01.467 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-10-14T10:35:01.467 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-10-14T10:35:01.467 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-10-14T10:35:01.467 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-10-14T10:35:01.468 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-10-14T10:35:01.468 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-10-14T10:35:01.468 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-10-14T10:35:01.468 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-10-14T10:35:01.468 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-10-14T10:35:01.469 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-10-14T10:35:01.469 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-10-14T10:35:01.469 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-10-14T10:35:01.469 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-10-14T10:35:01.469 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-10-14T10:35:01.469 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-10-14T10:35:01.470 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-10-14T10:35:01.470 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-10-14T10:35:01.470 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-10-14T10:35:01.470 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-10-14T10:35:01.471 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-10-14T10:35:01.471 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-10-14T10:35:01.471 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-10-14T10:35:01.471 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-10-14T10:35:01.472 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-10-14T10:35:01.472 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-10-14T10:35:01.472 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-10-14T10:35:01.473 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-10-14T10:35:01.473 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-10-14T10:35:01.473 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-10-14T10:35:01.473 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-10-14T10:35:01.473 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-10-14T10:35:01.474 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-10-14T10:35:01.474 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-10-14T10:35:01.474 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-10-14T10:35:01.474 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-10-14T10:35:01.475 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-10-14T10:35:01.475 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-10-14T10:35:01.475 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-10-14T10:35:01.475 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-10-14T10:35:01.476 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-10-14T10:35:01.476 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-10-14T10:35:01.476 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-10-14T10:35:01.476 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-10-14T10:35:01.476 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-10-14T10:35:01.477 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-10-14T10:35:01.477 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-10-14T10:35:01.477 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-10-14T10:35:01.477 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-10-14T10:35:01.477 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-10-14T10:35:01.478 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-10-14T10:35:01.478 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-10-14T10:35:01.478 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-10-14T10:35:01.478 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-10-14T10:35:01.478 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-10-14T10:35:01.479 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-10-14T10:35:01.479 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-10-14T10:35:01.479 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-10-14T10:35:01.479 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-10-14T10:35:01.480 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-10-14T10:35:01.480 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-10-14T10:35:01.480 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-10-14T10:35:01.481 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-10-14T10:35:01.481 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-10-14T10:35:01.482 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-10-14T10:35:01.482 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-10-14T10:35:01.482 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-10-14T10:35:01.483 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-10-14T10:35:01.483 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-10-14T10:35:01.483 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-10-14T10:35:01.484 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-10-14T10:35:01.484 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-10-14T10:35:01.484 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-10-14T10:35:01.485 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-10-14T10:35:01.485 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-10-14T10:35:01.485 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-10-14T10:35:01.486 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-10-14T10:35:01.486 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-10-14T10:35:01.486 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : librados2-2:18.0.0-6730.g4ba8a190.el8.x86_64 129/134 2023-10-14T10:35:01.486 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-10-14T10:35:01.487 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : librbd1-2:18.0.0-6730.g4ba8a190.el8.x86_64 131/134 2023-10-14T10:35:01.487 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-14T10:35:01.487 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-10-14T10:35:02.953 INFO:teuthology.orchestra.run.smithi067.stdout:Installed products updated. 2023-10-14T10:35:03.201 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-14T10:35:03.202 INFO:teuthology.orchestra.run.smithi067.stdout:Upgraded: 2023-10-14T10:35:03.202 INFO:teuthology.orchestra.run.smithi067.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-10-14T10:35:03.202 INFO:teuthology.orchestra.run.smithi067.stdout: librados2-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-14T10:35:03.203 INFO:teuthology.orchestra.run.smithi067.stdout: librbd1-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-14T10:35:03.203 INFO:teuthology.orchestra.run.smithi067.stdout:Installed: 2023-10-14T10:35:03.203 INFO:teuthology.orchestra.run.smithi067.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-10-14T10:35:03.203 INFO:teuthology.orchestra.run.smithi067.stdout: automake-1.16.1-7.el8.noarch 2023-10-14T10:35:03.203 INFO:teuthology.orchestra.run.smithi067.stdout: bison-3.0.4-10.el8.x86_64 2023-10-14T10:35:03.204 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-14T10:35:03.204 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-base-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-14T10:35:03.204 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-common-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-14T10:35:03.204 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-fuse-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-14T10:35:03.204 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-grafana-dashboards-2:18.0.0-6730.g4ba8a190.el8.noarch 2023-10-14T10:35:03.205 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-immutable-object-cache-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-14T10:35:03.205 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mds-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-14T10:35:03.205 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-14T10:35:03.205 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-cephadm-2:18.0.0-6730.g4ba8a190.el8.noarch 2023-10-14T10:35:03.205 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-dashboard-2:18.0.0-6730.g4ba8a190.el8.noarch 2023-10-14T10:35:03.206 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-diskprediction-local-2:18.0.0-6730.g4ba8a190.el8.noarch 2023-10-14T10:35:03.206 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-modules-core-2:18.0.0-6730.g4ba8a190.el8.noarch 2023-10-14T10:35:03.206 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-rook-2:18.0.0-6730.g4ba8a190.el8.noarch 2023-10-14T10:35:03.206 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mon-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-14T10:35:03.206 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-osd-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-14T10:35:03.207 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-prometheus-alerts-2:18.0.0-6730.g4ba8a190.el8.noarch 2023-10-14T10:35:03.207 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-radosgw-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-14T10:35:03.207 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-selinux-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-14T10:35:03.207 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-test-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-14T10:35:03.207 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-volume-2:18.0.0-6730.g4ba8a190.el8.noarch 2023-10-14T10:35:03.208 INFO:teuthology.orchestra.run.smithi067.stdout: cephadm-2:18.0.0-6730.g4ba8a190.el8.noarch 2023-10-14T10:35:03.208 INFO:teuthology.orchestra.run.smithi067.stdout: cephfs-mirror-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-14T10:35:03.208 INFO:teuthology.orchestra.run.smithi067.stdout: cephfs-top-2:18.0.0-6730.g4ba8a190.el8.noarch 2023-10-14T10:35:03.208 INFO:teuthology.orchestra.run.smithi067.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-10-14T10:35:03.208 INFO:teuthology.orchestra.run.smithi067.stdout: flex-2.6.1-9.el8.x86_64 2023-10-14T10:35:03.209 INFO:teuthology.orchestra.run.smithi067.stdout: fuse-2.9.7-15.el8.x86_64 2023-10-14T10:35:03.209 INFO:teuthology.orchestra.run.smithi067.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-10-14T10:35:03.209 INFO:teuthology.orchestra.run.smithi067.stdout: jq-1.6-3.el8.x86_64 2023-10-14T10:35:03.209 INFO:teuthology.orchestra.run.smithi067.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-10-14T10:35:03.209 INFO:teuthology.orchestra.run.smithi067.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-10-14T10:35:03.210 INFO:teuthology.orchestra.run.smithi067.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-10-14T10:35:03.210 INFO:teuthology.orchestra.run.smithi067.stdout: libcephfs-devel-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-14T10:35:03.210 INFO:teuthology.orchestra.run.smithi067.stdout: libcephfs2-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-14T10:35:03.210 INFO:teuthology.orchestra.run.smithi067.stdout: libcephsqlite-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-14T10:35:03.210 INFO:teuthology.orchestra.run.smithi067.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-10-14T10:35:03.211 INFO:teuthology.orchestra.run.smithi067.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-10-14T10:35:03.211 INFO:teuthology.orchestra.run.smithi067.stdout: liboath-2.6.2-3.el8.x86_64 2023-10-14T10:35:03.211 INFO:teuthology.orchestra.run.smithi067.stdout: libpq-13.5-1.el8.x86_64 2023-10-14T10:35:03.211 INFO:teuthology.orchestra.run.smithi067.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-10-14T10:35:03.211 INFO:teuthology.orchestra.run.smithi067.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-10-14T10:35:03.212 INFO:teuthology.orchestra.run.smithi067.stdout: librados-devel-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-14T10:35:03.212 INFO:teuthology.orchestra.run.smithi067.stdout: libradosstriper1-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-14T10:35:03.212 INFO:teuthology.orchestra.run.smithi067.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-10-14T10:35:03.212 INFO:teuthology.orchestra.run.smithi067.stdout: librgw2-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-14T10:35:03.212 INFO:teuthology.orchestra.run.smithi067.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-10-14T10:35:03.213 INFO:teuthology.orchestra.run.smithi067.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-10-14T10:35:03.213 INFO:teuthology.orchestra.run.smithi067.stdout: libtool-2.4.6-25.el8.x86_64 2023-10-14T10:35:03.213 INFO:teuthology.orchestra.run.smithi067.stdout: libunwind-1.3.1-3.el8.x86_64 2023-10-14T10:35:03.213 INFO:teuthology.orchestra.run.smithi067.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-10-14T10:35:03.213 INFO:teuthology.orchestra.run.smithi067.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-10-14T10:35:03.214 INFO:teuthology.orchestra.run.smithi067.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-10-14T10:35:03.214 INFO:teuthology.orchestra.run.smithi067.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-10-14T10:35:03.214 INFO:teuthology.orchestra.run.smithi067.stdout: openblas-0.3.15-3.el8.x86_64 2023-10-14T10:35:03.214 INFO:teuthology.orchestra.run.smithi067.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-10-14T10:35:03.214 INFO:teuthology.orchestra.run.smithi067.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-10-14T10:35:03.215 INFO:teuthology.orchestra.run.smithi067.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-10-14T10:35:03.215 INFO:teuthology.orchestra.run.smithi067.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-10-14T10:35:03.215 INFO:teuthology.orchestra.run.smithi067.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-10-14T10:35:03.215 INFO:teuthology.orchestra.run.smithi067.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-14T10:35:03.215 INFO:teuthology.orchestra.run.smithi067.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-14T10:35:03.217 INFO:teuthology.orchestra.run.smithi067.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-14T10:35:03.218 INFO:teuthology.orchestra.run.smithi067.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-10-14T10:35:03.218 INFO:teuthology.orchestra.run.smithi067.stdout: python3-babel-2.5.1-7.el8.noarch 2023-10-14T10:35:03.218 INFO:teuthology.orchestra.run.smithi067.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-10-14T10:35:03.219 INFO:teuthology.orchestra.run.smithi067.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-10-14T10:35:03.219 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-10-14T10:35:03.219 INFO:teuthology.orchestra.run.smithi067.stdout: python3-ceph-argparse-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-14T10:35:03.219 INFO:teuthology.orchestra.run.smithi067.stdout: python3-ceph-common-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-14T10:35:03.220 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cephfs-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-14T10:35:03.220 INFO:teuthology.orchestra.run.smithi067.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-10-14T10:35:03.220 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-10-14T10:35:03.220 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-10-14T10:35:03.221 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-10-14T10:35:03.221 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-10-14T10:35:03.221 INFO:teuthology.orchestra.run.smithi067.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-10-14T10:35:03.221 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco-6.2-6.el8.noarch 2023-10-14T10:35:03.222 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-10-14T10:35:03.222 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-10-14T10:35:03.222 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-10-14T10:35:03.222 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-10-14T10:35:03.223 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-10-14T10:35:03.223 INFO:teuthology.orchestra.run.smithi067.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-10-14T10:35:03.223 INFO:teuthology.orchestra.run.smithi067.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-10-14T10:35:03.223 INFO:teuthology.orchestra.run.smithi067.stdout: python3-mako-1.0.6-13.el8.noarch 2023-10-14T10:35:03.224 INFO:teuthology.orchestra.run.smithi067.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-10-14T10:35:03.224 INFO:teuthology.orchestra.run.smithi067.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-10-14T10:35:03.224 INFO:teuthology.orchestra.run.smithi067.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-10-14T10:35:03.224 INFO:teuthology.orchestra.run.smithi067.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-10-14T10:35:03.224 INFO:teuthology.orchestra.run.smithi067.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-10-14T10:35:03.225 INFO:teuthology.orchestra.run.smithi067.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-10-14T10:35:03.225 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-10-14T10:35:03.225 INFO:teuthology.orchestra.run.smithi067.stdout: python3-portend-2.6-1.el8.noarch 2023-10-14T10:35:03.225 INFO:teuthology.orchestra.run.smithi067.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-10-14T10:35:03.225 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-10-14T10:35:03.226 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-10-14T10:35:03.226 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-10-14T10:35:03.226 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pycparser-2.14-14.el8.noarch 2023-10-14T10:35:03.226 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pytz-2017.2-9.el8.noarch 2023-10-14T10:35:03.227 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rados-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-14T10:35:03.227 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rbd-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-14T10:35:03.227 INFO:teuthology.orchestra.run.smithi067.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-10-14T10:35:03.227 INFO:teuthology.orchestra.run.smithi067.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-10-14T10:35:03.228 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rgw-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-14T10:35:03.228 INFO:teuthology.orchestra.run.smithi067.stdout: python3-routes-2.4.1-12.el8.noarch 2023-10-14T10:35:03.228 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rsa-4.9-2.el8.noarch 2023-10-14T10:35:03.228 INFO:teuthology.orchestra.run.smithi067.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-10-14T10:35:03.228 INFO:teuthology.orchestra.run.smithi067.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-10-14T10:35:03.229 INFO:teuthology.orchestra.run.smithi067.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-10-14T10:35:03.229 INFO:teuthology.orchestra.run.smithi067.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-10-14T10:35:03.229 INFO:teuthology.orchestra.run.smithi067.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-10-14T10:35:03.229 INFO:teuthology.orchestra.run.smithi067.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-10-14T10:35:03.230 INFO:teuthology.orchestra.run.smithi067.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-10-14T10:35:03.230 INFO:teuthology.orchestra.run.smithi067.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-10-14T10:35:03.230 INFO:teuthology.orchestra.run.smithi067.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-10-14T10:35:03.230 INFO:teuthology.orchestra.run.smithi067.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-10-14T10:35:03.230 INFO:teuthology.orchestra.run.smithi067.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-10-14T10:35:03.231 INFO:teuthology.orchestra.run.smithi067.stdout: rbd-fuse-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-14T10:35:03.231 INFO:teuthology.orchestra.run.smithi067.stdout: rbd-mirror-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-14T10:35:03.231 INFO:teuthology.orchestra.run.smithi067.stdout: rbd-nbd-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-14T10:35:03.231 INFO:teuthology.orchestra.run.smithi067.stdout: socat-1.7.4.1-1.el8.x86_64 2023-10-14T10:35:03.231 INFO:teuthology.orchestra.run.smithi067.stdout: thrift-0.13.0-2.el8.x86_64 2023-10-14T10:35:03.232 INFO:teuthology.orchestra.run.smithi067.stdout: uuid-1.6.2-43.el8.x86_64 2023-10-14T10:35:03.232 INFO:teuthology.orchestra.run.smithi067.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-10-14T10:35:03.232 INFO:teuthology.orchestra.run.smithi067.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-10-14T10:35:03.233 INFO:teuthology.orchestra.run.smithi067.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-10-14T10:35:03.233 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-14T10:35:03.233 INFO:teuthology.orchestra.run.smithi067.stdout:Complete! 2023-10-14T10:35:03.383 DEBUG:teuthology.parallel:result is None 2023-10-14T10:35:04.276 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-10-14T10:35:04.276 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:01:58 ago on Sat 14 Oct 2023 10:33:06 AM UTC. 2023-10-14T10:35:06.869 INFO:teuthology.orchestra.run.smithi072.stdout:Installed products updated. 2023-10-14T10:35:07.104 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-14T10:35:07.105 INFO:teuthology.orchestra.run.smithi072.stdout:Upgraded: 2023-10-14T10:35:07.105 INFO:teuthology.orchestra.run.smithi072.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-10-14T10:35:07.105 INFO:teuthology.orchestra.run.smithi072.stdout: librados2-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-14T10:35:07.105 INFO:teuthology.orchestra.run.smithi072.stdout: librbd1-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-14T10:35:07.105 INFO:teuthology.orchestra.run.smithi072.stdout:Installed: 2023-10-14T10:35:07.106 INFO:teuthology.orchestra.run.smithi072.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-10-14T10:35:07.106 INFO:teuthology.orchestra.run.smithi072.stdout: automake-1.16.1-7.el8.noarch 2023-10-14T10:35:07.106 INFO:teuthology.orchestra.run.smithi072.stdout: bison-3.0.4-10.el8.x86_64 2023-10-14T10:35:07.106 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-14T10:35:07.107 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-base-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-14T10:35:07.107 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-common-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-14T10:35:07.107 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-fuse-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-14T10:35:07.107 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-grafana-dashboards-2:18.0.0-6730.g4ba8a190.el8.noarch 2023-10-14T10:35:07.107 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-immutable-object-cache-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-14T10:35:07.108 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mds-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-14T10:35:07.108 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-14T10:35:07.108 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-cephadm-2:18.0.0-6730.g4ba8a190.el8.noarch 2023-10-14T10:35:07.108 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-dashboard-2:18.0.0-6730.g4ba8a190.el8.noarch 2023-10-14T10:35:07.108 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-diskprediction-local-2:18.0.0-6730.g4ba8a190.el8.noarch 2023-10-14T10:35:07.108 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-modules-core-2:18.0.0-6730.g4ba8a190.el8.noarch 2023-10-14T10:35:07.109 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-rook-2:18.0.0-6730.g4ba8a190.el8.noarch 2023-10-14T10:35:07.109 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mon-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-14T10:35:07.109 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-osd-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-14T10:35:07.109 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-prometheus-alerts-2:18.0.0-6730.g4ba8a190.el8.noarch 2023-10-14T10:35:07.109 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-radosgw-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-14T10:35:07.110 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-selinux-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-14T10:35:07.110 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-test-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-14T10:35:07.110 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-volume-2:18.0.0-6730.g4ba8a190.el8.noarch 2023-10-14T10:35:07.110 INFO:teuthology.orchestra.run.smithi072.stdout: cephadm-2:18.0.0-6730.g4ba8a190.el8.noarch 2023-10-14T10:35:07.111 INFO:teuthology.orchestra.run.smithi072.stdout: cephfs-mirror-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-14T10:35:07.111 INFO:teuthology.orchestra.run.smithi072.stdout: cephfs-top-2:18.0.0-6730.g4ba8a190.el8.noarch 2023-10-14T10:35:07.111 INFO:teuthology.orchestra.run.smithi072.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-10-14T10:35:07.111 INFO:teuthology.orchestra.run.smithi072.stdout: flex-2.6.1-9.el8.x86_64 2023-10-14T10:35:07.112 INFO:teuthology.orchestra.run.smithi072.stdout: fuse-2.9.7-15.el8.x86_64 2023-10-14T10:35:07.112 INFO:teuthology.orchestra.run.smithi072.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-10-14T10:35:07.112 INFO:teuthology.orchestra.run.smithi072.stdout: jq-1.6-3.el8.x86_64 2023-10-14T10:35:07.112 INFO:teuthology.orchestra.run.smithi072.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-10-14T10:35:07.112 INFO:teuthology.orchestra.run.smithi072.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-10-14T10:35:07.113 INFO:teuthology.orchestra.run.smithi072.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-10-14T10:35:07.113 INFO:teuthology.orchestra.run.smithi072.stdout: libcephfs-devel-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-14T10:35:07.113 INFO:teuthology.orchestra.run.smithi072.stdout: libcephfs2-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-14T10:35:07.113 INFO:teuthology.orchestra.run.smithi072.stdout: libcephsqlite-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-14T10:35:07.113 INFO:teuthology.orchestra.run.smithi072.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-10-14T10:35:07.113 INFO:teuthology.orchestra.run.smithi072.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-10-14T10:35:07.114 INFO:teuthology.orchestra.run.smithi072.stdout: liboath-2.6.2-3.el8.x86_64 2023-10-14T10:35:07.114 INFO:teuthology.orchestra.run.smithi072.stdout: libpq-13.5-1.el8.x86_64 2023-10-14T10:35:07.114 INFO:teuthology.orchestra.run.smithi072.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-10-14T10:35:07.114 INFO:teuthology.orchestra.run.smithi072.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-10-14T10:35:07.115 INFO:teuthology.orchestra.run.smithi072.stdout: librados-devel-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-14T10:35:07.115 INFO:teuthology.orchestra.run.smithi072.stdout: libradosstriper1-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-14T10:35:07.115 INFO:teuthology.orchestra.run.smithi072.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-10-14T10:35:07.115 INFO:teuthology.orchestra.run.smithi072.stdout: librgw2-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-14T10:35:07.115 INFO:teuthology.orchestra.run.smithi072.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-10-14T10:35:07.115 INFO:teuthology.orchestra.run.smithi072.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-10-14T10:35:07.116 INFO:teuthology.orchestra.run.smithi072.stdout: libtool-2.4.6-25.el8.x86_64 2023-10-14T10:35:07.116 INFO:teuthology.orchestra.run.smithi072.stdout: libunwind-1.3.1-3.el8.x86_64 2023-10-14T10:35:07.116 INFO:teuthology.orchestra.run.smithi072.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-10-14T10:35:07.116 INFO:teuthology.orchestra.run.smithi072.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-10-14T10:35:07.116 INFO:teuthology.orchestra.run.smithi072.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-10-14T10:35:07.117 INFO:teuthology.orchestra.run.smithi072.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-10-14T10:35:07.117 INFO:teuthology.orchestra.run.smithi072.stdout: openblas-0.3.15-3.el8.x86_64 2023-10-14T10:35:07.117 INFO:teuthology.orchestra.run.smithi072.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-10-14T10:35:07.117 INFO:teuthology.orchestra.run.smithi072.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-10-14T10:35:07.117 INFO:teuthology.orchestra.run.smithi072.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-10-14T10:35:07.118 INFO:teuthology.orchestra.run.smithi072.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-10-14T10:35:07.118 INFO:teuthology.orchestra.run.smithi072.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-10-14T10:35:07.118 INFO:teuthology.orchestra.run.smithi072.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-14T10:35:07.118 INFO:teuthology.orchestra.run.smithi072.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-14T10:35:07.118 INFO:teuthology.orchestra.run.smithi072.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-14T10:35:07.119 INFO:teuthology.orchestra.run.smithi072.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-10-14T10:35:07.119 INFO:teuthology.orchestra.run.smithi072.stdout: python3-babel-2.5.1-7.el8.noarch 2023-10-14T10:35:07.119 INFO:teuthology.orchestra.run.smithi072.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-10-14T10:35:07.119 INFO:teuthology.orchestra.run.smithi072.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-10-14T10:35:07.119 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-10-14T10:35:07.119 INFO:teuthology.orchestra.run.smithi072.stdout: python3-ceph-argparse-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-14T10:35:07.120 INFO:teuthology.orchestra.run.smithi072.stdout: python3-ceph-common-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-14T10:35:07.120 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cephfs-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-14T10:35:07.120 INFO:teuthology.orchestra.run.smithi072.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-10-14T10:35:07.121 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-10-14T10:35:07.121 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-10-14T10:35:07.122 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-10-14T10:35:07.122 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-10-14T10:35:07.122 INFO:teuthology.orchestra.run.smithi072.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-10-14T10:35:07.123 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jaraco-6.2-6.el8.noarch 2023-10-14T10:35:07.123 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-10-14T10:35:07.123 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-10-14T10:35:07.123 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-10-14T10:35:07.124 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-10-14T10:35:07.124 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-10-14T10:35:07.124 INFO:teuthology.orchestra.run.smithi072.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-10-14T10:35:07.125 INFO:teuthology.orchestra.run.smithi072.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-10-14T10:35:07.125 INFO:teuthology.orchestra.run.smithi072.stdout: python3-mako-1.0.6-13.el8.noarch 2023-10-14T10:35:07.125 INFO:teuthology.orchestra.run.smithi072.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-10-14T10:35:07.126 INFO:teuthology.orchestra.run.smithi072.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-10-14T10:35:07.126 INFO:teuthology.orchestra.run.smithi072.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-10-14T10:35:07.126 INFO:teuthology.orchestra.run.smithi072.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-10-14T10:35:07.126 INFO:teuthology.orchestra.run.smithi072.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-10-14T10:35:07.127 INFO:teuthology.orchestra.run.smithi072.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-10-14T10:35:07.127 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-10-14T10:35:07.127 INFO:teuthology.orchestra.run.smithi072.stdout: python3-portend-2.6-1.el8.noarch 2023-10-14T10:35:07.127 INFO:teuthology.orchestra.run.smithi072.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-10-14T10:35:07.127 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-10-14T10:35:07.128 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-10-14T10:35:07.128 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-10-14T10:35:07.128 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pycparser-2.14-14.el8.noarch 2023-10-14T10:35:07.128 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pytz-2017.2-9.el8.noarch 2023-10-14T10:35:07.128 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rados-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-14T10:35:07.129 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rbd-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-14T10:35:07.129 INFO:teuthology.orchestra.run.smithi072.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-10-14T10:35:07.129 INFO:teuthology.orchestra.run.smithi072.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-10-14T10:35:07.129 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rgw-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-14T10:35:07.129 INFO:teuthology.orchestra.run.smithi072.stdout: python3-routes-2.4.1-12.el8.noarch 2023-10-14T10:35:07.129 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rsa-4.9-2.el8.noarch 2023-10-14T10:35:07.130 INFO:teuthology.orchestra.run.smithi072.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-10-14T10:35:07.130 INFO:teuthology.orchestra.run.smithi072.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-10-14T10:35:07.130 INFO:teuthology.orchestra.run.smithi072.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-10-14T10:35:07.130 INFO:teuthology.orchestra.run.smithi072.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-10-14T10:35:07.130 INFO:teuthology.orchestra.run.smithi072.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-10-14T10:35:07.131 INFO:teuthology.orchestra.run.smithi072.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-10-14T10:35:07.131 INFO:teuthology.orchestra.run.smithi072.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-10-14T10:35:07.131 INFO:teuthology.orchestra.run.smithi072.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-10-14T10:35:07.131 INFO:teuthology.orchestra.run.smithi072.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-10-14T10:35:07.131 INFO:teuthology.orchestra.run.smithi072.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-10-14T10:35:07.132 INFO:teuthology.orchestra.run.smithi072.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-10-14T10:35:07.132 INFO:teuthology.orchestra.run.smithi072.stdout: rbd-fuse-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-14T10:35:07.132 INFO:teuthology.orchestra.run.smithi072.stdout: rbd-mirror-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-14T10:35:07.132 INFO:teuthology.orchestra.run.smithi072.stdout: rbd-nbd-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-14T10:35:07.132 INFO:teuthology.orchestra.run.smithi072.stdout: socat-1.7.4.1-1.el8.x86_64 2023-10-14T10:35:07.133 INFO:teuthology.orchestra.run.smithi072.stdout: thrift-0.13.0-2.el8.x86_64 2023-10-14T10:35:07.133 INFO:teuthology.orchestra.run.smithi072.stdout: uuid-1.6.2-43.el8.x86_64 2023-10-14T10:35:07.133 INFO:teuthology.orchestra.run.smithi072.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-10-14T10:35:07.133 INFO:teuthology.orchestra.run.smithi072.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-10-14T10:35:07.134 INFO:teuthology.orchestra.run.smithi072.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-10-14T10:35:07.134 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-14T10:35:07.134 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2023-10-14T10:35:07.352 DEBUG:teuthology.parallel:result is None 2023-10-14T10:35:07.353 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 2023-10-14T10:35:07.575 DEBUG:teuthology.orchestra.run.smithi067:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-14T10:35:07.604 INFO:teuthology.orchestra.run.smithi067.stdout:18.0.0-6730.g4ba8a190.el8 2023-10-14T10:35:07.605 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-6730.g4ba8a190.el8 2023-10-14T10:35:07.605 INFO:teuthology.task.install:The correct ceph version 18.0.0-6730.g4ba8a190 is installed. 2023-10-14T10:35:07.606 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 2023-10-14T10:35:07.826 DEBUG:teuthology.orchestra.run.smithi072:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-14T10:35:07.852 INFO:teuthology.orchestra.run.smithi072.stdout:18.0.0-6730.g4ba8a190.el8 2023-10-14T10:35:07.852 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-6730.g4ba8a190.el8 2023-10-14T10:35:07.853 INFO:teuthology.task.install:The correct ceph version 18.0.0-6730.g4ba8a190 is installed. 2023-10-14T10:35:07.853 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-10-14T10:35:07.854 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2023-10-14T10:35:07.854 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-14T10:35:07.904 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-10-14T10:35:07.905 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-14T10:35:07.947 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-10-14T10:35:07.948 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2023-10-14T10:35:07.948 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/usr/bin/daemon-helper 2023-10-14T10:35:07.978 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-14T10:35:08.047 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-10-14T10:35:08.048 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/usr/bin/daemon-helper 2023-10-14T10:35:08.076 DEBUG:teuthology.orchestra.run.smithi072:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-14T10:35:08.143 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-10-14T10:35:08.144 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2023-10-14T10:35:08.144 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-14T10:35:08.173 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-14T10:35:08.242 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-10-14T10:35:08.242 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-14T10:35:08.270 DEBUG:teuthology.orchestra.run.smithi072:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-14T10:35:08.336 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-10-14T10:35:08.337 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2023-10-14T10:35:08.337 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/usr/bin/stdin-killer 2023-10-14T10:35:08.366 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-14T10:35:08.433 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-10-14T10:35:08.433 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/usr/bin/stdin-killer 2023-10-14T10:35:08.460 DEBUG:teuthology.orchestra.run.smithi072:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-14T10:35:08.526 INFO:teuthology.run_tasks:Running task ceph... 2023-10-14T10:35:08.607 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-10-14T10:35:08.608 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod 777 /var/log/ceph 2023-10-14T10:35:08.611 DEBUG:teuthology.orchestra.run.smithi072:> sudo chmod 777 /var/log/ceph 2023-10-14T10:35:08.638 INFO:tasks.ceph:Disabling ceph logrotate... 2023-10-14T10:35:08.639 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-14T10:35:08.680 DEBUG:teuthology.orchestra.run.smithi072:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-14T10:35:08.707 INFO:tasks.ceph:Creating extra log directories... 2023-10-14T10:35:08.708 DEBUG:teuthology.orchestra.run.smithi067:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-14T10:35:08.749 DEBUG:teuthology.orchestra.run.smithi072:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-14T10:35:08.781 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-10-14T10:35:08.782 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-10-14T10:35:08.782 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-10-14T10:35:08.782 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2023-10-14T10:35:08.783 DEBUG:teuthology.orchestra.run.smithi067:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-10-14T10:35:08.832 DEBUG:teuthology.orchestra.run.smithi067:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-10-14T10:35:08.984 DEBUG:teuthology.orchestra.run.smithi067:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-10-14T10:35:09.012 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-10-14T10:35:09.012 DEBUG:teuthology.orchestra.run.smithi072:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-10-14T10:35:09.029 DEBUG:teuthology.orchestra.run.smithi072:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-10-14T10:35:09.169 DEBUG:teuthology.orchestra.run.smithi072:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-10-14T10:35:09.200 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-10-14T10:35:09.200 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-10-14T10:35:09.201 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/rishabh-2023-10-13_18:04:14-fs-rishabh-2023oct13-testing-default-smithi/7425219', 'branch': 'rishabh-2023oct13', 'description': 'fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/basic}}', 'email': None, 'first_in_suite': False, 'job_id': '7425219', 'kernel': {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': '47672119149b103ec7cec2cc65c063ccbb001ac7'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'rishabh-2023-10-13_18:04:14-fs-rishabh-2023oct13-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'rhel', 'os_version': '8.6', 'overrides': {'admin_socket': {'branch': 'rishabh-2023oct13'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0'}}, 'kclient': {'syntax': 'v2'}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'rishabh-2023oct13', 'sha1': '4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0'}}, 'owner': 'scheduled_rishabh@teuthology', 'priority': 75, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 6006, 'sha1': '4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0', 'sleep_before_teardown': 0, 'subset': '402/512', 'suite': 'fs', 'suite_branch': 'rishabh-2023oct13', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi067.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs=', 'smithi072.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': '47672119149b103ec7cec2cc65c063ccbb001ac7'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0', 'cluster': 'ceph'}}, {'kclient': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_volumes.TestVolumes', 'tasks.cephfs.test_volumes.TestSubvolumeGroups', 'tasks.cephfs.test_volumes.TestSubvolumes', 'tasks.cephfs.test_subvolume.TestSubvolume']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '8cdab074dcca9a68965bc5a50e9c30b691949723', 'timestamp': '2023-10-13_18:04:14', 'tube': 'smithi', 'user': 'rishabh', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2830'} 2023-10-14T10:35:09.202 DEBUG:teuthology.orchestra.run.smithi067:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-14T10:35:09.205 DEBUG:teuthology.orchestra.run.smithi072:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-14T10:35:09.252 DEBUG:teuthology.orchestra.run.smithi067:> sudo install -d -m0777 -- /var/run/ceph 2023-10-14T10:35:09.262 DEBUG:teuthology.orchestra.run.smithi072:> sudo install -d -m0777 -- /var/run/ceph 2023-10-14T10:35:09.319 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2023-10-14T10:35:09.320 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/scratch_devs of=/dev/stdout 2023-10-14T10:35:09.345 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-14T10:35:09.345 DEBUG:teuthology.orchestra.run.smithi067:> stat /dev/vg_nvme/lv_1 2023-10-14T10:35:09.402 INFO:teuthology.orchestra.run.smithi067.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-14T10:35:09.402 INFO:teuthology.orchestra.run.smithi067.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-14T10:35:09.403 INFO:teuthology.orchestra.run.smithi067.stdout:Device: 6h/6d Inode: 390715 Links: 1 2023-10-14T10:35:09.403 INFO:teuthology.orchestra.run.smithi067.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-14T10:35:09.403 INFO:teuthology.orchestra.run.smithi067.stdout:Context: system_u:object_r:device_t:s0 2023-10-14T10:35:09.403 INFO:teuthology.orchestra.run.smithi067.stdout:Access: 2023-10-14 10:34:57.282633996 +0000 2023-10-14T10:35:09.403 INFO:teuthology.orchestra.run.smithi067.stdout:Modify: 2023-10-14 10:32:20.223695230 +0000 2023-10-14T10:35:09.404 INFO:teuthology.orchestra.run.smithi067.stdout:Change: 2023-10-14 10:32:20.223695230 +0000 2023-10-14T10:35:09.404 INFO:teuthology.orchestra.run.smithi067.stdout: Birth: - 2023-10-14T10:35:09.404 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-14T10:35:09.468 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records in 2023-10-14T10:35:09.469 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records out 2023-10-14T10:35:09.469 INFO:teuthology.orchestra.run.smithi067.stderr:512 bytes copied, 0.000232195 s, 2.2 MB/s 2023-10-14T10:35:09.470 DEBUG:teuthology.orchestra.run.smithi067:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-14T10:35:09.529 DEBUG:teuthology.orchestra.run.smithi067:> stat /dev/vg_nvme/lv_2 2023-10-14T10:35:09.586 INFO:teuthology.orchestra.run.smithi067.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-14T10:35:09.586 INFO:teuthology.orchestra.run.smithi067.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-14T10:35:09.586 INFO:teuthology.orchestra.run.smithi067.stdout:Device: 6h/6d Inode: 394574 Links: 1 2023-10-14T10:35:09.586 INFO:teuthology.orchestra.run.smithi067.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-14T10:35:09.587 INFO:teuthology.orchestra.run.smithi067.stdout:Context: system_u:object_r:device_t:s0 2023-10-14T10:35:09.587 INFO:teuthology.orchestra.run.smithi067.stdout:Access: 2023-10-14 10:34:57.282633996 +0000 2023-10-14T10:35:09.587 INFO:teuthology.orchestra.run.smithi067.stdout:Modify: 2023-10-14 10:32:20.735684239 +0000 2023-10-14T10:35:09.587 INFO:teuthology.orchestra.run.smithi067.stdout:Change: 2023-10-14 10:32:20.735684239 +0000 2023-10-14T10:35:09.587 INFO:teuthology.orchestra.run.smithi067.stdout: Birth: - 2023-10-14T10:35:09.588 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-14T10:35:09.651 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records in 2023-10-14T10:35:09.652 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records out 2023-10-14T10:35:09.652 INFO:teuthology.orchestra.run.smithi067.stderr:512 bytes copied, 0.000218483 s, 2.3 MB/s 2023-10-14T10:35:09.653 DEBUG:teuthology.orchestra.run.smithi067:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-14T10:35:09.711 DEBUG:teuthology.orchestra.run.smithi067:> stat /dev/vg_nvme/lv_3 2023-10-14T10:35:09.769 INFO:teuthology.orchestra.run.smithi067.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-14T10:35:09.769 INFO:teuthology.orchestra.run.smithi067.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-14T10:35:09.769 INFO:teuthology.orchestra.run.smithi067.stdout:Device: 6h/6d Inode: 396314 Links: 1 2023-10-14T10:35:09.770 INFO:teuthology.orchestra.run.smithi067.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-14T10:35:09.770 INFO:teuthology.orchestra.run.smithi067.stdout:Context: system_u:object_r:device_t:s0 2023-10-14T10:35:09.770 INFO:teuthology.orchestra.run.smithi067.stdout:Access: 2023-10-14 10:34:57.282633996 +0000 2023-10-14T10:35:09.770 INFO:teuthology.orchestra.run.smithi067.stdout:Modify: 2023-10-14 10:32:21.254673098 +0000 2023-10-14T10:35:09.771 INFO:teuthology.orchestra.run.smithi067.stdout:Change: 2023-10-14 10:32:21.254673098 +0000 2023-10-14T10:35:09.771 INFO:teuthology.orchestra.run.smithi067.stdout: Birth: - 2023-10-14T10:35:09.771 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-14T10:35:09.834 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records in 2023-10-14T10:35:09.835 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records out 2023-10-14T10:35:09.835 INFO:teuthology.orchestra.run.smithi067.stderr:512 bytes copied, 0.000223605 s, 2.3 MB/s 2023-10-14T10:35:09.836 DEBUG:teuthology.orchestra.run.smithi067:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-14T10:35:09.893 DEBUG:teuthology.orchestra.run.smithi067:> stat /dev/vg_nvme/lv_4 2023-10-14T10:35:09.951 INFO:teuthology.orchestra.run.smithi067.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-14T10:35:09.951 INFO:teuthology.orchestra.run.smithi067.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-14T10:35:09.952 INFO:teuthology.orchestra.run.smithi067.stdout:Device: 6h/6d Inode: 397331 Links: 1 2023-10-14T10:35:09.952 INFO:teuthology.orchestra.run.smithi067.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-14T10:35:09.952 INFO:teuthology.orchestra.run.smithi067.stdout:Context: system_u:object_r:device_t:s0 2023-10-14T10:35:09.953 INFO:teuthology.orchestra.run.smithi067.stdout:Access: 2023-10-14 10:34:57.282633996 +0000 2023-10-14T10:35:09.953 INFO:teuthology.orchestra.run.smithi067.stdout:Modify: 2023-10-14 10:32:21.853660240 +0000 2023-10-14T10:35:09.953 INFO:teuthology.orchestra.run.smithi067.stdout:Change: 2023-10-14 10:32:21.853660240 +0000 2023-10-14T10:35:09.954 INFO:teuthology.orchestra.run.smithi067.stdout: Birth: - 2023-10-14T10:35:09.954 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-14T10:35:10.016 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records in 2023-10-14T10:35:10.017 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records out 2023-10-14T10:35:10.017 INFO:teuthology.orchestra.run.smithi067.stderr:512 bytes copied, 0.000195759 s, 2.6 MB/s 2023-10-14T10:35:10.019 DEBUG:teuthology.orchestra.run.smithi067:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-14T10:35:10.076 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-14T10:35:10.077 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-10-14T10:35:10.077 DEBUG:teuthology.orchestra.run.smithi072:> dd if=/scratch_devs of=/dev/stdout 2023-10-14T10:35:10.097 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-14T10:35:10.097 DEBUG:teuthology.orchestra.run.smithi072:> stat /dev/vg_nvme/lv_1 2023-10-14T10:35:10.155 INFO:teuthology.orchestra.run.smithi072.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-14T10:35:10.156 INFO:teuthology.orchestra.run.smithi072.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-14T10:35:10.156 INFO:teuthology.orchestra.run.smithi072.stdout:Device: 6h/6d Inode: 390143 Links: 1 2023-10-14T10:35:10.156 INFO:teuthology.orchestra.run.smithi072.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-14T10:35:10.156 INFO:teuthology.orchestra.run.smithi072.stdout:Context: system_u:object_r:device_t:s0 2023-10-14T10:35:10.157 INFO:teuthology.orchestra.run.smithi072.stdout:Access: 2023-10-14 10:35:01.148137174 +0000 2023-10-14T10:35:10.157 INFO:teuthology.orchestra.run.smithi072.stdout:Modify: 2023-10-14 10:32:09.561453038 +0000 2023-10-14T10:35:10.157 INFO:teuthology.orchestra.run.smithi072.stdout:Change: 2023-10-14 10:32:09.561453038 +0000 2023-10-14T10:35:10.157 INFO:teuthology.orchestra.run.smithi072.stdout: Birth: - 2023-10-14T10:35:10.158 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-14T10:35:10.225 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records in 2023-10-14T10:35:10.225 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records out 2023-10-14T10:35:10.225 INFO:teuthology.orchestra.run.smithi072.stderr:512 bytes copied, 0.000240803 s, 2.1 MB/s 2023-10-14T10:35:10.226 DEBUG:teuthology.orchestra.run.smithi072:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-14T10:35:10.286 DEBUG:teuthology.orchestra.run.smithi072:> stat /dev/vg_nvme/lv_2 2023-10-14T10:35:10.345 INFO:teuthology.orchestra.run.smithi072.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-14T10:35:10.345 INFO:teuthology.orchestra.run.smithi072.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-14T10:35:10.346 INFO:teuthology.orchestra.run.smithi072.stdout:Device: 6h/6d Inode: 392487 Links: 1 2023-10-14T10:35:10.346 INFO:teuthology.orchestra.run.smithi072.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-14T10:35:10.346 INFO:teuthology.orchestra.run.smithi072.stdout:Context: system_u:object_r:device_t:s0 2023-10-14T10:35:10.346 INFO:teuthology.orchestra.run.smithi072.stdout:Access: 2023-10-14 10:35:01.148137174 +0000 2023-10-14T10:35:10.346 INFO:teuthology.orchestra.run.smithi072.stdout:Modify: 2023-10-14 10:32:10.045443176 +0000 2023-10-14T10:35:10.347 INFO:teuthology.orchestra.run.smithi072.stdout:Change: 2023-10-14 10:32:10.045443176 +0000 2023-10-14T10:35:10.347 INFO:teuthology.orchestra.run.smithi072.stdout: Birth: - 2023-10-14T10:35:10.347 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-14T10:35:10.413 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records in 2023-10-14T10:35:10.413 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records out 2023-10-14T10:35:10.413 INFO:teuthology.orchestra.run.smithi072.stderr:512 bytes copied, 0.000231031 s, 2.2 MB/s 2023-10-14T10:35:10.414 DEBUG:teuthology.orchestra.run.smithi072:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-14T10:35:10.473 DEBUG:teuthology.orchestra.run.smithi072:> stat /dev/vg_nvme/lv_3 2023-10-14T10:35:10.531 INFO:teuthology.orchestra.run.smithi072.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-14T10:35:10.531 INFO:teuthology.orchestra.run.smithi072.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-14T10:35:10.532 INFO:teuthology.orchestra.run.smithi072.stdout:Device: 6h/6d Inode: 390755 Links: 1 2023-10-14T10:35:10.532 INFO:teuthology.orchestra.run.smithi072.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-14T10:35:10.532 INFO:teuthology.orchestra.run.smithi072.stdout:Context: system_u:object_r:device_t:s0 2023-10-14T10:35:10.532 INFO:teuthology.orchestra.run.smithi072.stdout:Access: 2023-10-14 10:35:01.148137174 +0000 2023-10-14T10:35:10.532 INFO:teuthology.orchestra.run.smithi072.stdout:Modify: 2023-10-14 10:32:10.536433173 +0000 2023-10-14T10:35:10.533 INFO:teuthology.orchestra.run.smithi072.stdout:Change: 2023-10-14 10:32:10.536433173 +0000 2023-10-14T10:35:10.533 INFO:teuthology.orchestra.run.smithi072.stdout: Birth: - 2023-10-14T10:35:10.533 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-14T10:35:10.598 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records in 2023-10-14T10:35:10.598 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records out 2023-10-14T10:35:10.598 INFO:teuthology.orchestra.run.smithi072.stderr:512 bytes copied, 0.000230227 s, 2.2 MB/s 2023-10-14T10:35:10.599 DEBUG:teuthology.orchestra.run.smithi072:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-14T10:35:10.658 DEBUG:teuthology.orchestra.run.smithi072:> stat /dev/vg_nvme/lv_4 2023-10-14T10:35:10.715 INFO:teuthology.orchestra.run.smithi072.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-14T10:35:10.715 INFO:teuthology.orchestra.run.smithi072.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-14T10:35:10.716 INFO:teuthology.orchestra.run.smithi072.stdout:Device: 6h/6d Inode: 390823 Links: 1 2023-10-14T10:35:10.716 INFO:teuthology.orchestra.run.smithi072.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-14T10:35:10.716 INFO:teuthology.orchestra.run.smithi072.stdout:Context: system_u:object_r:device_t:s0 2023-10-14T10:35:10.716 INFO:teuthology.orchestra.run.smithi072.stdout:Access: 2023-10-14 10:35:01.148137174 +0000 2023-10-14T10:35:10.716 INFO:teuthology.orchestra.run.smithi072.stdout:Modify: 2023-10-14 10:32:11.023423251 +0000 2023-10-14T10:35:10.717 INFO:teuthology.orchestra.run.smithi072.stdout:Change: 2023-10-14 10:32:11.023423251 +0000 2023-10-14T10:35:10.717 INFO:teuthology.orchestra.run.smithi072.stdout: Birth: - 2023-10-14T10:35:10.717 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-14T10:35:10.781 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records in 2023-10-14T10:35:10.782 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records out 2023-10-14T10:35:10.782 INFO:teuthology.orchestra.run.smithi072.stderr:512 bytes copied, 0.000199764 s, 2.6 MB/s 2023-10-14T10:35:10.783 DEBUG:teuthology.orchestra.run.smithi072:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-14T10:35:10.841 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-14T10:35:10.842 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi067.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi072.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-10-14T10:35:10.842 INFO:tasks.ceph:Generating config... 2023-10-14T10:35:10.843 INFO:tasks.ceph:[client] client mount timeout = 600 2023-10-14T10:35:10.844 INFO:tasks.ceph:[client] debug client = 20 2023-10-14T10:35:10.844 INFO:tasks.ceph:[client] debug ms = 1 2023-10-14T10:35:10.844 INFO:tasks.ceph:[client] pid file = 2023-10-14T10:35:10.844 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-10-14T10:35:10.845 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-10-14T10:35:10.845 INFO:tasks.ceph:[global] ms die on skipped message = False 2023-10-14T10:35:10.845 INFO:tasks.ceph:[mds] debug mds = 20 2023-10-14T10:35:10.845 INFO:tasks.ceph:[mds] debug mds balancer = 20 2023-10-14T10:35:10.846 INFO:tasks.ceph:[mds] debug ms = 1 2023-10-14T10:35:10.846 INFO:tasks.ceph:[mds] mds debug frag = True 2023-10-14T10:35:10.846 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-10-14T10:35:10.847 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-10-14T10:35:10.847 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-10-14T10:35:10.847 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-10-14T10:35:10.847 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-10-14T10:35:10.848 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-10-14T10:35:10.848 INFO:tasks.ceph:[mgr] debug client = 20 2023-10-14T10:35:10.848 INFO:tasks.ceph:[mgr] debug finisher = 20 2023-10-14T10:35:10.848 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-10-14T10:35:10.848 INFO:tasks.ceph:[mgr] debug ms = 1 2023-10-14T10:35:10.849 INFO:tasks.ceph:[mon] debug mon = 20 2023-10-14T10:35:10.849 INFO:tasks.ceph:[mon] debug ms = 1 2023-10-14T10:35:10.849 INFO:tasks.ceph:[mon] debug paxos = 20 2023-10-14T10:35:10.849 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-10-14T10:35:10.850 INFO:tasks.ceph:[osd] bdev async discard = True 2023-10-14T10:35:10.850 INFO:tasks.ceph:[osd] bdev enable discard = True 2023-10-14T10:35:10.850 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2023-10-14T10:35:10.851 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-10-14T10:35:10.851 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-10-14T10:35:10.851 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2023-10-14T10:35:10.851 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2023-10-14T10:35:10.851 INFO:tasks.ceph:[osd] debug ms = 1 2023-10-14T10:35:10.852 INFO:tasks.ceph:[osd] debug osd = 20 2023-10-14T10:35:10.852 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2023-10-14T10:35:10.852 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-10-14T10:35:10.853 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-10-14T10:35:10.853 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-10-14T10:35:10.853 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-10-14T10:35:10.853 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-10-14T10:35:10.854 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-10-14T10:35:10.854 INFO:tasks.ceph:Setting up mon.a... 2023-10-14T10:35:10.854 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-10-14T10:35:10.911 INFO:teuthology.orchestra.run.smithi067.stdout:creating /etc/ceph/ceph.keyring 2023-10-14T10:35:10.914 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-10-14T10:35:10.964 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-14T10:35:10.993 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.67'), ('mon.b', '172.21.15.72'), ('mon.c', '[v2:172.21.15.72:3301,v1:172.21.15.72:6790]')] 2023-10-14T10:35:10.994 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.67,172.21.15.72,[v2:172.21.15.72:3301,v1:172.21.15.72:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20, 'debug finisher': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-10-14T10:35:10.995 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2023-10-14T10:35:10.995 DEBUG:teuthology.orchestra.run.smithi067:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-10-14T10:35:11.050 DEBUG:teuthology.orchestra.run.smithi067:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.67 --add b 172.21.15.72 --addv c '[v2:172.21.15.72:3301,v1:172.21.15.72:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-10-14T10:35:11.142 INFO:teuthology.orchestra.run.smithi067.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-10-14T10:35:11.142 INFO:teuthology.orchestra.run.smithi067.stdout:monmaptool: generated fsid 37f9ab46-3082-4b71-8b24-08d39897b008 2023-10-14T10:35:11.142 INFO:teuthology.orchestra.run.smithi067.stdout:setting min_mon_release = pacific 2023-10-14T10:35:11.143 INFO:teuthology.orchestra.run.smithi067.stdout:epoch 0 2023-10-14T10:35:11.143 INFO:teuthology.orchestra.run.smithi067.stdout:fsid 37f9ab46-3082-4b71-8b24-08d39897b008 2023-10-14T10:35:11.143 INFO:teuthology.orchestra.run.smithi067.stdout:last_changed 2023-10-14T10:35:11.141338+0000 2023-10-14T10:35:11.143 INFO:teuthology.orchestra.run.smithi067.stdout:created 2023-10-14T10:35:11.141338+0000 2023-10-14T10:35:11.144 INFO:teuthology.orchestra.run.smithi067.stdout:min_mon_release 16 (pacific) 2023-10-14T10:35:11.144 INFO:teuthology.orchestra.run.smithi067.stdout:election_strategy: 1 2023-10-14T10:35:11.144 INFO:teuthology.orchestra.run.smithi067.stdout:0: [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] mon.a 2023-10-14T10:35:11.144 INFO:teuthology.orchestra.run.smithi067.stdout:1: [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] mon.b 2023-10-14T10:35:11.144 INFO:teuthology.orchestra.run.smithi067.stdout:2: [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] mon.c 2023-10-14T10:35:11.145 INFO:teuthology.orchestra.run.smithi067.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-10-14T10:35:11.146 DEBUG:teuthology.orchestra.run.smithi067:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-10-14T10:35:11.163 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 37f9ab46-3082-4b71-8b24-08d39897b008... 2023-10-14T10:35:11.164 DEBUG:teuthology.orchestra.run.smithi067:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-14T10:35:11.205 DEBUG:teuthology.orchestra.run.smithi072:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-14T10:35:11.249 INFO:teuthology.orchestra.run.smithi067.stdout:[global] 2023-10-14T10:35:11.250 INFO:teuthology.orchestra.run.smithi067.stdout: chdir = "" 2023-10-14T10:35:11.250 INFO:teuthology.orchestra.run.smithi067.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-14T10:35:11.250 INFO:teuthology.orchestra.run.smithi067.stdout: auth supported = cephx 2023-10-14T10:35:11.250 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-14T10:35:11.250 INFO:teuthology.orchestra.run.smithi067.stdout: filestore xattr use omap = true 2023-10-14T10:35:11.251 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-14T10:35:11.251 INFO:teuthology.orchestra.run.smithi067.stdout: mon clock drift allowed = 1.000 2023-10-14T10:35:11.251 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-14T10:35:11.251 INFO:teuthology.orchestra.run.smithi067.stdout: osd crush chooseleaf type = 0 2023-10-14T10:35:11.251 INFO:teuthology.orchestra.run.smithi067.stdout: auth debug = true 2023-10-14T10:35:11.252 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-14T10:35:11.252 INFO:teuthology.orchestra.run.smithi067.stdout: ms die on old message = true 2023-10-14T10:35:11.252 INFO:teuthology.orchestra.run.smithi067.stdout: ms die on bug = true 2023-10-14T10:35:11.252 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-14T10:35:11.252 INFO:teuthology.orchestra.run.smithi067.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-14T10:35:11.253 INFO:teuthology.orchestra.run.smithi067.stdout: mon pg warn max object skew = 0 2023-10-14T10:35:11.253 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-14T10:35:11.253 INFO:teuthology.orchestra.run.smithi067.stdout: # disable pg_autoscaler by default for new pools 2023-10-14T10:35:11.253 INFO:teuthology.orchestra.run.smithi067.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-14T10:35:11.253 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-14T10:35:11.254 INFO:teuthology.orchestra.run.smithi067.stdout: osd pool default size = 2 2023-10-14T10:35:11.254 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-14T10:35:11.254 INFO:teuthology.orchestra.run.smithi067.stdout: mon osd allow primary affinity = true 2023-10-14T10:35:11.254 INFO:teuthology.orchestra.run.smithi067.stdout: mon osd allow pg remap = true 2023-10-14T10:35:11.254 INFO:teuthology.orchestra.run.smithi067.stdout: mon warn on legacy crush tunables = false 2023-10-14T10:35:11.255 INFO:teuthology.orchestra.run.smithi067.stdout: mon warn on crush straw calc version zero = false 2023-10-14T10:35:11.255 INFO:teuthology.orchestra.run.smithi067.stdout: mon warn on no sortbitwise = false 2023-10-14T10:35:11.255 INFO:teuthology.orchestra.run.smithi067.stdout: mon warn on osd down out interval zero = false 2023-10-14T10:35:11.255 INFO:teuthology.orchestra.run.smithi067.stdout: mon warn on too few osds = false 2023-10-14T10:35:11.255 INFO:teuthology.orchestra.run.smithi067.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-14T10:35:11.256 INFO:teuthology.orchestra.run.smithi067.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-14T10:35:11.256 INFO:teuthology.orchestra.run.smithi067.stdout: mon_allow_pool_size_one = true 2023-10-14T10:35:11.256 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-14T10:35:11.256 INFO:teuthology.orchestra.run.smithi067.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-14T10:35:11.257 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-14T10:35:11.257 INFO:teuthology.orchestra.run.smithi067.stdout: osd default data pool replay window = 5 2023-10-14T10:35:11.257 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-14T10:35:11.257 INFO:teuthology.orchestra.run.smithi067.stdout: mon allow pool delete = true 2023-10-14T10:35:11.257 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-14T10:35:11.257 INFO:teuthology.orchestra.run.smithi067.stdout: mon cluster log file level = debug 2023-10-14T10:35:11.258 INFO:teuthology.orchestra.run.smithi067.stdout: debug asserts on shutdown = true 2023-10-14T10:35:11.258 INFO:teuthology.orchestra.run.smithi067.stdout: mon health detail to clog = false 2023-10-14T10:35:11.258 INFO:teuthology.orchestra.run.smithi067.stdout: mon host = "172.21.15.67,172.21.15.72,[v2:172.21.15.72:3301,v1:172.21.15.72:6790]" 2023-10-14T10:35:11.258 INFO:teuthology.orchestra.run.smithi067.stdout: ms die on skipped message = False 2023-10-14T10:35:11.258 INFO:teuthology.orchestra.run.smithi067.stdout: fsid = 37f9ab46-3082-4b71-8b24-08d39897b008 2023-10-14T10:35:11.259 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-14T10:35:11.259 INFO:teuthology.orchestra.run.smithi067.stdout:[osd] 2023-10-14T10:35:11.259 INFO:teuthology.orchestra.run.smithi067.stdout: osd journal size = 100 2023-10-14T10:35:11.259 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-14T10:35:11.259 INFO:teuthology.orchestra.run.smithi067.stdout: osd scrub load threshold = 5.0 2023-10-14T10:35:11.260 INFO:teuthology.orchestra.run.smithi067.stdout: osd scrub max interval = 600 2023-10-14T10:35:11.260 INFO:teuthology.orchestra.run.smithi067.stdout: osd mclock profile = high_recovery_ops 2023-10-14T10:35:11.260 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-14T10:35:11.260 INFO:teuthology.orchestra.run.smithi067.stdout: osd recover clone overlap = true 2023-10-14T10:35:11.260 INFO:teuthology.orchestra.run.smithi067.stdout: osd recovery max chunk = 1048576 2023-10-14T10:35:11.261 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-14T10:35:11.261 INFO:teuthology.orchestra.run.smithi067.stdout: osd debug shutdown = true 2023-10-14T10:35:11.261 INFO:teuthology.orchestra.run.smithi067.stdout: osd debug op order = true 2023-10-14T10:35:11.261 INFO:teuthology.orchestra.run.smithi067.stdout: osd debug verify stray on activate = true 2023-10-14T10:35:11.261 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-14T10:35:11.261 INFO:teuthology.orchestra.run.smithi067.stdout: osd open classes on start = true 2023-10-14T10:35:11.262 INFO:teuthology.orchestra.run.smithi067.stdout: osd debug pg log writeout = true 2023-10-14T10:35:11.262 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-14T10:35:11.262 INFO:teuthology.orchestra.run.smithi067.stdout: osd deep scrub update digest min age = 30 2023-10-14T10:35:11.262 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-14T10:35:11.262 INFO:teuthology.orchestra.run.smithi067.stdout: osd map max advance = 10 2023-10-14T10:35:11.263 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-14T10:35:11.263 INFO:teuthology.orchestra.run.smithi067.stdout: journal zero on create = true 2023-10-14T10:35:11.263 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-14T10:35:11.263 INFO:teuthology.orchestra.run.smithi067.stdout: filestore ondisk finisher threads = 3 2023-10-14T10:35:11.263 INFO:teuthology.orchestra.run.smithi067.stdout: filestore apply finisher threads = 3 2023-10-14T10:35:11.264 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-14T10:35:11.264 INFO:teuthology.orchestra.run.smithi067.stdout: bdev debug aio = true 2023-10-14T10:35:11.264 INFO:teuthology.orchestra.run.smithi067.stdout: osd debug misdirected ops = true 2023-10-14T10:35:11.264 INFO:teuthology.orchestra.run.smithi067.stdout: bdev async discard = True 2023-10-14T10:35:11.264 INFO:teuthology.orchestra.run.smithi067.stdout: bdev enable discard = True 2023-10-14T10:35:11.264 INFO:teuthology.orchestra.run.smithi067.stdout: bluestore allocator = bitmap 2023-10-14T10:35:11.265 INFO:teuthology.orchestra.run.smithi067.stdout: bluestore block size = 96636764160 2023-10-14T10:35:11.265 INFO:teuthology.orchestra.run.smithi067.stdout: bluestore fsck on mount = True 2023-10-14T10:35:11.265 INFO:teuthology.orchestra.run.smithi067.stdout: debug bluefs = 1/20 2023-10-14T10:35:11.265 INFO:teuthology.orchestra.run.smithi067.stdout: debug bluestore = 1/20 2023-10-14T10:35:11.265 INFO:teuthology.orchestra.run.smithi067.stdout: debug ms = 1 2023-10-14T10:35:11.266 INFO:teuthology.orchestra.run.smithi067.stdout: debug osd = 20 2023-10-14T10:35:11.266 INFO:teuthology.orchestra.run.smithi067.stdout: debug rocksdb = 4/10 2023-10-14T10:35:11.266 INFO:teuthology.orchestra.run.smithi067.stdout: mon osd backfillfull_ratio = 0.85 2023-10-14T10:35:11.266 INFO:teuthology.orchestra.run.smithi067.stdout: mon osd full ratio = 0.9 2023-10-14T10:35:11.266 INFO:teuthology.orchestra.run.smithi067.stdout: mon osd nearfull ratio = 0.8 2023-10-14T10:35:11.266 INFO:teuthology.orchestra.run.smithi067.stdout: osd failsafe full ratio = 0.95 2023-10-14T10:35:11.267 INFO:teuthology.orchestra.run.smithi067.stdout: osd objectstore = bluestore 2023-10-14T10:35:11.267 INFO:teuthology.orchestra.run.smithi067.stdout: osd op complaint time = 180 2023-10-14T10:35:11.267 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-14T10:35:11.267 INFO:teuthology.orchestra.run.smithi067.stdout:[mgr] 2023-10-14T10:35:11.267 INFO:teuthology.orchestra.run.smithi067.stdout: debug ms = 1 2023-10-14T10:35:11.268 INFO:teuthology.orchestra.run.smithi067.stdout: debug mgr = 20 2023-10-14T10:35:11.268 INFO:teuthology.orchestra.run.smithi067.stdout: debug mon = 20 2023-10-14T10:35:11.268 INFO:teuthology.orchestra.run.smithi067.stdout: debug auth = 20 2023-10-14T10:35:11.268 INFO:teuthology.orchestra.run.smithi067.stdout: mon reweight min pgs per osd = 4 2023-10-14T10:35:11.269 INFO:teuthology.orchestra.run.smithi067.stdout: mon reweight min bytes per osd = 10 2023-10-14T10:35:11.269 INFO:teuthology.orchestra.run.smithi067.stdout: mgr/telemetry/nag = false 2023-10-14T10:35:11.269 INFO:teuthology.orchestra.run.smithi067.stdout: debug client = 20 2023-10-14T10:35:11.269 INFO:teuthology.orchestra.run.smithi067.stdout: debug finisher = 20 2023-10-14T10:35:11.269 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-14T10:35:11.270 INFO:teuthology.orchestra.run.smithi067.stdout:[mon] 2023-10-14T10:35:11.270 INFO:teuthology.orchestra.run.smithi067.stdout: debug ms = 1 2023-10-14T10:35:11.270 INFO:teuthology.orchestra.run.smithi067.stdout: debug mon = 20 2023-10-14T10:35:11.270 INFO:teuthology.orchestra.run.smithi067.stdout: debug paxos = 20 2023-10-14T10:35:11.271 INFO:teuthology.orchestra.run.smithi067.stdout: debug auth = 20 2023-10-14T10:35:11.271 INFO:teuthology.orchestra.run.smithi067.stdout: mon data avail warn = 5 2023-10-14T10:35:11.271 INFO:teuthology.orchestra.run.smithi067.stdout: mon mgr mkfs grace = 240 2023-10-14T10:35:11.271 INFO:teuthology.orchestra.run.smithi067.stdout: mon reweight min pgs per osd = 4 2023-10-14T10:35:11.271 INFO:teuthology.orchestra.run.smithi067.stdout: mon osd reporter subtree level = osd 2023-10-14T10:35:11.272 INFO:teuthology.orchestra.run.smithi067.stdout: mon osd prime pg temp = true 2023-10-14T10:35:11.272 INFO:teuthology.orchestra.run.smithi067.stdout: mon reweight min bytes per osd = 10 2023-10-14T10:35:11.272 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-14T10:35:11.272 INFO:teuthology.orchestra.run.smithi067.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-14T10:35:11.272 INFO:teuthology.orchestra.run.smithi067.stdout: auth mon ticket ttl = 660 # 11m 2023-10-14T10:35:11.273 INFO:teuthology.orchestra.run.smithi067.stdout: auth service ticket ttl = 240 # 4m 2023-10-14T10:35:11.273 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-14T10:35:11.273 INFO:teuthology.orchestra.run.smithi067.stdout: # don't complain about insecure global_id in the test suite 2023-10-14T10:35:11.273 INFO:teuthology.orchestra.run.smithi067.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-14T10:35:11.273 INFO:teuthology.orchestra.run.smithi067.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-14T10:35:11.274 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-14T10:35:11.274 INFO:teuthology.orchestra.run.smithi067.stdout: # 1m isn't quite enough 2023-10-14T10:35:11.274 INFO:teuthology.orchestra.run.smithi067.stdout: mon_down_mkfs_grace = 2m 2023-10-14T10:35:11.274 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-14T10:35:11.274 INFO:teuthology.orchestra.run.smithi067.stdout: mon_warn_on_filestore_osds = false 2023-10-14T10:35:11.275 INFO:teuthology.orchestra.run.smithi067.stdout: mon op complaint time = 120 2023-10-14T10:35:11.275 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-14T10:35:11.275 INFO:teuthology.orchestra.run.smithi067.stdout:[client] 2023-10-14T10:35:11.275 INFO:teuthology.orchestra.run.smithi067.stdout: rgw cache enabled = true 2023-10-14T10:35:11.276 INFO:teuthology.orchestra.run.smithi067.stdout: rgw enable ops log = true 2023-10-14T10:35:11.276 INFO:teuthology.orchestra.run.smithi067.stdout: rgw enable usage log = true 2023-10-14T10:35:11.276 INFO:teuthology.orchestra.run.smithi067.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-14T10:35:11.276 INFO:teuthology.orchestra.run.smithi067.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-14T10:35:11.276 INFO:teuthology.orchestra.run.smithi067.stdout: client mount timeout = 600 2023-10-14T10:35:11.277 INFO:teuthology.orchestra.run.smithi067.stdout: debug client = 20 2023-10-14T10:35:11.277 INFO:teuthology.orchestra.run.smithi067.stdout: debug ms = 1 2023-10-14T10:35:11.277 INFO:teuthology.orchestra.run.smithi067.stdout: pid file = "" 2023-10-14T10:35:11.277 INFO:teuthology.orchestra.run.smithi067.stdout: rados mon op timeout = 900 2023-10-14T10:35:11.277 INFO:teuthology.orchestra.run.smithi067.stdout: rados osd op timeout = 900 2023-10-14T10:35:11.278 INFO:teuthology.orchestra.run.smithi067.stdout:[mon.a] 2023-10-14T10:35:11.278 INFO:teuthology.orchestra.run.smithi067.stdout:[mon.b] 2023-10-14T10:35:11.278 INFO:teuthology.orchestra.run.smithi067.stdout:[mon.c] 2023-10-14T10:35:11.278 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.a] 2023-10-14T10:35:11.278 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.c] 2023-10-14T10:35:11.278 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.b] 2023-10-14T10:35:11.279 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.d] 2023-10-14T10:35:11.279 INFO:teuthology.orchestra.run.smithi067.stdout:[mds] 2023-10-14T10:35:11.279 INFO:teuthology.orchestra.run.smithi067.stdout: debug mds = 20 2023-10-14T10:35:11.279 INFO:teuthology.orchestra.run.smithi067.stdout: debug mds balancer = 20 2023-10-14T10:35:11.279 INFO:teuthology.orchestra.run.smithi067.stdout: debug ms = 1 2023-10-14T10:35:11.280 INFO:teuthology.orchestra.run.smithi067.stdout: mds debug frag = True 2023-10-14T10:35:11.280 INFO:teuthology.orchestra.run.smithi067.stdout: mds debug scatterstat = True 2023-10-14T10:35:11.280 INFO:teuthology.orchestra.run.smithi067.stdout: mds op complaint time = 180 2023-10-14T10:35:11.280 INFO:teuthology.orchestra.run.smithi067.stdout: mds verify scatter = True 2023-10-14T10:35:11.280 INFO:teuthology.orchestra.run.smithi067.stdout: osd op complaint time = 180 2023-10-14T10:35:11.280 INFO:teuthology.orchestra.run.smithi067.stdout: rados mon op timeout = 900 2023-10-14T10:35:11.281 INFO:teuthology.orchestra.run.smithi067.stdout: rados osd op timeout = 900 2023-10-14T10:35:11.282 INFO:teuthology.orchestra.run.smithi072.stdout:[global] 2023-10-14T10:35:11.282 INFO:teuthology.orchestra.run.smithi072.stdout: chdir = "" 2023-10-14T10:35:11.283 INFO:teuthology.orchestra.run.smithi072.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-14T10:35:11.283 INFO:teuthology.orchestra.run.smithi072.stdout: auth supported = cephx 2023-10-14T10:35:11.283 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-14T10:35:11.283 INFO:teuthology.orchestra.run.smithi072.stdout: filestore xattr use omap = true 2023-10-14T10:35:11.283 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-14T10:35:11.284 INFO:teuthology.orchestra.run.smithi072.stdout: mon clock drift allowed = 1.000 2023-10-14T10:35:11.284 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-14T10:35:11.284 INFO:teuthology.orchestra.run.smithi072.stdout: osd crush chooseleaf type = 0 2023-10-14T10:35:11.284 INFO:teuthology.orchestra.run.smithi072.stdout: auth debug = true 2023-10-14T10:35:11.284 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-14T10:35:11.285 INFO:teuthology.orchestra.run.smithi072.stdout: ms die on old message = true 2023-10-14T10:35:11.285 INFO:teuthology.orchestra.run.smithi072.stdout: ms die on bug = true 2023-10-14T10:35:11.285 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-14T10:35:11.285 INFO:teuthology.orchestra.run.smithi072.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-14T10:35:11.285 INFO:teuthology.orchestra.run.smithi072.stdout: mon pg warn max object skew = 0 2023-10-14T10:35:11.286 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-14T10:35:11.286 INFO:teuthology.orchestra.run.smithi072.stdout: # disable pg_autoscaler by default for new pools 2023-10-14T10:35:11.286 INFO:teuthology.orchestra.run.smithi072.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-14T10:35:11.286 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-14T10:35:11.286 INFO:teuthology.orchestra.run.smithi072.stdout: osd pool default size = 2 2023-10-14T10:35:11.287 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-14T10:35:11.287 INFO:teuthology.orchestra.run.smithi072.stdout: mon osd allow primary affinity = true 2023-10-14T10:35:11.287 INFO:teuthology.orchestra.run.smithi072.stdout: mon osd allow pg remap = true 2023-10-14T10:35:11.287 INFO:teuthology.orchestra.run.smithi072.stdout: mon warn on legacy crush tunables = false 2023-10-14T10:35:11.287 INFO:teuthology.orchestra.run.smithi072.stdout: mon warn on crush straw calc version zero = false 2023-10-14T10:35:11.288 INFO:teuthology.orchestra.run.smithi072.stdout: mon warn on no sortbitwise = false 2023-10-14T10:35:11.288 INFO:teuthology.orchestra.run.smithi072.stdout: mon warn on osd down out interval zero = false 2023-10-14T10:35:11.288 INFO:teuthology.orchestra.run.smithi072.stdout: mon warn on too few osds = false 2023-10-14T10:35:11.288 INFO:teuthology.orchestra.run.smithi072.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-14T10:35:11.288 INFO:teuthology.orchestra.run.smithi072.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-14T10:35:11.289 INFO:teuthology.orchestra.run.smithi072.stdout: mon_allow_pool_size_one = true 2023-10-14T10:35:11.289 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-14T10:35:11.289 INFO:teuthology.orchestra.run.smithi072.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-14T10:35:11.289 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-14T10:35:11.289 INFO:teuthology.orchestra.run.smithi072.stdout: osd default data pool replay window = 5 2023-10-14T10:35:11.290 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-14T10:35:11.290 INFO:teuthology.orchestra.run.smithi072.stdout: mon allow pool delete = true 2023-10-14T10:35:11.290 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-14T10:35:11.290 INFO:teuthology.orchestra.run.smithi072.stdout: mon cluster log file level = debug 2023-10-14T10:35:11.290 INFO:teuthology.orchestra.run.smithi072.stdout: debug asserts on shutdown = true 2023-10-14T10:35:11.291 INFO:teuthology.orchestra.run.smithi072.stdout: mon health detail to clog = false 2023-10-14T10:35:11.291 INFO:teuthology.orchestra.run.smithi072.stdout: mon host = "172.21.15.67,172.21.15.72,[v2:172.21.15.72:3301,v1:172.21.15.72:6790]" 2023-10-14T10:35:11.291 INFO:teuthology.orchestra.run.smithi072.stdout: ms die on skipped message = False 2023-10-14T10:35:11.291 INFO:teuthology.orchestra.run.smithi072.stdout: fsid = 37f9ab46-3082-4b71-8b24-08d39897b008 2023-10-14T10:35:11.291 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-14T10:35:11.291 INFO:teuthology.orchestra.run.smithi072.stdout:[osd] 2023-10-14T10:35:11.292 INFO:teuthology.orchestra.run.smithi072.stdout: osd journal size = 100 2023-10-14T10:35:11.292 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-14T10:35:11.292 INFO:teuthology.orchestra.run.smithi072.stdout: osd scrub load threshold = 5.0 2023-10-14T10:35:11.292 INFO:teuthology.orchestra.run.smithi072.stdout: osd scrub max interval = 600 2023-10-14T10:35:11.292 INFO:teuthology.orchestra.run.smithi072.stdout: osd mclock profile = high_recovery_ops 2023-10-14T10:35:11.293 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-14T10:35:11.293 INFO:teuthology.orchestra.run.smithi072.stdout: osd recover clone overlap = true 2023-10-14T10:35:11.293 INFO:teuthology.orchestra.run.smithi072.stdout: osd recovery max chunk = 1048576 2023-10-14T10:35:11.293 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-14T10:35:11.293 INFO:teuthology.orchestra.run.smithi072.stdout: osd debug shutdown = true 2023-10-14T10:35:11.293 INFO:teuthology.orchestra.run.smithi072.stdout: osd debug op order = true 2023-10-14T10:35:11.294 INFO:teuthology.orchestra.run.smithi072.stdout: osd debug verify stray on activate = true 2023-10-14T10:35:11.294 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-14T10:35:11.294 INFO:teuthology.orchestra.run.smithi072.stdout: osd open classes on start = true 2023-10-14T10:35:11.294 INFO:teuthology.orchestra.run.smithi072.stdout: osd debug pg log writeout = true 2023-10-14T10:35:11.294 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-14T10:35:11.295 INFO:teuthology.orchestra.run.smithi072.stdout: osd deep scrub update digest min age = 30 2023-10-14T10:35:11.295 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-14T10:35:11.295 INFO:teuthology.orchestra.run.smithi072.stdout: osd map max advance = 10 2023-10-14T10:35:11.295 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-14T10:35:11.295 INFO:teuthology.orchestra.run.smithi072.stdout: journal zero on create = true 2023-10-14T10:35:11.296 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-14T10:35:11.296 INFO:teuthology.orchestra.run.smithi072.stdout: filestore ondisk finisher threads = 3 2023-10-14T10:35:11.296 INFO:teuthology.orchestra.run.smithi072.stdout: filestore apply finisher threads = 3 2023-10-14T10:35:11.296 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-14T10:35:11.296 INFO:teuthology.orchestra.run.smithi072.stdout: bdev debug aio = true 2023-10-14T10:35:11.297 INFO:teuthology.orchestra.run.smithi072.stdout: osd debug misdirected ops = true 2023-10-14T10:35:11.297 INFO:teuthology.orchestra.run.smithi072.stdout: bdev async discard = True 2023-10-14T10:35:11.297 INFO:teuthology.orchestra.run.smithi072.stdout: bdev enable discard = True 2023-10-14T10:35:11.297 INFO:teuthology.orchestra.run.smithi072.stdout: bluestore allocator = bitmap 2023-10-14T10:35:11.297 INFO:teuthology.orchestra.run.smithi072.stdout: bluestore block size = 96636764160 2023-10-14T10:35:11.297 INFO:teuthology.orchestra.run.smithi072.stdout: bluestore fsck on mount = True 2023-10-14T10:35:11.298 INFO:teuthology.orchestra.run.smithi072.stdout: debug bluefs = 1/20 2023-10-14T10:35:11.298 INFO:teuthology.orchestra.run.smithi072.stdout: debug bluestore = 1/20 2023-10-14T10:35:11.298 INFO:teuthology.orchestra.run.smithi072.stdout: debug ms = 1 2023-10-14T10:35:11.298 INFO:teuthology.orchestra.run.smithi072.stdout: debug osd = 20 2023-10-14T10:35:11.298 INFO:teuthology.orchestra.run.smithi072.stdout: debug rocksdb = 4/10 2023-10-14T10:35:11.299 INFO:teuthology.orchestra.run.smithi072.stdout: mon osd backfillfull_ratio = 0.85 2023-10-14T10:35:11.299 INFO:teuthology.orchestra.run.smithi072.stdout: mon osd full ratio = 0.9 2023-10-14T10:35:11.299 INFO:teuthology.orchestra.run.smithi072.stdout: mon osd nearfull ratio = 0.8 2023-10-14T10:35:11.299 INFO:teuthology.orchestra.run.smithi072.stdout: osd failsafe full ratio = 0.95 2023-10-14T10:35:11.299 INFO:teuthology.orchestra.run.smithi072.stdout: osd objectstore = bluestore 2023-10-14T10:35:11.300 INFO:teuthology.orchestra.run.smithi072.stdout: osd op complaint time = 180 2023-10-14T10:35:11.300 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-14T10:35:11.300 INFO:teuthology.orchestra.run.smithi072.stdout:[mgr] 2023-10-14T10:35:11.300 INFO:teuthology.orchestra.run.smithi072.stdout: debug ms = 1 2023-10-14T10:35:11.300 INFO:teuthology.orchestra.run.smithi072.stdout: debug mgr = 20 2023-10-14T10:35:11.301 INFO:teuthology.orchestra.run.smithi072.stdout: debug mon = 20 2023-10-14T10:35:11.301 INFO:teuthology.orchestra.run.smithi072.stdout: debug auth = 20 2023-10-14T10:35:11.301 INFO:teuthology.orchestra.run.smithi072.stdout: mon reweight min pgs per osd = 4 2023-10-14T10:35:11.301 INFO:teuthology.orchestra.run.smithi072.stdout: mon reweight min bytes per osd = 10 2023-10-14T10:35:11.301 INFO:teuthology.orchestra.run.smithi072.stdout: mgr/telemetry/nag = false 2023-10-14T10:35:11.301 INFO:teuthology.orchestra.run.smithi072.stdout: debug client = 20 2023-10-14T10:35:11.302 INFO:teuthology.orchestra.run.smithi072.stdout: debug finisher = 20 2023-10-14T10:35:11.302 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-14T10:35:11.302 INFO:teuthology.orchestra.run.smithi072.stdout:[mon] 2023-10-14T10:35:11.302 INFO:teuthology.orchestra.run.smithi072.stdout: debug ms = 1 2023-10-14T10:35:11.302 INFO:teuthology.orchestra.run.smithi072.stdout: debug mon = 20 2023-10-14T10:35:11.303 INFO:teuthology.orchestra.run.smithi072.stdout: debug paxos = 20 2023-10-14T10:35:11.303 INFO:teuthology.orchestra.run.smithi072.stdout: debug auth = 20 2023-10-14T10:35:11.303 INFO:teuthology.orchestra.run.smithi072.stdout: mon data avail warn = 5 2023-10-14T10:35:11.303 INFO:teuthology.orchestra.run.smithi072.stdout: mon mgr mkfs grace = 240 2023-10-14T10:35:11.303 INFO:teuthology.orchestra.run.smithi072.stdout: mon reweight min pgs per osd = 4 2023-10-14T10:35:11.304 INFO:teuthology.orchestra.run.smithi072.stdout: mon osd reporter subtree level = osd 2023-10-14T10:35:11.304 INFO:teuthology.orchestra.run.smithi072.stdout: mon osd prime pg temp = true 2023-10-14T10:35:11.304 INFO:teuthology.orchestra.run.smithi072.stdout: mon reweight min bytes per osd = 10 2023-10-14T10:35:11.304 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-14T10:35:11.304 INFO:teuthology.orchestra.run.smithi072.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-14T10:35:11.304 INFO:teuthology.orchestra.run.smithi072.stdout: auth mon ticket ttl = 660 # 11m 2023-10-14T10:35:11.305 INFO:teuthology.orchestra.run.smithi072.stdout: auth service ticket ttl = 240 # 4m 2023-10-14T10:35:11.305 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-14T10:35:11.305 INFO:teuthology.orchestra.run.smithi072.stdout: # don't complain about insecure global_id in the test suite 2023-10-14T10:35:11.305 INFO:teuthology.orchestra.run.smithi072.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-14T10:35:11.305 INFO:teuthology.orchestra.run.smithi072.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-14T10:35:11.306 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-14T10:35:11.306 INFO:teuthology.orchestra.run.smithi072.stdout: # 1m isn't quite enough 2023-10-14T10:35:11.306 INFO:teuthology.orchestra.run.smithi072.stdout: mon_down_mkfs_grace = 2m 2023-10-14T10:35:11.306 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-14T10:35:11.306 INFO:teuthology.orchestra.run.smithi072.stdout: mon_warn_on_filestore_osds = false 2023-10-14T10:35:11.307 INFO:teuthology.orchestra.run.smithi072.stdout: mon op complaint time = 120 2023-10-14T10:35:11.307 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-14T10:35:11.307 INFO:teuthology.orchestra.run.smithi072.stdout:[client] 2023-10-14T10:35:11.307 INFO:teuthology.orchestra.run.smithi072.stdout: rgw cache enabled = true 2023-10-14T10:35:11.307 INFO:teuthology.orchestra.run.smithi072.stdout: rgw enable ops log = true 2023-10-14T10:35:11.308 INFO:teuthology.orchestra.run.smithi072.stdout: rgw enable usage log = true 2023-10-14T10:35:11.308 INFO:teuthology.orchestra.run.smithi072.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-14T10:35:11.308 INFO:teuthology.orchestra.run.smithi072.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-14T10:35:11.308 INFO:teuthology.orchestra.run.smithi072.stdout: client mount timeout = 600 2023-10-14T10:35:11.308 INFO:teuthology.orchestra.run.smithi072.stdout: debug client = 20 2023-10-14T10:35:11.308 INFO:teuthology.orchestra.run.smithi072.stdout: debug ms = 1 2023-10-14T10:35:11.309 INFO:teuthology.orchestra.run.smithi072.stdout: pid file = "" 2023-10-14T10:35:11.309 INFO:teuthology.orchestra.run.smithi072.stdout: rados mon op timeout = 900 2023-10-14T10:35:11.309 INFO:teuthology.orchestra.run.smithi072.stdout: rados osd op timeout = 900 2023-10-14T10:35:11.309 INFO:teuthology.orchestra.run.smithi072.stdout:[mon.a] 2023-10-14T10:35:11.309 INFO:teuthology.orchestra.run.smithi072.stdout:[mon.b] 2023-10-14T10:35:11.310 INFO:teuthology.orchestra.run.smithi072.stdout:[mon.c] 2023-10-14T10:35:11.310 INFO:teuthology.orchestra.run.smithi072.stdout:[mds.a] 2023-10-14T10:35:11.310 INFO:teuthology.orchestra.run.smithi072.stdout:[mds.c] 2023-10-14T10:35:11.310 INFO:teuthology.orchestra.run.smithi072.stdout:[mds.b] 2023-10-14T10:35:11.310 INFO:teuthology.orchestra.run.smithi072.stdout:[mds.d] 2023-10-14T10:35:11.311 INFO:teuthology.orchestra.run.smithi072.stdout:[mds] 2023-10-14T10:35:11.311 INFO:teuthology.orchestra.run.smithi072.stdout: debug mds = 20 2023-10-14T10:35:11.311 INFO:teuthology.orchestra.run.smithi072.stdout: debug mds balancer = 20 2023-10-14T10:35:11.311 INFO:teuthology.orchestra.run.smithi072.stdout: debug ms = 1 2023-10-14T10:35:11.311 INFO:teuthology.orchestra.run.smithi072.stdout: mds debug frag = True 2023-10-14T10:35:11.311 INFO:teuthology.orchestra.run.smithi072.stdout: mds debug scatterstat = True 2023-10-14T10:35:11.312 INFO:teuthology.orchestra.run.smithi072.stdout: mds op complaint time = 180 2023-10-14T10:35:11.312 INFO:teuthology.orchestra.run.smithi072.stdout: mds verify scatter = True 2023-10-14T10:35:11.312 INFO:teuthology.orchestra.run.smithi072.stdout: osd op complaint time = 180 2023-10-14T10:35:11.312 INFO:teuthology.orchestra.run.smithi072.stdout: rados mon op timeout = 900 2023-10-14T10:35:11.312 INFO:teuthology.orchestra.run.smithi072.stdout: rados osd op timeout = 900 2023-10-14T10:35:11.313 INFO:tasks.ceph:Creating admin key on mon.a... 2023-10-14T10:35:11.313 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-10-14T10:35:11.381 INFO:tasks.ceph:Copying monmap to all nodes... 2023-10-14T10:35:11.381 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2023-10-14T10:35:11.382 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-10-14T10:35:11.399 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2023-10-14T10:35:11.400 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-10-14T10:35:11.456 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi067.front.sepia.ceph.com 2023-10-14T10:35:11.456 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2023-10-14T10:35:11.457 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-14T10:35:11.457 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-14T10:35:11.533 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2023-10-14T10:35:11.534 DEBUG:teuthology.orchestra.run.smithi067:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-14T10:35:11.591 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi072.front.sepia.ceph.com 2023-10-14T10:35:11.592 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-10-14T10:35:11.592 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-14T10:35:11.592 DEBUG:teuthology.orchestra.run.smithi072:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-14T10:35:11.628 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-10-14T10:35:11.629 DEBUG:teuthology.orchestra.run.smithi072:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-14T10:35:11.685 INFO:tasks.ceph:Setting up mon nodes... 2023-10-14T10:35:11.686 INFO:tasks.ceph:Setting up mgr nodes... 2023-10-14T10:35:11.686 DEBUG:teuthology.orchestra.run.smithi067:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-10-14T10:35:11.741 INFO:teuthology.orchestra.run.smithi067.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-10-14T10:35:11.744 DEBUG:teuthology.orchestra.run.smithi072:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-10-14T10:35:11.833 INFO:teuthology.orchestra.run.smithi072.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-10-14T10:35:11.836 INFO:tasks.ceph:Setting up mds nodes... 2023-10-14T10:35:11.836 DEBUG:teuthology.orchestra.run.smithi067:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-10-14T10:35:11.893 INFO:teuthology.orchestra.run.smithi067.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-10-14T10:35:11.895 DEBUG:teuthology.orchestra.run.smithi067:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-10-14T10:35:11.922 DEBUG:teuthology.orchestra.run.smithi067:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-10-14T10:35:12.017 INFO:teuthology.orchestra.run.smithi067.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-10-14T10:35:12.020 DEBUG:teuthology.orchestra.run.smithi067:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-10-14T10:35:12.047 DEBUG:teuthology.orchestra.run.smithi072:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-10-14T10:35:12.101 INFO:teuthology.orchestra.run.smithi072.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-10-14T10:35:12.104 DEBUG:teuthology.orchestra.run.smithi072:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-10-14T10:35:12.130 DEBUG:teuthology.orchestra.run.smithi072:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-10-14T10:35:12.224 INFO:teuthology.orchestra.run.smithi072.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-10-14T10:35:12.226 DEBUG:teuthology.orchestra.run.smithi072:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-10-14T10:35:12.252 INFO:tasks.ceph_client:Setting up client nodes... 2023-10-14T10:35:12.253 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-10-14T10:35:12.298 INFO:teuthology.orchestra.run.smithi067.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-10-14T10:35:12.312 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-10-14T10:35:12.355 INFO:teuthology.orchestra.run.smithi067.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-10-14T10:35:12.368 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-10-14T10:35:12.412 INFO:teuthology.orchestra.run.smithi072.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-10-14T10:35:12.438 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-10-14T10:35:12.520 INFO:teuthology.orchestra.run.smithi072.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-10-14T10:35:12.533 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-10-14T10:35:12.533 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi067.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi072.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-10-14T10:35:12.534 DEBUG:teuthology.orchestra.run.smithi067:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-10-14T10:35:12.561 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-14T10:35:12.562 INFO:tasks.ceph:role: osd.0 2023-10-14T10:35:12.562 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi067.front.sepia.ceph.com 2023-10-14T10:35:12.563 DEBUG:teuthology.orchestra.run.smithi067:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-14T10:35:12.636 INFO:teuthology.orchestra.run.smithi067.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-14T10:35:12.636 INFO:teuthology.orchestra.run.smithi067.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-14T10:35:12.637 INFO:teuthology.orchestra.run.smithi067.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-14T10:35:12.637 INFO:teuthology.orchestra.run.smithi067.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-14T10:35:12.637 INFO:teuthology.orchestra.run.smithi067.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-14T10:35:12.637 INFO:teuthology.orchestra.run.smithi067.stdout: = sunit=0 swidth=0 blks 2023-10-14T10:35:12.637 INFO:teuthology.orchestra.run.smithi067.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-14T10:35:12.638 INFO:teuthology.orchestra.run.smithi067.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-14T10:35:12.638 INFO:teuthology.orchestra.run.smithi067.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-14T10:35:12.638 INFO:teuthology.orchestra.run.smithi067.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-14T10:35:13.067 INFO:teuthology.orchestra.run.smithi067.stdout:Discarding blocks...Done. 2023-10-14T10:35:13.068 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi067.front.sepia.ceph.com -o noatime 2023-10-14T10:35:13.069 DEBUG:teuthology.orchestra.run.smithi067:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-10-14T10:35:13.236 DEBUG:teuthology.orchestra.run.smithi067:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-10-14T10:35:13.312 DEBUG:teuthology.orchestra.run.smithi067:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-10-14T10:35:13.382 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-14T10:35:13.382 INFO:tasks.ceph:role: osd.1 2023-10-14T10:35:13.383 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi067.front.sepia.ceph.com 2023-10-14T10:35:13.383 DEBUG:teuthology.orchestra.run.smithi067:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-14T10:35:13.460 INFO:teuthology.orchestra.run.smithi067.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-14T10:35:13.460 INFO:teuthology.orchestra.run.smithi067.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-14T10:35:13.460 INFO:teuthology.orchestra.run.smithi067.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-14T10:35:13.461 INFO:teuthology.orchestra.run.smithi067.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-14T10:35:13.461 INFO:teuthology.orchestra.run.smithi067.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-14T10:35:13.461 INFO:teuthology.orchestra.run.smithi067.stdout: = sunit=0 swidth=0 blks 2023-10-14T10:35:13.461 INFO:teuthology.orchestra.run.smithi067.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-14T10:35:13.462 INFO:teuthology.orchestra.run.smithi067.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-14T10:35:13.462 INFO:teuthology.orchestra.run.smithi067.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-14T10:35:13.462 INFO:teuthology.orchestra.run.smithi067.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-14T10:35:13.861 INFO:teuthology.orchestra.run.smithi067.stdout:Discarding blocks...Done. 2023-10-14T10:35:13.864 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi067.front.sepia.ceph.com -o noatime 2023-10-14T10:35:13.864 DEBUG:teuthology.orchestra.run.smithi067:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-10-14T10:35:13.986 DEBUG:teuthology.orchestra.run.smithi067:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-10-14T10:35:14.018 DEBUG:teuthology.orchestra.run.smithi067:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-10-14T10:35:14.087 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-14T10:35:14.087 INFO:tasks.ceph:role: osd.2 2023-10-14T10:35:14.087 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi067.front.sepia.ceph.com 2023-10-14T10:35:14.088 DEBUG:teuthology.orchestra.run.smithi067:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-10-14T10:35:14.161 INFO:teuthology.orchestra.run.smithi067.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-10-14T10:35:14.162 INFO:teuthology.orchestra.run.smithi067.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-14T10:35:14.162 INFO:teuthology.orchestra.run.smithi067.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-14T10:35:14.162 INFO:teuthology.orchestra.run.smithi067.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-14T10:35:14.162 INFO:teuthology.orchestra.run.smithi067.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-14T10:35:14.163 INFO:teuthology.orchestra.run.smithi067.stdout: = sunit=0 swidth=0 blks 2023-10-14T10:35:14.163 INFO:teuthology.orchestra.run.smithi067.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-14T10:35:14.163 INFO:teuthology.orchestra.run.smithi067.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-14T10:35:14.163 INFO:teuthology.orchestra.run.smithi067.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-14T10:35:14.163 INFO:teuthology.orchestra.run.smithi067.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-14T10:35:14.612 INFO:teuthology.orchestra.run.smithi067.stdout:Discarding blocks...Done. 2023-10-14T10:35:14.614 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi067.front.sepia.ceph.com -o noatime 2023-10-14T10:35:14.615 DEBUG:teuthology.orchestra.run.smithi067:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-10-14T10:35:14.763 DEBUG:teuthology.orchestra.run.smithi067:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-10-14T10:35:14.798 DEBUG:teuthology.orchestra.run.smithi067:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-10-14T10:35:14.870 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-14T10:35:14.870 INFO:tasks.ceph:role: osd.3 2023-10-14T10:35:14.871 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi067.front.sepia.ceph.com 2023-10-14T10:35:14.871 DEBUG:teuthology.orchestra.run.smithi067:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-10-14T10:35:14.951 INFO:teuthology.orchestra.run.smithi067.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-10-14T10:35:14.952 INFO:teuthology.orchestra.run.smithi067.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-14T10:35:14.952 INFO:teuthology.orchestra.run.smithi067.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-14T10:35:14.952 INFO:teuthology.orchestra.run.smithi067.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-14T10:35:14.952 INFO:teuthology.orchestra.run.smithi067.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-14T10:35:14.953 INFO:teuthology.orchestra.run.smithi067.stdout: = sunit=0 swidth=0 blks 2023-10-14T10:35:14.953 INFO:teuthology.orchestra.run.smithi067.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-14T10:35:14.953 INFO:teuthology.orchestra.run.smithi067.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-14T10:35:14.953 INFO:teuthology.orchestra.run.smithi067.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-14T10:35:14.954 INFO:teuthology.orchestra.run.smithi067.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-14T10:35:15.404 INFO:teuthology.orchestra.run.smithi067.stdout:Discarding blocks...Done. 2023-10-14T10:35:15.407 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi067.front.sepia.ceph.com -o noatime 2023-10-14T10:35:15.407 DEBUG:teuthology.orchestra.run.smithi067:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-10-14T10:35:15.598 DEBUG:teuthology.orchestra.run.smithi067:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-10-14T10:35:15.635 DEBUG:teuthology.orchestra.run.smithi067:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-14T10:35:15.746 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:15.744+0000 7f7ac3bf1640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-10-14T10:35:15.747 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:15.744+0000 7f7ac3bf1640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-10-14T10:35:15.747 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:15.744+0000 7f7ac3bf1640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-14T10:35:15.747 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:15.744+0000 7f7ac3bf1640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-14T10:35:15.747 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:15.744+0000 7f7ac3bf1640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-14T10:35:15.748 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:15.745+0000 7f7ac3bf1640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-10-14T10:35:18.616 DEBUG:teuthology.orchestra.run.smithi067:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-10-14T10:35:18.648 DEBUG:teuthology.orchestra.run.smithi067:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-14T10:35:18.742 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:18.740+0000 7fedb16be640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-10-14T10:35:18.743 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:18.740+0000 7fedb16be640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-10-14T10:35:18.743 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:18.740+0000 7fedb16be640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-14T10:35:18.743 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:18.740+0000 7fedb16be640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-14T10:35:18.743 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:18.740+0000 7fedb16be640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-14T10:35:18.743 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:18.741+0000 7fedb16be640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-10-14T10:35:21.613 DEBUG:teuthology.orchestra.run.smithi067:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-10-14T10:35:21.646 DEBUG:teuthology.orchestra.run.smithi067:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-14T10:35:21.740 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:21.738+0000 7f872cf40640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-10-14T10:35:21.741 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:21.738+0000 7f872cf40640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-10-14T10:35:21.741 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:21.738+0000 7f872cf40640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-14T10:35:21.741 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:21.738+0000 7f872cf40640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-14T10:35:21.741 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:21.738+0000 7f872cf40640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-14T10:35:21.742 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:21.739+0000 7f872cf40640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-10-14T10:35:24.603 DEBUG:teuthology.orchestra.run.smithi067:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-10-14T10:35:24.636 DEBUG:teuthology.orchestra.run.smithi067:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-14T10:35:24.730 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:24.728+0000 7f4a675d2640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-10-14T10:35:24.730 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:24.728+0000 7f4a675d2640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-10-14T10:35:24.730 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:24.729+0000 7f4a675d2640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-14T10:35:24.731 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:24.729+0000 7f4a675d2640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-14T10:35:24.731 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:24.729+0000 7f4a675d2640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-14T10:35:24.731 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:24.729+0000 7f4a675d2640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-10-14T10:35:27.598 DEBUG:teuthology.orchestra.run.smithi067:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-10-14T10:35:27.631 DEBUG:teuthology.orchestra.run.smithi072:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-10-14T10:35:27.666 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-14T10:35:27.667 INFO:tasks.ceph:role: osd.4 2023-10-14T10:35:27.667 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi072.front.sepia.ceph.com 2023-10-14T10:35:27.668 DEBUG:teuthology.orchestra.run.smithi072:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-14T10:35:27.745 INFO:teuthology.orchestra.run.smithi072.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-14T10:35:27.746 INFO:teuthology.orchestra.run.smithi072.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-14T10:35:27.746 INFO:teuthology.orchestra.run.smithi072.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-14T10:35:27.746 INFO:teuthology.orchestra.run.smithi072.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-14T10:35:27.746 INFO:teuthology.orchestra.run.smithi072.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-14T10:35:27.746 INFO:teuthology.orchestra.run.smithi072.stdout: = sunit=0 swidth=0 blks 2023-10-14T10:35:27.747 INFO:teuthology.orchestra.run.smithi072.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-14T10:35:27.747 INFO:teuthology.orchestra.run.smithi072.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-14T10:35:27.747 INFO:teuthology.orchestra.run.smithi072.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-14T10:35:27.747 INFO:teuthology.orchestra.run.smithi072.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-14T10:35:28.167 INFO:teuthology.orchestra.run.smithi072.stdout:Discarding blocks...Done. 2023-10-14T10:35:28.169 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi072.front.sepia.ceph.com -o noatime 2023-10-14T10:35:28.169 DEBUG:teuthology.orchestra.run.smithi072:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-10-14T10:35:28.355 DEBUG:teuthology.orchestra.run.smithi072:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-10-14T10:35:28.395 DEBUG:teuthology.orchestra.run.smithi072:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-10-14T10:35:28.464 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-14T10:35:28.465 INFO:tasks.ceph:role: osd.5 2023-10-14T10:35:28.465 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi072.front.sepia.ceph.com 2023-10-14T10:35:28.465 DEBUG:teuthology.orchestra.run.smithi072:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-14T10:35:28.540 INFO:teuthology.orchestra.run.smithi072.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-14T10:35:28.541 INFO:teuthology.orchestra.run.smithi072.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-14T10:35:28.541 INFO:teuthology.orchestra.run.smithi072.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-14T10:35:28.541 INFO:teuthology.orchestra.run.smithi072.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-14T10:35:28.543 INFO:teuthology.orchestra.run.smithi072.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-14T10:35:28.543 INFO:teuthology.orchestra.run.smithi072.stdout: = sunit=0 swidth=0 blks 2023-10-14T10:35:28.544 INFO:teuthology.orchestra.run.smithi072.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-14T10:35:28.544 INFO:teuthology.orchestra.run.smithi072.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-14T10:35:28.544 INFO:teuthology.orchestra.run.smithi072.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-14T10:35:28.544 INFO:teuthology.orchestra.run.smithi072.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-14T10:35:28.967 INFO:teuthology.orchestra.run.smithi072.stdout:Discarding blocks...Done. 2023-10-14T10:35:28.969 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi072.front.sepia.ceph.com -o noatime 2023-10-14T10:35:28.970 DEBUG:teuthology.orchestra.run.smithi072:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-10-14T10:35:29.079 DEBUG:teuthology.orchestra.run.smithi072:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-10-14T10:35:29.115 DEBUG:teuthology.orchestra.run.smithi072:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-10-14T10:35:29.186 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-14T10:35:29.186 INFO:tasks.ceph:role: osd.6 2023-10-14T10:35:29.186 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi072.front.sepia.ceph.com 2023-10-14T10:35:29.187 DEBUG:teuthology.orchestra.run.smithi072:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-10-14T10:35:29.262 INFO:teuthology.orchestra.run.smithi072.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-10-14T10:35:29.262 INFO:teuthology.orchestra.run.smithi072.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-14T10:35:29.262 INFO:teuthology.orchestra.run.smithi072.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-14T10:35:29.263 INFO:teuthology.orchestra.run.smithi072.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-14T10:35:29.263 INFO:teuthology.orchestra.run.smithi072.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-14T10:35:29.263 INFO:teuthology.orchestra.run.smithi072.stdout: = sunit=0 swidth=0 blks 2023-10-14T10:35:29.263 INFO:teuthology.orchestra.run.smithi072.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-14T10:35:29.263 INFO:teuthology.orchestra.run.smithi072.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-14T10:35:29.264 INFO:teuthology.orchestra.run.smithi072.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-14T10:35:29.264 INFO:teuthology.orchestra.run.smithi072.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-14T10:35:29.711 INFO:teuthology.orchestra.run.smithi072.stdout:Discarding blocks...Done. 2023-10-14T10:35:29.713 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi072.front.sepia.ceph.com -o noatime 2023-10-14T10:35:29.713 DEBUG:teuthology.orchestra.run.smithi072:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-10-14T10:35:29.889 DEBUG:teuthology.orchestra.run.smithi072:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-10-14T10:35:29.923 DEBUG:teuthology.orchestra.run.smithi072:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-10-14T10:35:29.993 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-14T10:35:29.993 INFO:tasks.ceph:role: osd.7 2023-10-14T10:35:29.993 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi072.front.sepia.ceph.com 2023-10-14T10:35:29.994 DEBUG:teuthology.orchestra.run.smithi072:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-10-14T10:35:30.068 INFO:teuthology.orchestra.run.smithi072.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-10-14T10:35:30.068 INFO:teuthology.orchestra.run.smithi072.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-14T10:35:30.068 INFO:teuthology.orchestra.run.smithi072.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-14T10:35:30.068 INFO:teuthology.orchestra.run.smithi072.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-14T10:35:30.069 INFO:teuthology.orchestra.run.smithi072.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-14T10:35:30.069 INFO:teuthology.orchestra.run.smithi072.stdout: = sunit=0 swidth=0 blks 2023-10-14T10:35:30.069 INFO:teuthology.orchestra.run.smithi072.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-14T10:35:30.069 INFO:teuthology.orchestra.run.smithi072.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-14T10:35:30.069 INFO:teuthology.orchestra.run.smithi072.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-14T10:35:30.070 INFO:teuthology.orchestra.run.smithi072.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-14T10:35:30.503 INFO:teuthology.orchestra.run.smithi072.stdout:Discarding blocks...Done. 2023-10-14T10:35:30.506 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi072.front.sepia.ceph.com -o noatime 2023-10-14T10:35:30.506 DEBUG:teuthology.orchestra.run.smithi072:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-10-14T10:35:30.684 DEBUG:teuthology.orchestra.run.smithi072:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-10-14T10:35:30.722 DEBUG:teuthology.orchestra.run.smithi072:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-14T10:35:31.065 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:31.062+0000 7f9da4bb9640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-10-14T10:35:31.065 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:31.062+0000 7f9da4bb9640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-10-14T10:35:31.065 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:31.062+0000 7f9da4bb9640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-14T10:35:31.065 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:31.063+0000 7f9da4bb9640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-14T10:35:31.066 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:31.063+0000 7f9da4bb9640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-14T10:35:31.066 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:31.063+0000 7f9da4bb9640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-10-14T10:35:33.936 DEBUG:teuthology.orchestra.run.smithi072:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-10-14T10:35:33.967 DEBUG:teuthology.orchestra.run.smithi072:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-14T10:35:34.059 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:34.056+0000 7fae1f0e3640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-10-14T10:35:34.059 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:34.057+0000 7fae1f0e3640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-10-14T10:35:34.059 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:34.057+0000 7fae1f0e3640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-14T10:35:34.059 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:34.057+0000 7fae1f0e3640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-14T10:35:34.060 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:34.057+0000 7fae1f0e3640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-14T10:35:34.060 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:34.057+0000 7fae1f0e3640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-10-14T10:35:36.929 DEBUG:teuthology.orchestra.run.smithi072:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-10-14T10:35:36.961 DEBUG:teuthology.orchestra.run.smithi072:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-14T10:35:37.054 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:37.052+0000 7f53e8031640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-10-14T10:35:37.054 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:37.052+0000 7f53e8031640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-10-14T10:35:37.055 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:37.052+0000 7f53e8031640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-14T10:35:37.055 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:37.052+0000 7f53e8031640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-14T10:35:37.055 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:37.052+0000 7f53e8031640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-14T10:35:37.055 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:37.052+0000 7f53e8031640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-10-14T10:35:39.204 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-14T10:35:39.206 DEBUG:teuthology.orchestra.run.smithi072:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-14T10:35:39.924 DEBUG:teuthology.orchestra.run.smithi072:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-10-14T10:35:39.955 DEBUG:teuthology.orchestra.run.smithi072:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-14T10:35:40.046 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:40.043+0000 7f75fb5db640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-10-14T10:35:40.047 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:40.044+0000 7f75fb5db640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-10-14T10:35:40.047 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:40.044+0000 7f75fb5db640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-14T10:35:40.047 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:40.044+0000 7f75fb5db640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-14T10:35:40.047 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:40.044+0000 7f75fb5db640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-14T10:35:40.047 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:40.044+0000 7f75fb5db640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-10-14T10:35:42.913 DEBUG:teuthology.orchestra.run.smithi072:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-10-14T10:35:42.944 INFO:tasks.ceph:Reading keys from all nodes... 2023-10-14T10:35:42.945 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2023-10-14T10:35:42.945 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-10-14T10:35:42.978 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2023-10-14T10:35:42.978 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-10-14T10:35:43.050 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2023-10-14T10:35:43.050 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-10-14T10:35:43.120 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2023-10-14T10:35:43.120 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-10-14T10:35:43.189 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2023-10-14T10:35:43.190 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-10-14T10:35:43.259 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2023-10-14T10:35:43.259 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-10-14T10:35:43.329 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2023-10-14T10:35:43.330 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-10-14T10:35:43.399 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-10-14T10:35:43.399 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-10-14T10:35:43.431 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-10-14T10:35:43.431 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-10-14T10:35:43.501 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-10-14T10:35:43.501 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-10-14T10:35:43.570 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-10-14T10:35:43.570 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-10-14T10:35:43.638 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-10-14T10:35:43.639 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-10-14T10:35:43.705 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-10-14T10:35:43.706 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-10-14T10:35:43.772 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-10-14T10:35:43.772 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-10-14T10:35:43.839 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2023-10-14T10:35:43.839 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-10-14T10:35:43.860 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2023-10-14T10:35:43.860 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-10-14T10:35:43.920 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-10-14T10:35:43.921 DEBUG:teuthology.orchestra.run.smithi072:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-10-14T10:35:43.939 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-10-14T10:35:43.939 DEBUG:teuthology.orchestra.run.smithi072:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-10-14T10:35:43.996 INFO:tasks.ceph:Adding keys to all mons... 2023-10-14T10:35:43.996 DEBUG:teuthology.orchestra.run.smithi067:> sudo tee -a /etc/ceph/ceph.keyring 2023-10-14T10:35:43.999 DEBUG:teuthology.orchestra.run.smithi072:> sudo tee -a /etc/ceph/ceph.keyring 2023-10-14T10:35:44.039 INFO:teuthology.orchestra.run.smithi067.stdout:[mgr.y] 2023-10-14T10:35:44.039 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQDfbiplkwIpLBAAG1UzPJBzmMBeRev2BTrBEA== 2023-10-14T10:35:44.039 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.a] 2023-10-14T10:35:44.040 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQDfbiplhHkzNRAAu6V9TUs+PZgWJTL1PslPTQ== 2023-10-14T10:35:44.040 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.c] 2023-10-14T10:35:44.040 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQDgbiplLMACARAAtLK9Ksyh+PWJMflIG7N0gg== 2023-10-14T10:35:44.040 INFO:teuthology.orchestra.run.smithi067.stdout:[osd.0] 2023-10-14T10:35:44.040 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQDjbiplJJhtLBAA68bGP9xw1933h7ZwGRvrTQ== 2023-10-14T10:35:44.041 INFO:teuthology.orchestra.run.smithi067.stdout:[osd.1] 2023-10-14T10:35:44.041 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQDmbiplbkEwLBAASrn+o/58EZvt8jxm/x9Ing== 2023-10-14T10:35:44.041 INFO:teuthology.orchestra.run.smithi067.stdout:[osd.2] 2023-10-14T10:35:44.041 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQDpbipl+X0ULBAAjZfhTIYrX7SGQcl+21d04Q== 2023-10-14T10:35:44.042 INFO:teuthology.orchestra.run.smithi067.stdout:[osd.3] 2023-10-14T10:35:44.042 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQDsbipldEFxKxAAbNwsAPRkpQOAYiQjBsTTDw== 2023-10-14T10:35:44.042 INFO:teuthology.orchestra.run.smithi067.stdout:[mgr.x] 2023-10-14T10:35:44.042 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQDfbipl5Z+hMRAA5qRfiQzDEjpQ93opgGqVOQ== 2023-10-14T10:35:44.042 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.b] 2023-10-14T10:35:44.043 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQDgbiplYRkFBhAALpCIvRKJUBBNWk7ulMhsHg== 2023-10-14T10:35:44.043 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.d] 2023-10-14T10:35:44.043 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQDgbipldutQDRAAptZ4A6PV8ngnjdyDLP01eg== 2023-10-14T10:35:44.043 INFO:teuthology.orchestra.run.smithi067.stdout:[osd.4] 2023-10-14T10:35:44.043 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQDzbiplkgfNAxAAwtb4YKBFvu0SPQ81F9KphQ== 2023-10-14T10:35:44.044 INFO:teuthology.orchestra.run.smithi067.stdout:[osd.5] 2023-10-14T10:35:44.044 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQD2bipltwp0AxAAiduwcaM3rVGQa05otP99yA== 2023-10-14T10:35:44.044 INFO:teuthology.orchestra.run.smithi067.stdout:[osd.6] 2023-10-14T10:35:44.044 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQD5biplD9QqAxAA4jezF4GFarOijLCF5XTXMQ== 2023-10-14T10:35:44.045 INFO:teuthology.orchestra.run.smithi067.stdout:[osd.7] 2023-10-14T10:35:44.045 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQD8biplubOvAhAA28OZRR1FtTk0x3sIFk9K7w== 2023-10-14T10:35:44.045 INFO:teuthology.orchestra.run.smithi067.stdout:[client.0] 2023-10-14T10:35:44.045 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQDgbiplgPnBERAAH7t74KYvneMtdrMTWfuThw== 2023-10-14T10:35:44.045 INFO:teuthology.orchestra.run.smithi067.stdout:[client.2] 2023-10-14T10:35:44.046 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQDgbiplHnsmFRAA8jpTGMcY/uLMdRdS9/+QbQ== 2023-10-14T10:35:44.046 INFO:teuthology.orchestra.run.smithi067.stdout:[client.1] 2023-10-14T10:35:44.046 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQDgbiplo1CCGBAApLIw2B2Gg0oOfJmwaWeSsA== 2023-10-14T10:35:44.046 INFO:teuthology.orchestra.run.smithi067.stdout:[client.3] 2023-10-14T10:35:44.046 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQDgbiplFIX8HhAAhID21aVUrm/0+OvEr+LSEw== 2023-10-14T10:35:44.061 INFO:teuthology.orchestra.run.smithi072.stdout:[mgr.y] 2023-10-14T10:35:44.061 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQDfbiplkwIpLBAAG1UzPJBzmMBeRev2BTrBEA== 2023-10-14T10:35:44.061 INFO:teuthology.orchestra.run.smithi072.stdout:[mds.a] 2023-10-14T10:35:44.062 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQDfbiplhHkzNRAAu6V9TUs+PZgWJTL1PslPTQ== 2023-10-14T10:35:44.062 INFO:teuthology.orchestra.run.smithi072.stdout:[mds.c] 2023-10-14T10:35:44.062 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQDgbiplLMACARAAtLK9Ksyh+PWJMflIG7N0gg== 2023-10-14T10:35:44.062 INFO:teuthology.orchestra.run.smithi072.stdout:[osd.0] 2023-10-14T10:35:44.062 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQDjbiplJJhtLBAA68bGP9xw1933h7ZwGRvrTQ== 2023-10-14T10:35:44.063 INFO:teuthology.orchestra.run.smithi072.stdout:[osd.1] 2023-10-14T10:35:44.063 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQDmbiplbkEwLBAASrn+o/58EZvt8jxm/x9Ing== 2023-10-14T10:35:44.063 INFO:teuthology.orchestra.run.smithi072.stdout:[osd.2] 2023-10-14T10:35:44.063 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQDpbipl+X0ULBAAjZfhTIYrX7SGQcl+21d04Q== 2023-10-14T10:35:44.063 INFO:teuthology.orchestra.run.smithi072.stdout:[osd.3] 2023-10-14T10:35:44.064 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQDsbipldEFxKxAAbNwsAPRkpQOAYiQjBsTTDw== 2023-10-14T10:35:44.064 INFO:teuthology.orchestra.run.smithi072.stdout:[mgr.x] 2023-10-14T10:35:44.064 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQDfbipl5Z+hMRAA5qRfiQzDEjpQ93opgGqVOQ== 2023-10-14T10:35:44.064 INFO:teuthology.orchestra.run.smithi072.stdout:[mds.b] 2023-10-14T10:35:44.064 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQDgbiplYRkFBhAALpCIvRKJUBBNWk7ulMhsHg== 2023-10-14T10:35:44.065 INFO:teuthology.orchestra.run.smithi072.stdout:[mds.d] 2023-10-14T10:35:44.065 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQDgbipldutQDRAAptZ4A6PV8ngnjdyDLP01eg== 2023-10-14T10:35:44.065 INFO:teuthology.orchestra.run.smithi072.stdout:[osd.4] 2023-10-14T10:35:44.065 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQDzbiplkgfNAxAAwtb4YKBFvu0SPQ81F9KphQ== 2023-10-14T10:35:44.065 INFO:teuthology.orchestra.run.smithi072.stdout:[osd.5] 2023-10-14T10:35:44.065 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQD2bipltwp0AxAAiduwcaM3rVGQa05otP99yA== 2023-10-14T10:35:44.066 INFO:teuthology.orchestra.run.smithi072.stdout:[osd.6] 2023-10-14T10:35:44.066 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQD5biplD9QqAxAA4jezF4GFarOijLCF5XTXMQ== 2023-10-14T10:35:44.066 INFO:teuthology.orchestra.run.smithi072.stdout:[osd.7] 2023-10-14T10:35:44.066 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQD8biplubOvAhAA28OZRR1FtTk0x3sIFk9K7w== 2023-10-14T10:35:44.066 INFO:teuthology.orchestra.run.smithi072.stdout:[client.0] 2023-10-14T10:35:44.067 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQDgbiplgPnBERAAH7t74KYvneMtdrMTWfuThw== 2023-10-14T10:35:44.067 INFO:teuthology.orchestra.run.smithi072.stdout:[client.2] 2023-10-14T10:35:44.067 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQDgbiplHnsmFRAA8jpTGMcY/uLMdRdS9/+QbQ== 2023-10-14T10:35:44.067 INFO:teuthology.orchestra.run.smithi072.stdout:[client.1] 2023-10-14T10:35:44.069 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQDgbiplo1CCGBAApLIw2B2Gg0oOfJmwaWeSsA== 2023-10-14T10:35:44.069 INFO:teuthology.orchestra.run.smithi072.stdout:[client.3] 2023-10-14T10:35:44.070 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQDgbiplFIX8HhAAhID21aVUrm/0+OvEr+LSEw== 2023-10-14T10:35:44.071 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-14T10:35:44.090 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-14T10:35:44.164 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-14T10:35:44.172 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-14T10:35:44.230 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-14T10:35:44.233 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-14T10:35:44.289 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-14T10:35:44.292 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-14T10:35:44.347 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-14T10:35:44.350 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-14T10:35:44.403 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-14T10:35:44.407 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-14T10:35:44.460 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-14T10:35:44.463 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-14T10:35:44.517 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-14T10:35:44.521 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-14T10:35:44.574 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-14T10:35:44.580 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-14T10:35:44.636 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-14T10:35:44.639 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-14T10:35:44.695 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-14T10:35:44.698 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-14T10:35:44.751 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-14T10:35:44.753 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-14T10:35:44.806 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-14T10:35:44.808 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-14T10:35:44.861 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-14T10:35:44.864 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-14T10:35:44.916 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-14T10:35:44.919 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-14T10:35:44.975 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-14T10:35:44.977 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-14T10:35:45.033 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-14T10:35:45.036 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-14T10:35:45.092 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-14T10:35:45.094 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-14T10:35:45.150 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-10-14T10:35:45.151 DEBUG:teuthology.orchestra.run.smithi067:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-10-14T10:35:45.178 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-14T10:35:45.277 DEBUG:teuthology.orchestra.run.smithi067:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-10-14T10:35:45.304 DEBUG:teuthology.orchestra.run.smithi072:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-10-14T10:35:45.331 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-14T10:35:45.430 DEBUG:teuthology.orchestra.run.smithi072:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-10-14T10:35:45.456 DEBUG:teuthology.orchestra.run.smithi072:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-10-14T10:35:45.523 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-14T10:35:45.620 DEBUG:teuthology.orchestra.run.smithi072:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-10-14T10:35:45.646 DEBUG:teuthology.orchestra.run.smithi067:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-10-14T10:35:45.649 DEBUG:teuthology.orchestra.run.smithi072:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-10-14T10:35:45.703 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-10-14T10:35:45.704 INFO:tasks.ceph.mon.a:Restarting daemon 2023-10-14T10:35:45.704 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-10-14T10:35:45.707 INFO:tasks.ceph.mon.a:Started 2023-10-14T10:35:45.707 INFO:tasks.ceph.mon.b:Restarting daemon 2023-10-14T10:35:45.707 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-10-14T10:35:45.746 INFO:tasks.ceph.mon.b:Started 2023-10-14T10:35:45.746 INFO:tasks.ceph.mon.c:Restarting daemon 2023-10-14T10:35:45.747 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-10-14T10:35:45.749 INFO:tasks.ceph.mon.c:Started 2023-10-14T10:35:45.749 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-10-14T10:35:45.749 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-10-14T10:35:45.750 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-10-14T10:35:45.752 INFO:tasks.ceph.mgr.y:Started 2023-10-14T10:35:45.753 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-10-14T10:35:45.753 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-10-14T10:35:45.755 INFO:tasks.ceph.mgr.x:Started 2023-10-14T10:35:45.756 INFO:tasks.ceph:Setting crush tunables to default 2023-10-14T10:35:45.756 DEBUG:teuthology.orchestra.run.smithi067:> sudo ceph --cluster ceph osd crush tunables default 2023-10-14T10:35:45.883 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:45.880+0000 7faefc0d7700 1 Processor -- start 2023-10-14T10:35:45.883 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:45.881+0000 7faefc0d7700 1 -- start start 2023-10-14T10:35:45.884 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:45.881+0000 7faefc0d7700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7faef415e2f0 0x7faef40cede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:35:45.885 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:45.881+0000 7faefc0d7700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faef40b2d60 con 0x7faef40cd950 2023-10-14T10:35:45.885 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:45.882+0000 7faefc0d7700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faef40cdd30 con 0x7faef406a840 2023-10-14T10:35:45.885 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:45.882+0000 7faefc0d7700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faef41619a0 con 0x7faef415e2f0 2023-10-14T10:35:45.885 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:45.882+0000 7faef9e73700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7faef40cd950 0x7faef415f570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:51062/0 (socket says 172.21.15.67:51062) 2023-10-14T10:35:45.886 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:45.882+0000 7faef9e73700 1 -- 172.21.15.67:0/2124305450 learned_addr learned my addr 172.21.15.67:0/2124305450 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:35:45.886 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:45.882+0000 7faef9672700 1 --2- 172.21.15.67:0/2124305450 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7faef415e2f0 0x7faef40cede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:35:45.886 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:45.882+0000 7faef9672700 1 -- 172.21.15.67:0/2124305450 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7faef415e2f0 msgr2=0x7faef40cede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-14T10:35:45.886 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:45.882+0000 7faef9672700 1 -- 172.21.15.67:0/2124305450 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7faef415e2f0 msgr2=0x7faef40cede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-14T10:35:45.886 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:45.882+0000 7faef9672700 1 --2- 172.21.15.67:0/2124305450 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7faef415e2f0 0x7faef40cede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-14T10:35:45.887 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:45.882+0000 7faef9672700 1 --2- 172.21.15.67:0/2124305450 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7faef415e2f0 0x7faef40cede0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-14T10:35:46.085 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:46.083+0000 7faef9672700 1 --2- 172.21.15.67:0/2124305450 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7faef415e2f0 0x7faef40cede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:35:46.086 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:46.083+0000 7faef9672700 1 -- 172.21.15.67:0/2124305450 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7faef415e2f0 msgr2=0x7faef40cede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-14T10:35:46.086 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:46.083+0000 7faef9672700 1 -- 172.21.15.67:0/2124305450 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7faef415e2f0 msgr2=0x7faef40cede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-14T10:35:46.086 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:46.083+0000 7faef9672700 1 --2- 172.21.15.67:0/2124305450 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7faef415e2f0 0x7faef40cede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-14T10:35:46.086 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:46.083+0000 7faef9672700 1 --2- 172.21.15.67:0/2124305450 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7faef415e2f0 0x7faef40cede0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-14T10:35:46.094 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:46.092+0000 7faef8e71700 1 -- 172.21.15.67:0/2124305450 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1243648181 0 0) 0x7faef40b2d60 con 0x7faef40cd950 2023-10-14T10:35:46.094 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:46.092+0000 7faef8e71700 1 -- 172.21.15.67:0/2124305450 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faee00035b0 con 0x7faef40cd950 2023-10-14T10:35:46.097 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:46.093+0000 7faef8e71700 1 -- 172.21.15.67:0/2124305450 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3151635854 0 0) 0x7faee00035b0 con 0x7faef40cd950 2023-10-14T10:35:46.097 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:46.093+0000 7faef8e71700 1 -- 172.21.15.67:0/2124305450 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7faef415e2f0 msgr2=0x7faef40cede0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-14T10:35:46.097 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:46.093+0000 7faef8e71700 1 --2- 172.21.15.67:0/2124305450 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7faef415e2f0 0x7faef40cede0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:46.098 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:46.093+0000 7faef8e71700 1 -- 172.21.15.67:0/2124305450 >> v1:172.21.15.72:6789/0 conn(0x7faef406a840 legacy=0x7faef4161df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:35:46.098 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:46.093+0000 7faef8e71700 1 -- 172.21.15.67:0/2124305450 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faef415e920 con 0x7faef40cd950 2023-10-14T10:35:46.098 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:46.093+0000 7faef8e71700 1 -- 172.21.15.67:0/2124305450 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3180043755 0 0) 0x7faee4001670 con 0x7faef40cd950 2023-10-14T10:35:46.099 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:46.093+0000 7faef8e71700 1 -- 172.21.15.67:0/2124305450 >> v1:172.21.15.67:6789/0 conn(0x7faef40cd950 legacy=0x7faef415f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:35:46.099 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:46.093+0000 7faef8e71700 1 --2- 172.21.15.67:0/2124305450 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7faef415e2f0 0x7faee0003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:35:46.099 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:46.093+0000 7faef8e71700 1 --2- 172.21.15.67:0/2124305450 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faee0004060 0x7faee0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:35:46.099 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:46.093+0000 7faef8e71700 1 --2- 172.21.15.67:0/2124305450 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faee0004d90 0x7faee0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:35:46.100 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:46.093+0000 7faef8e71700 1 -- 172.21.15.67:0/2124305450 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faef41619a0 con 0x7faee0004d90 2023-10-14T10:35:46.100 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:46.093+0000 7faef8e71700 1 -- 172.21.15.67:0/2124305450 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faee000bbd0 con 0x7faee0004060 2023-10-14T10:35:46.100 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:46.093+0000 7faef8e71700 1 -- 172.21.15.67:0/2124305450 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faee000bd90 con 0x7faef415e2f0 2023-10-14T10:35:46.100 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:46.093+0000 7faefa674700 1 --2- 172.21.15.67:0/2124305450 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faee0004d90 0x7faee0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:35:46.100 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:46.093+0000 7faef9672700 1 --2- 172.21.15.67:0/2124305450 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faee0004060 0x7faee0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:35:46.101 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:46.093+0000 7faef9e73700 1 --2- 172.21.15.67:0/2124305450 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7faef415e2f0 0x7faee0003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:35:46.101 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:46.093+0000 7faefa674700 1 -- 172.21.15.67:0/2124305450 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7faef415e2f0 msgr2=0x7faee0003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:35:46.101 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:46.093+0000 7faefa674700 1 --2- 172.21.15.67:0/2124305450 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7faef415e2f0 0x7faee0003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:46.101 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:46.093+0000 7faefa674700 1 -- 172.21.15.67:0/2124305450 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faee0004060 msgr2=0x7faee0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:35:46.101 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:46.093+0000 7faefa674700 1 --2- 172.21.15.67:0/2124305450 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faee0004060 0x7faee0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:46.101 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:46.093+0000 7faefa674700 1 -- 172.21.15.67:0/2124305450 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faee00035b0 con 0x7faee0004d90 2023-10-14T10:35:46.102 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:46.094+0000 7faefa674700 1 --2- 172.21.15.67:0/2124305450 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faee0004d90 0x7faee0005210 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7faeec001ed0 tx=0x7faee0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa5e84cd3d4b5edb server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:35:46.102 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:46.094+0000 7faef8e71700 1 -- 172.21.15.67:0/2124305450 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeec00cca0 con 0x7faee0004d90 2023-10-14T10:35:46.102 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:46.094+0000 7faef8e71700 1 -- 172.21.15.67:0/2124305450 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faeec00ce00 con 0x7faee0004d90 2023-10-14T10:35:46.102 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:46.094+0000 7faef8e71700 1 -- 172.21.15.67:0/2124305450 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeec003c40 con 0x7faee0004d90 2023-10-14T10:35:46.102 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:46.095+0000 7faefc0d7700 1 -- 172.21.15.67:0/2124305450 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faee0004d90 msgr2=0x7faee0005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:35:46.103 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:46.095+0000 7faefc0d7700 1 --2- 172.21.15.67:0/2124305450 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faee0004d90 0x7faee0005210 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7faeec001ed0 tx=0x7faee0005880 comp rx=0 tx=0).stop 2023-10-14T10:35:46.103 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:46.095+0000 7faefa674700 1 -- 172.21.15.67:0/2124305450 reap_dead start 2023-10-14T10:35:46.103 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:46.095+0000 7faefc0d7700 1 -- 172.21.15.67:0/2124305450 shutdown_connections 2023-10-14T10:35:46.103 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:46.095+0000 7faefc0d7700 1 -- 172.21.15.67:0/2124305450 >> 172.21.15.67:0/2124305450 conn(0x7faef406dae0 msgr2=0x7faef406def0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:35:46.103 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:46.095+0000 7faefc0d7700 1 -- 172.21.15.67:0/2124305450 shutdown_connections 2023-10-14T10:35:46.104 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:46.095+0000 7faefc0d7700 1 -- 172.21.15.67:0/2124305450 wait complete. 2023-10-14T10:35:46.106 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:46.096+0000 7faefc0d7700 1 Processor -- start 2023-10-14T10:35:46.106 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:46.096+0000 7faefc0d7700 1 -- start start 2023-10-14T10:35:46.106 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:46.097+0000 7faefc0d7700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7faee0004d90 0x7faef415d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:35:46.106 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:46.097+0000 7faefc0d7700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faef414a1e0 0x7faef40c7630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:35:46.107 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:46.097+0000 7faefc0d7700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faef40c7b70 0x7faef41f2370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:35:46.107 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:46.097+0000 7faefc0d7700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faef40c8020 con 0x7faef414a1e0 2023-10-14T10:35:46.107 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:46.097+0000 7faefc0d7700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faef41f3060 con 0x7faef40c7b70 2023-10-14T10:35:46.107 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:46.097+0000 7faefc0d7700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faef4149700 con 0x7faee0004d90 2023-10-14T10:35:46.107 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:46.098+0000 7faef9e73700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7faee0004d90 0x7faef415d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:35:46.107 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:46.098+0000 7faef9672700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faef414a1e0 0x7faef40c7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:35:46.108 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:46.098+0000 7faefa674700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faef40c7b70 0x7faef41f2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:35:46.108 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:46.098+0000 7faef9e73700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7faee0004d90 0x7faef415d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3301/0 says I am v2:172.21.15.67:59610/0 (socket says 172.21.15.67:59610) 2023-10-14T10:35:46.108 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:46.098+0000 7faef9e73700 1 -- 172.21.15.67:0/3599733441 learned_addr learned my addr 172.21.15.67:0/3599733441 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:35:46.108 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:46.098+0000 7faefa674700 1 -- 172.21.15.67:0/3599733441 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7faee0004d90 msgr2=0x7faef415d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:35:46.109 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:46.098+0000 7faefa674700 1 --2- 172.21.15.67:0/3599733441 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7faee0004d90 0x7faef415d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:46.109 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:46.098+0000 7faefa674700 1 -- 172.21.15.67:0/3599733441 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faef414a1e0 msgr2=0x7faef40c7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:35:46.109 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:46.098+0000 7faefa674700 1 --2- 172.21.15.67:0/3599733441 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faef414a1e0 0x7faef40c7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:46.109 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:46.098+0000 7faefa674700 1 -- 172.21.15.67:0/3599733441 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faef41499e0 con 0x7faef40c7b70 2023-10-14T10:35:46.109 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:46.098+0000 7faef9672700 1 --2- 172.21.15.67:0/3599733441 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faef414a1e0 0x7faef40c7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-14T10:35:46.109 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:46.098+0000 7faefa674700 1 --2- 172.21.15.67:0/3599733441 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faef40c7b70 0x7faef41f2370 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7faeec01c000 tx=0x7faeec0014a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:35:46.110 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:46.099+0000 7faeea7fc700 1 -- 172.21.15.67:0/3599733441 <== mon.1 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeec003e30 con 0x7faef40c7b70 2023-10-14T10:35:46.110 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:46.099+0000 7faeea7fc700 1 -- 172.21.15.67:0/3599733441 <== mon.1 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faeec0011c0 con 0x7faef40c7b70 2023-10-14T10:35:46.110 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:46.099+0000 7faefc0d7700 1 -- 172.21.15.67:0/3599733441 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faef4149cc0 con 0x7faef40c7b70 2023-10-14T10:35:46.110 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:46.099+0000 7faeea7fc700 1 -- 172.21.15.67:0/3599733441 <== mon.1 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeec005520 con 0x7faef40c7b70 2023-10-14T10:35:46.111 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:46.099+0000 7faefc0d7700 1 -- 172.21.15.67:0/3599733441 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faef415dd90 con 0x7faef40c7b70 2023-10-14T10:35:46.111 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:46.099+0000 7faeea7fc700 1 -- 172.21.15.67:0/3599733441 <== mon.1 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7faeec005680 con 0x7faef40c7b70 2023-10-14T10:35:46.111 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:46.099+0000 7faeea7fc700 1 -- 172.21.15.67:0/3599733441 <== mon.1 v2:172.21.15.72:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7faeec016cc0 con 0x7faef40c7b70 2023-10-14T10:35:46.111 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:46.100+0000 7faefc0d7700 1 -- 172.21.15.67:0/3599733441 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faef4067d20 con 0x7faef40c7b70 2023-10-14T10:35:46.111 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:46.103+0000 7faeea7fc700 1 -- 172.21.15.67:0/3599733441 <== mon.1 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7faef415dd90 con 0x7faef40c7b70 2023-10-14T10:35:46.193 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:46.191+0000 7faefc0d7700 1 -- 172.21.15.67:0/3599733441 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7faef40b2630 con 0x7faef40c7b70 2023-10-14T10:35:46.200 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:46.198+0000 7faeea7fc700 1 -- 172.21.15.67:0/3599733441 <== mon.1 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7faeec033020 con 0x7faef40c7b70 2023-10-14T10:35:46.202 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:46.199+0000 7faefc0d7700 1 -- 172.21.15.67:0/3599733441 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faef40c7b70 msgr2=0x7faef41f2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:35:46.202 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:46.199+0000 7faefc0d7700 1 --2- 172.21.15.67:0/3599733441 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faef40c7b70 0x7faef41f2370 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7faeec01c000 tx=0x7faeec0014a0 comp rx=0 tx=0).stop 2023-10-14T10:35:46.202 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:46.199+0000 7faefc0d7700 1 -- 172.21.15.67:0/3599733441 shutdown_connections 2023-10-14T10:35:46.203 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:46.199+0000 7faefc0d7700 1 --2- 172.21.15.67:0/3599733441 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faef40c7b70 0x7faef41f2370 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:46.203 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:46.199+0000 7faefc0d7700 1 --2- 172.21.15.67:0/3599733441 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7faee0004d90 0x7faef415d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:46.203 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:46.199+0000 7faefc0d7700 1 --2- 172.21.15.67:0/3599733441 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faef414a1e0 0x7faef40c7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:46.203 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:46.199+0000 7faefc0d7700 1 -- 172.21.15.67:0/3599733441 >> 172.21.15.67:0/3599733441 conn(0x7faef406dae0 msgr2=0x7faef406e340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:35:46.203 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:46.199+0000 7faefc0d7700 1 -- 172.21.15.67:0/3599733441 shutdown_connections 2023-10-14T10:35:46.204 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:46.199+0000 7faefc0d7700 1 -- 172.21.15.67:0/3599733441 wait complete. 2023-10-14T10:35:46.204 INFO:teuthology.orchestra.run.smithi067.stderr:adjusted tunables profile to default 2023-10-14T10:35:46.208 INFO:tasks.ceph.mgr.x.smithi072.stderr:2023-10-14T10:35:46.206+0000 7fa493d36200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-14T10:35:46.213 INFO:tasks.ceph:check_enable_crimson: False 2023-10-14T10:35:46.214 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-10-14T10:35:46.214 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2023-10-14T10:35:46.215 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-10-14T10:35:46.225 INFO:tasks.ceph.mgr.y.smithi067.stderr:2023-10-14T10:35:46.223+0000 7f928a49e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-14T10:35:46.242 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2023-10-14T10:35:46.242 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-10-14T10:35:46.273 INFO:tasks.ceph.mgr.x.smithi072.stderr:2023-10-14T10:35:46.271+0000 7fa493d36200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-14T10:35:46.294 INFO:tasks.ceph.mgr.y.smithi067.stderr:2023-10-14T10:35:46.293+0000 7f928a49e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-14T10:35:46.308 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2023-10-14T10:35:46.309 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-10-14T10:35:46.349 INFO:tasks.ceph.mgr.x.smithi072.stderr:2023-10-14T10:35:46.347+0000 7fa493d36200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-14T10:35:46.376 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2023-10-14T10:35:46.376 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-10-14T10:35:46.377 INFO:tasks.ceph.mgr.y.smithi067.stderr:2023-10-14T10:35:46.374+0000 7f928a49e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-14T10:35:46.403 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-10-14T10:35:46.404 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-10-14T10:35:46.430 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-10-14T10:35:46.430 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-10-14T10:35:46.431 INFO:tasks.ceph.mgr.x.smithi072.stderr:2023-10-14T10:35:46.427+0000 7fa493d36200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-14T10:35:46.461 INFO:tasks.ceph.mgr.y.smithi067.stderr:2023-10-14T10:35:46.460+0000 7f928a49e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-14T10:35:46.497 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-10-14T10:35:46.497 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-10-14T10:35:46.564 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-10-14T10:35:46.564 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-10-14T10:35:46.580 INFO:tasks.ceph.mgr.x.smithi072.stderr:2023-10-14T10:35:46.578+0000 7fa493d36200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-14T10:35:46.606 DEBUG:teuthology.orchestra.run.smithi072:> sudo ceph --cluster ceph osd new 2b59a07e-11ef-440c-b766-9f64ac885838 0 2023-10-14T10:35:46.611 INFO:tasks.ceph.mgr.y.smithi067.stderr:2023-10-14T10:35:46.610+0000 7f928a49e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-14T10:35:46.767 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.765+0000 7ffa7e95e700 1 Processor -- start 2023-10-14T10:35:46.768 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.765+0000 7ffa7e95e700 1 -- start start 2023-10-14T10:35:46.770 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.766+0000 7ffa7e95e700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7ffa7815e2f0 0x7ffa780cede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:35:46.771 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.766+0000 7ffa7e95e700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa780b2d60 con 0x7ffa7806a840 2023-10-14T10:35:46.771 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.766+0000 7ffa7e95e700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa780cdd30 con 0x7ffa780cd950 2023-10-14T10:35:46.771 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.766+0000 7ffa7e95e700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa781619a0 con 0x7ffa7815e2f0 2023-10-14T10:35:46.771 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.766+0000 7ffa777fe700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7ffa7815e2f0 0x7ffa780cede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:35:46.772 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.766+0000 7ffa77fff700 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7ffa780cd950 0x7ffa7815f570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:46870/0 (socket says 172.21.15.72:46870) 2023-10-14T10:35:46.772 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.766+0000 7ffa777fe700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7ffa7815e2f0 0x7ffa780cede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3301/0 says I am v2:172.21.15.72:54886/0 (socket says 172.21.15.72:54886) 2023-10-14T10:35:46.772 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.766+0000 7ffa77fff700 1 -- 172.21.15.72:0/1765388506 learned_addr learned my addr 172.21.15.72:0/1765388506 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-10-14T10:35:46.772 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.766+0000 7ffa777fe700 1 -- 172.21.15.72:0/1765388506 >> v1:172.21.15.72:6789/0 conn(0x7ffa780cd950 legacy=0x7ffa7815f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:35:46.773 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.766+0000 7ffa777fe700 1 -- 172.21.15.72:0/1765388506 >> v1:172.21.15.67:6789/0 conn(0x7ffa7806a840 legacy=0x7ffa78161df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:35:46.773 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.766+0000 7ffa777fe700 1 -- 172.21.15.72:0/1765388506 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa7815e920 con 0x7ffa7815e2f0 2023-10-14T10:35:46.773 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.766+0000 7ffa76ffd700 1 -- 172.21.15.72:0/1765388506 <== mon.1 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2060169024 0 0) 0x7ffa780cdd30 con 0x7ffa780cd950 2023-10-14T10:35:46.773 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.766+0000 7ffa777fe700 1 --2- 172.21.15.72:0/1765388506 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7ffa7815e2f0 0x7ffa780cede0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c009850 tx=0x7ffa6c009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0bc986ed88843b0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:35:46.773 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.767+0000 7ffa76ffd700 1 -- 172.21.15.72:0/1765388506 <== mon.2 v2:172.21.15.72:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa6c005e30 con 0x7ffa7815e2f0 2023-10-14T10:35:46.774 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.767+0000 7ffa76ffd700 1 -- 172.21.15.72:0/1765388506 <== mon.2 v2:172.21.15.72:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa6c006030 con 0x7ffa7815e2f0 2023-10-14T10:35:46.774 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.767+0000 7ffa76ffd700 1 -- 172.21.15.72:0/1765388506 <== mon.2 v2:172.21.15.72:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa6c0063c0 con 0x7ffa7815e2f0 2023-10-14T10:35:46.774 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.767+0000 7ffa7e95e700 1 -- 172.21.15.72:0/1765388506 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7ffa7815e2f0 msgr2=0x7ffa780cede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:35:46.774 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.767+0000 7ffa7e95e700 1 --2- 172.21.15.72:0/1765388506 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7ffa7815e2f0 0x7ffa780cede0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c009850 tx=0x7ffa6c009b60 comp rx=0 tx=0).stop 2023-10-14T10:35:46.775 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.767+0000 7ffa7e95e700 1 -- 172.21.15.72:0/1765388506 shutdown_connections 2023-10-14T10:35:46.775 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.767+0000 7ffa7e95e700 1 --2- 172.21.15.72:0/1765388506 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7ffa7815e2f0 0x7ffa780cede0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:46.775 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.767+0000 7ffa7e95e700 1 -- 172.21.15.72:0/1765388506 >> 172.21.15.72:0/1765388506 conn(0x7ffa7806dae0 msgr2=0x7ffa7806def0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:35:46.775 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.767+0000 7ffa7e95e700 1 -- 172.21.15.72:0/1765388506 shutdown_connections 2023-10-14T10:35:46.775 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.767+0000 7ffa7e95e700 1 -- 172.21.15.72:0/1765388506 wait complete. 2023-10-14T10:35:46.775 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.768+0000 7ffa7e95e700 1 Processor -- start 2023-10-14T10:35:46.776 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.769+0000 7ffa7e95e700 1 -- start start 2023-10-14T10:35:46.778 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.769+0000 7ffa7e95e700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffa7806a840 0x7ffa780adbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:35:46.778 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.769+0000 7ffa7e95e700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7ffa780cd950 0x7ffa780aebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:35:46.778 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.769+0000 7ffa7e95e700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffa7815e2f0 0x7ffa780c51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:35:46.779 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.769+0000 7ffa7e95e700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa781f37e0 con 0x7ffa7815e2f0 2023-10-14T10:35:46.779 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.769+0000 7ffa7e95e700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa781543f0 con 0x7ffa7806a840 2023-10-14T10:35:46.779 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.769+0000 7ffa7e95e700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa780c4130 con 0x7ffa780cd950 2023-10-14T10:35:46.779 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.769+0000 7ffa77fff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffa7806a840 0x7ffa780adbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:35:46.780 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.769+0000 7ffa777fe700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7ffa780cd950 0x7ffa780aebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:35:46.780 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.769+0000 7ffa77fff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffa7806a840 0x7ffa780adbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:51432/0 (socket says 172.21.15.72:51432) 2023-10-14T10:35:46.780 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.769+0000 7ffa77fff700 1 -- 172.21.15.72:0/3358406322 learned_addr learned my addr 172.21.15.72:0/3358406322 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-14T10:35:46.780 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.769+0000 7ffa7cefb700 1 --2- 172.21.15.72:0/3358406322 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffa7815e2f0 0x7ffa780c51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:35:46.780 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.770+0000 7ffa777fe700 1 -- 172.21.15.72:0/3358406322 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffa7806a840 msgr2=0x7ffa780adbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:35:46.781 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.770+0000 7ffa777fe700 1 --2- 172.21.15.72:0/3358406322 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffa7806a840 0x7ffa780adbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:46.781 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.770+0000 7ffa777fe700 1 -- 172.21.15.72:0/3358406322 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffa7815e2f0 msgr2=0x7ffa780c51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:35:46.781 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.770+0000 7ffa777fe700 1 --2- 172.21.15.72:0/3358406322 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffa7815e2f0 0x7ffa780c51d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:46.781 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.770+0000 7ffa777fe700 1 -- 172.21.15.72:0/3358406322 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa6c009430 con 0x7ffa780cd950 2023-10-14T10:35:46.781 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.770+0000 7ffa777fe700 1 --2- 172.21.15.72:0/3358406322 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7ffa780cd950 0x7ffa780aebb0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c015000 tx=0x7ffa6c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:35:46.782 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.770+0000 7ffa74ff9700 1 -- 172.21.15.72:0/3358406322 <== mon.2 v2:172.21.15.72:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa6c025480 con 0x7ffa780cd950 2023-10-14T10:35:46.782 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.770+0000 7ffa74ff9700 1 -- 172.21.15.72:0/3358406322 <== mon.2 v2:172.21.15.72:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa6c020070 con 0x7ffa780cd950 2023-10-14T10:35:46.782 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.770+0000 7ffa7e95e700 1 -- 172.21.15.72:0/3358406322 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa780c4410 con 0x7ffa780cd950 2023-10-14T10:35:46.782 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.770+0000 7ffa74ff9700 1 -- 172.21.15.72:0/3358406322 <== mon.2 v2:172.21.15.72:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa6c025840 con 0x7ffa780cd950 2023-10-14T10:35:46.782 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.770+0000 7ffa7e95e700 1 -- 172.21.15.72:0/3358406322 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa780c49d0 con 0x7ffa780cd950 2023-10-14T10:35:46.783 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.770+0000 7ffa74ff9700 1 -- 172.21.15.72:0/3358406322 <== mon.2 v2:172.21.15.72:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7ffa6c022030 con 0x7ffa780cd950 2023-10-14T10:35:46.783 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.770+0000 7ffa74ff9700 1 -- 172.21.15.72:0/3358406322 <== mon.2 v2:172.21.15.72:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7ffa6c011960 con 0x7ffa780cd950 2023-10-14T10:35:46.783 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.771+0000 7ffa7e95e700 1 -- 172.21.15.72:0/3358406322 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa78067d20 con 0x7ffa780cd950 2023-10-14T10:35:46.783 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.773+0000 7ffa74ff9700 1 -- 172.21.15.72:0/3358406322 <== mon.2 v2:172.21.15.72:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7ffa6c02a020 con 0x7ffa780cd950 2023-10-14T10:35:46.857 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.855+0000 7ffa7e95e700 1 -- 172.21.15.72:0/3358406322 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_command({"prefix": "osd new", "uuid": "2b59a07e-11ef-440c-b766-9f64ac885838", "id": 0} v 0) v1 -- 0x7ffa7806e550 con 0x7ffa780cd950 2023-10-14T10:35:46.860 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.858+0000 7ffa74ff9700 1 -- 172.21.15.72:0/3358406322 <== mon.2 v2:172.21.15.72:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2b59a07e-11ef-440c-b766-9f64ac885838", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7ffa6c011420 con 0x7ffa780cd950 2023-10-14T10:35:46.861 INFO:teuthology.orchestra.run.smithi072.stdout:0 2023-10-14T10:35:46.862 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.859+0000 7ffa7e95e700 1 -- 172.21.15.72:0/3358406322 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7ffa780cd950 msgr2=0x7ffa780aebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:35:46.862 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.859+0000 7ffa7e95e700 1 --2- 172.21.15.72:0/3358406322 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7ffa780cd950 0x7ffa780aebb0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c015000 tx=0x7ffa6c004750 comp rx=0 tx=0).stop 2023-10-14T10:35:46.862 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.859+0000 7ffa7e95e700 1 -- 172.21.15.72:0/3358406322 shutdown_connections 2023-10-14T10:35:46.863 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.859+0000 7ffa7e95e700 1 --2- 172.21.15.72:0/3358406322 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffa7815e2f0 0x7ffa780c51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:46.863 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.859+0000 7ffa7e95e700 1 --2- 172.21.15.72:0/3358406322 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffa7806a840 0x7ffa780adbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:46.863 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.859+0000 7ffa7e95e700 1 --2- 172.21.15.72:0/3358406322 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7ffa780cd950 0x7ffa780aebb0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:46.863 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.859+0000 7ffa7e95e700 1 -- 172.21.15.72:0/3358406322 >> 172.21.15.72:0/3358406322 conn(0x7ffa7806dae0 msgr2=0x7ffa780b19e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:35:46.864 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.859+0000 7ffa7e95e700 1 -- 172.21.15.72:0/3358406322 shutdown_connections 2023-10-14T10:35:46.864 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.859+0000 7ffa7e95e700 1 -- 172.21.15.72:0/3358406322 wait complete. 2023-10-14T10:35:46.873 DEBUG:teuthology.orchestra.run.smithi072:> sudo ceph --cluster ceph osd new 6f813bd6-fa0b-487d-a204-431eb1276a60 1 2023-10-14T10:35:46.996 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.994+0000 7f9b512f2700 1 Processor -- start 2023-10-14T10:35:46.996 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.994+0000 7f9b512f2700 1 -- start start 2023-10-14T10:35:46.999 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.994+0000 7f9b512f2700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9b4c15ab10 0x7f9b4c15f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:35:46.999 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.994+0000 7f9b512f2700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9b4c0b1f50 0x7f9b4c0cf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:35:46.999 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.994+0000 7f9b512f2700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f9b4c0b2400 0x7f9b4c0cfc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:35:46.999 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.994+0000 7f9b512f2700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b4c15e7a0 con 0x7f9b4c15ab10 2023-10-14T10:35:47.000 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.994+0000 7f9b512f2700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b4c15f730 con 0x7f9b4c0b1f50 2023-10-14T10:35:47.000 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.994+0000 7f9b512f2700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b4c0b29a0 con 0x7f9b4c0b2400 2023-10-14T10:35:47.000 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.995+0000 7f9b4a7fc700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9b4c0b1f50 0x7f9b4c0cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:35:47.000 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.995+0000 7f9b4b7fe700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f9b4c0b2400 0x7f9b4c0cfc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:35:47.000 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.995+0000 7f9b4a7fc700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9b4c0b1f50 0x7f9b4c0cf720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:51440/0 (socket says 172.21.15.72:51440) 2023-10-14T10:35:47.001 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.995+0000 7f9b4a7fc700 1 -- 172.21.15.72:0/1408446398 learned_addr learned my addr 172.21.15.72:0/1408446398 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-14T10:35:47.001 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.995+0000 7f9b4affd700 1 --2- 172.21.15.72:0/1408446398 >> v2:172.21.15.67:3300/0 conn(0x7f9b4c15ab10 0x7f9b4c15f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:35:47.001 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.995+0000 7f9b4a7fc700 1 -- 172.21.15.72:0/1408446398 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f9b4c0b2400 msgr2=0x7f9b4c0cfc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:35:47.001 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.995+0000 7f9b4a7fc700 1 --2- 172.21.15.72:0/1408446398 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f9b4c0b2400 0x7f9b4c0cfc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:47.001 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.995+0000 7f9b4a7fc700 1 -- 172.21.15.72:0/1408446398 >> v2:172.21.15.67:3300/0 conn(0x7f9b4c15ab10 msgr2=0x7f9b4c15f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:35:47.002 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.995+0000 7f9b4a7fc700 1 --2- 172.21.15.72:0/1408446398 >> v2:172.21.15.67:3300/0 conn(0x7f9b4c15ab10 0x7f9b4c15f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:47.002 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.995+0000 7f9b4a7fc700 1 -- 172.21.15.72:0/1408446398 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b4c0b2c80 con 0x7f9b4c0b1f50 2023-10-14T10:35:47.002 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.995+0000 7f9b4affd700 1 --2- 172.21.15.72:0/1408446398 >> v2:172.21.15.67:3300/0 conn(0x7f9b4c15ab10 0x7f9b4c15f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-14T10:35:47.002 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.995+0000 7f9b4a7fc700 1 --2- 172.21.15.72:0/1408446398 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9b4c0b1f50 0x7f9b4c0cf720 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c06d470 tx=0x7f9b40009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e27db1e7ca84fad server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:35:47.002 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.995+0000 7f9b49ffb700 1 -- 172.21.15.72:0/1408446398 <== mon.1 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b40005e30 con 0x7f9b4c0b1f50 2023-10-14T10:35:47.003 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.995+0000 7f9b49ffb700 1 -- 172.21.15.72:0/1408446398 <== mon.1 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b40004030 con 0x7f9b4c0b1f50 2023-10-14T10:35:47.003 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.995+0000 7f9b49ffb700 1 -- 172.21.15.72:0/1408446398 <== mon.1 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b400043e0 con 0x7f9b4c0b1f50 2023-10-14T10:35:47.003 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.996+0000 7f9b512f2700 1 -- 172.21.15.72:0/1408446398 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9b4c0b1f50 msgr2=0x7f9b4c0cf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:35:47.003 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.996+0000 7f9b512f2700 1 --2- 172.21.15.72:0/1408446398 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9b4c0b1f50 0x7f9b4c0cf720 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c06d470 tx=0x7f9b40009bd0 comp rx=0 tx=0).stop 2023-10-14T10:35:47.003 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.996+0000 7f9b512f2700 1 -- 172.21.15.72:0/1408446398 shutdown_connections 2023-10-14T10:35:47.004 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.996+0000 7f9b512f2700 1 --2- 172.21.15.72:0/1408446398 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f9b4c0b2400 0x7f9b4c0cfc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:47.004 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.996+0000 7f9b512f2700 1 --2- 172.21.15.72:0/1408446398 >> v2:172.21.15.67:3300/0 conn(0x7f9b4c15ab10 0x7f9b4c15f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:47.004 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.996+0000 7f9b512f2700 1 --2- 172.21.15.72:0/1408446398 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9b4c0b1f50 0x7f9b4c0cf720 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:47.004 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.996+0000 7f9b512f2700 1 -- 172.21.15.72:0/1408446398 >> 172.21.15.72:0/1408446398 conn(0x7f9b4c06dae0 msgr2=0x7f9b4c06def0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:35:47.004 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.996+0000 7f9b512f2700 1 -- 172.21.15.72:0/1408446398 shutdown_connections 2023-10-14T10:35:47.005 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.996+0000 7f9b512f2700 1 -- 172.21.15.72:0/1408446398 wait complete. 2023-10-14T10:35:47.005 INFO:tasks.ceph.mgr.x.smithi072.stderr:2023-10-14T10:35:46.996+0000 7fa493d36200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-14T10:35:47.008 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.997+0000 7f9b512f2700 1 Processor -- start 2023-10-14T10:35:47.008 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.997+0000 7f9b512f2700 1 -- start start 2023-10-14T10:35:47.008 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.998+0000 7f9b512f2700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9b4c0b1f50 0x7f9b4c0b4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:35:47.008 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.998+0000 7f9b512f2700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b4c0b2400 0x7f9b4c151370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:35:47.009 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.998+0000 7f9b512f2700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f9b4c15ab10 0x7f9b4c0c42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:35:47.009 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.998+0000 7f9b512f2700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b4c0b4720 con 0x7f9b4c0b2400 2023-10-14T10:35:47.009 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.998+0000 7f9b512f2700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b4c0b3ae0 con 0x7f9b4c0b1f50 2023-10-14T10:35:47.009 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.998+0000 7f9b512f2700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b4c0b3d80 con 0x7f9b4c15ab10 2023-10-14T10:35:47.010 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.998+0000 7f9b4b7fe700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f9b4c15ab10 0x7f9b4c0c42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:35:47.010 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.998+0000 7f9b4affd700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9b4c0b1f50 0x7f9b4c0b4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:35:47.010 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.998+0000 7f9b4affd700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9b4c0b1f50 0x7f9b4c0b4ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:51444/0 (socket says 172.21.15.72:51444) 2023-10-14T10:35:47.010 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.998+0000 7f9b4b7fe700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f9b4c15ab10 0x7f9b4c0c42e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3301/0 says I am v2:172.21.15.72:54906/0 (socket says 172.21.15.72:54906) 2023-10-14T10:35:47.011 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.998+0000 7f9b4affd700 1 -- 172.21.15.72:0/189854826 learned_addr learned my addr 172.21.15.72:0/189854826 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-14T10:35:47.011 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.998+0000 7f9b4a7fc700 1 --2- 172.21.15.72:0/189854826 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b4c0b2400 0x7f9b4c151370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:35:47.011 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.998+0000 7f9b4affd700 1 -- 172.21.15.72:0/189854826 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f9b4c15ab10 msgr2=0x7f9b4c0c42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:35:47.011 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.998+0000 7f9b4affd700 1 --2- 172.21.15.72:0/189854826 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f9b4c15ab10 0x7f9b4c0c42e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:47.011 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.998+0000 7f9b4affd700 1 -- 172.21.15.72:0/189854826 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b4c0b2400 msgr2=0x7f9b4c151370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:35:47.012 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.998+0000 7f9b4affd700 1 --2- 172.21.15.72:0/189854826 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b4c0b2400 0x7f9b4c151370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:47.012 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.998+0000 7f9b4affd700 1 -- 172.21.15.72:0/189854826 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b40009500 con 0x7f9b4c0b1f50 2023-10-14T10:35:47.012 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.999+0000 7f9b4affd700 1 --2- 172.21.15.72:0/189854826 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9b4c0b1f50 0x7f9b4c0b4ff0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c00e810 tx=0x7f9b3c00eb20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:35:47.013 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.999+0000 7f9b337fe700 1 -- 172.21.15.72:0/189854826 <== mon.1 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b3c00c070 con 0x7f9b4c0b1f50 2023-10-14T10:35:47.013 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.999+0000 7f9b337fe700 1 -- 172.21.15.72:0/189854826 <== mon.1 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b3c004230 con 0x7f9b4c0b1f50 2023-10-14T10:35:47.013 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.999+0000 7f9b337fe700 1 -- 172.21.15.72:0/189854826 <== mon.1 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b3c0036a0 con 0x7f9b4c0b1f50 2023-10-14T10:35:47.013 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.999+0000 7f9b512f2700 1 -- 172.21.15.72:0/189854826 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b4c0b4060 con 0x7f9b4c0b1f50 2023-10-14T10:35:47.013 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.999+0000 7f9b512f2700 1 -- 172.21.15.72:0/189854826 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b4c0b5530 con 0x7f9b4c0b1f50 2023-10-14T10:35:47.014 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.999+0000 7f9b337fe700 1 -- 172.21.15.72:0/189854826 <== mon.1 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f9b3c003b00 con 0x7f9b4c0b1f50 2023-10-14T10:35:47.014 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:46.999+0000 7f9b337fe700 1 -- 172.21.15.72:0/189854826 <== mon.1 v2:172.21.15.72:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f9b3c01c400 con 0x7f9b4c0b1f50 2023-10-14T10:35:47.014 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.000+0000 7f9b512f2700 1 -- 172.21.15.72:0/189854826 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b4c0c4130 con 0x7f9b4c0b1f50 2023-10-14T10:35:47.015 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.002+0000 7f9b337fe700 1 -- 172.21.15.72:0/189854826 <== mon.1 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f9b3c00f040 con 0x7f9b4c0b1f50 2023-10-14T10:35:47.018 INFO:tasks.ceph.mgr.y.smithi067.stderr:2023-10-14T10:35:47.016+0000 7f928a49e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-14T10:35:47.088 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.086+0000 7f9b512f2700 1 -- 172.21.15.72:0/189854826 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd new", "uuid": "6f813bd6-fa0b-487d-a204-431eb1276a60", "id": 1} v 0) v1 -- 0x7f9b4c067d20 con 0x7f9b4c0b1f50 2023-10-14T10:35:47.095 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.093+0000 7f9b337fe700 1 -- 172.21.15.72:0/189854826 <== mon.1 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6f813bd6-fa0b-487d-a204-431eb1276a60", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f9b3c013410 con 0x7f9b4c0b1f50 2023-10-14T10:35:47.096 INFO:teuthology.orchestra.run.smithi072.stdout:1 2023-10-14T10:35:47.097 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.094+0000 7f9b512f2700 1 -- 172.21.15.72:0/189854826 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9b4c0b1f50 msgr2=0x7f9b4c0b4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:35:47.097 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.094+0000 7f9b512f2700 1 --2- 172.21.15.72:0/189854826 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9b4c0b1f50 0x7f9b4c0b4ff0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c00e810 tx=0x7f9b3c00eb20 comp rx=0 tx=0).stop 2023-10-14T10:35:47.097 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.094+0000 7f9b512f2700 1 -- 172.21.15.72:0/189854826 shutdown_connections 2023-10-14T10:35:47.098 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.094+0000 7f9b512f2700 1 --2- 172.21.15.72:0/189854826 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f9b4c15ab10 0x7f9b4c0c42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:47.098 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.094+0000 7f9b512f2700 1 --2- 172.21.15.72:0/189854826 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9b4c0b1f50 0x7f9b4c0b4ff0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:47.098 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.094+0000 7f9b512f2700 1 --2- 172.21.15.72:0/189854826 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b4c0b2400 0x7f9b4c151370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:47.098 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.094+0000 7f9b512f2700 1 -- 172.21.15.72:0/189854826 >> 172.21.15.72:0/189854826 conn(0x7f9b4c06dae0 msgr2=0x7f9b4c06a840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:35:47.098 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.094+0000 7f9b512f2700 1 -- 172.21.15.72:0/189854826 shutdown_connections 2023-10-14T10:35:47.099 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.094+0000 7f9b512f2700 1 -- 172.21.15.72:0/189854826 wait complete. 2023-10-14T10:35:47.108 DEBUG:teuthology.orchestra.run.smithi072:> sudo ceph --cluster ceph osd new 6af42c6a-1b59-4669-9c04-b6876788446d 2 2023-10-14T10:35:47.125 INFO:tasks.ceph.mgr.x.smithi072.stderr:2023-10-14T10:35:47.123+0000 7fa493d36200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-14T10:35:47.141 INFO:tasks.ceph.mgr.y.smithi067.stderr:2023-10-14T10:35:47.139+0000 7f928a49e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-14T10:35:47.208 INFO:tasks.ceph.mgr.x.smithi072.stderr:2023-10-14T10:35:47.206+0000 7fa493d36200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-14T10:35:47.231 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.228+0000 7fb1caa36700 1 Processor -- start 2023-10-14T10:35:47.231 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.228+0000 7fb1caa36700 1 -- start start 2023-10-14T10:35:47.234 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.229+0000 7fb1caa36700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fb1c415e030 0x7fb1c40cb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:35:47.234 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.229+0000 7fb1caa36700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fb1c406a840 0x7fb1c40cdb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:35:47.234 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.229+0000 7fb1caa36700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1c406adb0 con 0x7fb1c406be50 2023-10-14T10:35:47.234 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.229+0000 7fb1caa36700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1c406b090 con 0x7fb1c406a840 2023-10-14T10:35:47.234 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.229+0000 7fb1caa36700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c406b370 con 0x7fb1c415e030 2023-10-14T10:35:47.235 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.229+0000 7fb1c37fe700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fb1c406a840 0x7fb1c40cdb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:35:47.235 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.229+0000 7fb1c3fff700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fb1c415e030 0x7fb1c40cb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:35:47.235 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.229+0000 7fb1c37fe700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fb1c406a840 0x7fb1c40cdb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:51452/0 (socket says 172.21.15.72:51452) 2023-10-14T10:35:47.235 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.229+0000 7fb1c37fe700 1 -- 172.21.15.72:0/1074222854 learned_addr learned my addr 172.21.15.72:0/1074222854 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-14T10:35:47.235 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.230+0000 7fb1c37fe700 1 -- 172.21.15.72:0/1074222854 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fb1c415e030 msgr2=0x7fb1c40cb3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:35:47.236 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.230+0000 7fb1c37fe700 1 --2- 172.21.15.72:0/1074222854 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fb1c415e030 0x7fb1c40cb3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:47.236 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.230+0000 7fb1c37fe700 1 -- 172.21.15.72:0/1074222854 >> v1:172.21.15.67:6789/0 conn(0x7fb1c406be50 legacy=0x7fb1c406d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:35:47.236 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.230+0000 7fb1c37fe700 1 -- 172.21.15.72:0/1074222854 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1c406b650 con 0x7fb1c406a840 2023-10-14T10:35:47.236 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.230+0000 7fb1c3fff700 1 --2- 172.21.15.72:0/1074222854 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fb1c415e030 0x7fb1c40cb3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-14T10:35:47.236 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.230+0000 7fb1c37fe700 1 --2- 172.21.15.72:0/1074222854 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb1c406a840 0x7fb1c40cdb70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb1b80098f0 tx=0x7fb1b8009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b062b5117f330dad server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:35:47.237 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.230+0000 7fb1c2ffd700 1 -- 172.21.15.72:0/1074222854 <== mon.1 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b8011830 con 0x7fb1c406a840 2023-10-14T10:35:47.237 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.230+0000 7fb1c2ffd700 1 -- 172.21.15.72:0/1074222854 <== mon.1 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1b8011990 con 0x7fb1c406a840 2023-10-14T10:35:47.237 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.230+0000 7fb1c2ffd700 1 -- 172.21.15.72:0/1074222854 <== mon.1 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b8011d20 con 0x7fb1c406a840 2023-10-14T10:35:47.237 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.230+0000 7fb1caa36700 1 -- 172.21.15.72:0/1074222854 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb1c406a840 msgr2=0x7fb1c40cdb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:35:47.237 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.230+0000 7fb1caa36700 1 --2- 172.21.15.72:0/1074222854 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb1c406a840 0x7fb1c40cdb70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb1b80098f0 tx=0x7fb1b8009c00 comp rx=0 tx=0).stop 2023-10-14T10:35:47.238 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.230+0000 7fb1caa36700 1 -- 172.21.15.72:0/1074222854 shutdown_connections 2023-10-14T10:35:47.238 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.230+0000 7fb1caa36700 1 --2- 172.21.15.72:0/1074222854 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fb1c415e030 0x7fb1c40cb3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:47.238 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.230+0000 7fb1caa36700 1 --2- 172.21.15.72:0/1074222854 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb1c406a840 0x7fb1c40cdb70 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:47.238 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.230+0000 7fb1caa36700 1 -- 172.21.15.72:0/1074222854 >> 172.21.15.72:0/1074222854 conn(0x7fb1c40cc6f0 msgr2=0x7fb1c40ccb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:35:47.238 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.231+0000 7fb1caa36700 1 -- 172.21.15.72:0/1074222854 shutdown_connections 2023-10-14T10:35:47.239 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.231+0000 7fb1caa36700 1 -- 172.21.15.72:0/1074222854 wait complete. 2023-10-14T10:35:47.239 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.232+0000 7fb1caa36700 1 Processor -- start 2023-10-14T10:35:47.239 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.232+0000 7fb1caa36700 1 -- start start 2023-10-14T10:35:47.240 INFO:tasks.ceph.mgr.y.smithi067.stderr:2023-10-14T10:35:47.230+0000 7f928a49e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-14T10:35:47.242 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.233+0000 7fb1caa36700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fb1c406a840 0x7fb1c40b3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:35:47.242 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.233+0000 7fb1caa36700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb1c406be50 0x7fb1c41f7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:35:47.242 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.233+0000 7fb1caa36700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb1c415e030 0x7fb1c41f75d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:35:47.243 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.233+0000 7fb1caa36700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c4159d10 con 0x7fb1c415e030 2023-10-14T10:35:47.243 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.233+0000 7fb1caa36700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c40b3d90 con 0x7fb1c406be50 2023-10-14T10:35:47.243 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.233+0000 7fb1caa36700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c4160d60 con 0x7fb1c406a840 2023-10-14T10:35:47.243 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.233+0000 7fb1c37fe700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb1c406be50 0x7fb1c41f7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:35:47.243 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.233+0000 7fb1c3fff700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fb1c406a840 0x7fb1c40b3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:35:47.244 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.233+0000 7fb1c37fe700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb1c406be50 0x7fb1c41f7090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:51458/0 (socket says 172.21.15.72:51458) 2023-10-14T10:35:47.244 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.233+0000 7fb1c37fe700 1 -- 172.21.15.72:0/3042341993 learned_addr learned my addr 172.21.15.72:0/3042341993 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-14T10:35:47.244 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.233+0000 7fb1c8fd3700 1 --2- 172.21.15.72:0/3042341993 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb1c415e030 0x7fb1c41f75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:35:47.244 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.233+0000 7fb1c37fe700 1 -- 172.21.15.72:0/3042341993 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fb1c406a840 msgr2=0x7fb1c40b3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:35:47.244 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.233+0000 7fb1c37fe700 1 --2- 172.21.15.72:0/3042341993 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fb1c406a840 0x7fb1c40b3850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:47.245 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.233+0000 7fb1c37fe700 1 -- 172.21.15.72:0/3042341993 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb1c415e030 msgr2=0x7fb1c41f75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:35:47.245 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.233+0000 7fb1c37fe700 1 --2- 172.21.15.72:0/3042341993 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb1c415e030 0x7fb1c41f75d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:47.245 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.233+0000 7fb1c37fe700 1 -- 172.21.15.72:0/3042341993 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1b40094a0 con 0x7fb1c406be50 2023-10-14T10:35:47.245 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.233+0000 7fb1c37fe700 1 --2- 172.21.15.72:0/3042341993 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb1c406be50 0x7fb1c41f7090 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb1b8014040 tx=0x7fb1b8004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:35:47.245 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.233+0000 7fb1c0ff9700 1 -- 172.21.15.72:0/3042341993 <== mon.1 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b8004510 con 0x7fb1c406be50 2023-10-14T10:35:47.246 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.233+0000 7fb1c0ff9700 1 -- 172.21.15.72:0/3042341993 <== mon.1 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1b801e070 con 0x7fb1c406be50 2023-10-14T10:35:47.246 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.233+0000 7fb1caa36700 1 -- 172.21.15.72:0/3042341993 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1b8009550 con 0x7fb1c406be50 2023-10-14T10:35:47.246 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.233+0000 7fb1c0ff9700 1 -- 172.21.15.72:0/3042341993 <== mon.1 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b80038e0 con 0x7fb1c406be50 2023-10-14T10:35:47.246 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.233+0000 7fb1caa36700 1 -- 172.21.15.72:0/3042341993 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1c415ffa0 con 0x7fb1c406be50 2023-10-14T10:35:47.246 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.234+0000 7fb1c0ff9700 1 -- 172.21.15.72:0/3042341993 <== mon.1 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fb1b8003d40 con 0x7fb1c406be50 2023-10-14T10:35:47.247 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.234+0000 7fb1c0ff9700 1 -- 172.21.15.72:0/3042341993 <== mon.1 v2:172.21.15.72:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7fb1b8024400 con 0x7fb1c406be50 2023-10-14T10:35:47.247 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.234+0000 7fb1caa36700 1 -- 172.21.15.72:0/3042341993 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1c4067d20 con 0x7fb1c406be50 2023-10-14T10:35:47.247 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.236+0000 7fb1c0ff9700 1 -- 172.21.15.72:0/3042341993 <== mon.1 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7fb1b8032060 con 0x7fb1c406be50 2023-10-14T10:35:47.295 INFO:tasks.ceph.mgr.x.smithi072.stderr:2023-10-14T10:35:47.293+0000 7fa493d36200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-14T10:35:47.317 INFO:tasks.ceph.mgr.y.smithi067.stderr:2023-10-14T10:35:47.315+0000 7f928a49e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-14T10:35:47.321 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.319+0000 7fb1caa36700 1 -- 172.21.15.72:0/3042341993 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd new", "uuid": "6af42c6a-1b59-4669-9c04-b6876788446d", "id": 2} v 0) v1 -- 0x7fb1c40c9d70 con 0x7fb1c406be50 2023-10-14T10:35:47.326 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.324+0000 7fb1c0ff9700 1 -- 172.21.15.72:0/3042341993 <== mon.1 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6af42c6a-1b59-4669-9c04-b6876788446d", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7fb1b8020070 con 0x7fb1c406be50 2023-10-14T10:35:47.326 INFO:teuthology.orchestra.run.smithi072.stdout:2 2023-10-14T10:35:47.328 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.325+0000 7fb1caa36700 1 -- 172.21.15.72:0/3042341993 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb1c406be50 msgr2=0x7fb1c41f7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:35:47.328 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.325+0000 7fb1caa36700 1 --2- 172.21.15.72:0/3042341993 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb1c406be50 0x7fb1c41f7090 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb1b8014040 tx=0x7fb1b8004750 comp rx=0 tx=0).stop 2023-10-14T10:35:47.328 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.325+0000 7fb1caa36700 1 -- 172.21.15.72:0/3042341993 shutdown_connections 2023-10-14T10:35:47.328 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.325+0000 7fb1caa36700 1 --2- 172.21.15.72:0/3042341993 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb1c415e030 0x7fb1c41f75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:47.329 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.325+0000 7fb1caa36700 1 --2- 172.21.15.72:0/3042341993 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fb1c406a840 0x7fb1c40b3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:47.329 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.325+0000 7fb1caa36700 1 --2- 172.21.15.72:0/3042341993 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb1c406be50 0x7fb1c41f7090 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:47.329 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.325+0000 7fb1caa36700 1 -- 172.21.15.72:0/3042341993 >> 172.21.15.72:0/3042341993 conn(0x7fb1c40cc6f0 msgr2=0x7fb1c415c8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:35:47.329 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.325+0000 7fb1caa36700 1 -- 172.21.15.72:0/3042341993 shutdown_connections 2023-10-14T10:35:47.330 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.325+0000 7fb1caa36700 1 -- 172.21.15.72:0/3042341993 wait complete. 2023-10-14T10:35:47.339 DEBUG:teuthology.orchestra.run.smithi072:> sudo ceph --cluster ceph osd new 1d889d76-a646-499d-bfff-ed5bae5434ff 3 2023-10-14T10:35:47.502 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.500+0000 7f5c7473a700 1 Processor -- start 2023-10-14T10:35:47.502 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.500+0000 7f5c7473a700 1 -- start start 2023-10-14T10:35:47.505 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.501+0000 7f5c7473a700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f5c6c15ab10 0x7f5c6c15f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:35:47.505 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.501+0000 7f5c7473a700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f5c6c0cfe30 0x7f5c6c0cbf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:35:47.505 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.501+0000 7f5c7473a700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c6c15e7a0 con 0x7f5c6c0b1f50 2023-10-14T10:35:47.506 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.501+0000 7f5c7473a700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c6c15f730 con 0x7f5c6c15ab10 2023-10-14T10:35:47.506 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.501+0000 7f5c7473a700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c6c0b29a0 con 0x7f5c6c0cfe30 2023-10-14T10:35:47.506 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.501+0000 7f5c72cd7700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f5c6c0cfe30 0x7f5c6c0cbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:35:47.506 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.501+0000 7f5c72cd7700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f5c6c0cfe30 0x7f5c6c0cbf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:51466/0 (socket says 172.21.15.72:51466) 2023-10-14T10:35:47.506 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.501+0000 7f5c72cd7700 1 -- 172.21.15.72:0/3084323770 learned_addr learned my addr 172.21.15.72:0/3084323770 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-14T10:35:47.507 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.501+0000 7f5c724d6700 1 --2- 172.21.15.72:0/3084323770 >> v2:172.21.15.67:3300/0 conn(0x7f5c6c15ab10 0x7f5c6c15f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:35:47.507 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.501+0000 7f5c72cd7700 1 -- 172.21.15.72:0/3084323770 >> v2:172.21.15.67:3300/0 conn(0x7f5c6c15ab10 msgr2=0x7f5c6c15f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:35:47.507 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.501+0000 7f5c72cd7700 1 --2- 172.21.15.72:0/3084323770 >> v2:172.21.15.67:3300/0 conn(0x7f5c6c15ab10 0x7f5c6c15f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:47.507 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.501+0000 7f5c72cd7700 1 -- 172.21.15.72:0/3084323770 >> v1:172.21.15.67:6789/0 conn(0x7f5c6c0b1f50 legacy=0x7f5c6c0cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:35:47.507 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.501+0000 7f5c72cd7700 1 -- 172.21.15.72:0/3084323770 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c6c0b2c80 con 0x7f5c6c0cfe30 2023-10-14T10:35:47.508 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.501+0000 7f5c72cd7700 1 --2- 172.21.15.72:0/3084323770 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5c6c0cfe30 0x7f5c6c0cbf40 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c0d03e0 tx=0x7f5c6400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85c2816732737185 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:35:47.508 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.502+0000 7f5c714d4700 1 -- 172.21.15.72:0/3084323770 <== mon.1 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6400e070 con 0x7f5c6c0cfe30 2023-10-14T10:35:47.508 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.502+0000 7f5c714d4700 1 -- 172.21.15.72:0/3084323770 <== mon.1 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c6400b910 con 0x7f5c6c0cfe30 2023-10-14T10:35:47.509 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.502+0000 7f5c714d4700 1 -- 172.21.15.72:0/3084323770 <== mon.1 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6400bcc0 con 0x7f5c6c0cfe30 2023-10-14T10:35:47.509 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.502+0000 7f5c7473a700 1 -- 172.21.15.72:0/3084323770 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5c6c0cfe30 msgr2=0x7f5c6c0cbf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:35:47.509 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.502+0000 7f5c7473a700 1 --2- 172.21.15.72:0/3084323770 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5c6c0cfe30 0x7f5c6c0cbf40 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c0d03e0 tx=0x7f5c6400b330 comp rx=0 tx=0).stop 2023-10-14T10:35:47.509 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.502+0000 7f5c7473a700 1 -- 172.21.15.72:0/3084323770 shutdown_connections 2023-10-14T10:35:47.509 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.502+0000 7f5c7473a700 1 --2- 172.21.15.72:0/3084323770 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5c6c0cfe30 0x7f5c6c0cbf40 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:47.510 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.502+0000 7f5c7473a700 1 --2- 172.21.15.72:0/3084323770 >> v2:172.21.15.67:3300/0 conn(0x7f5c6c15ab10 0x7f5c6c15f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:47.510 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.502+0000 7f5c7473a700 1 -- 172.21.15.72:0/3084323770 >> 172.21.15.72:0/3084323770 conn(0x7f5c6c06dae0 msgr2=0x7f5c6c06def0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:35:47.510 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.502+0000 7f5c7473a700 1 -- 172.21.15.72:0/3084323770 shutdown_connections 2023-10-14T10:35:47.510 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.502+0000 7f5c7473a700 1 -- 172.21.15.72:0/3084323770 wait complete. 2023-10-14T10:35:47.513 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.503+0000 7f5c7473a700 1 Processor -- start 2023-10-14T10:35:47.513 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.504+0000 7f5c7473a700 1 -- start start 2023-10-14T10:35:47.514 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.504+0000 7f5c7473a700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f5c6c0b1f50 0x7f5c6c15ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:35:47.514 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.504+0000 7f5c7473a700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5c6c0cfe30 0x7f5c6c156ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:35:47.514 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.504+0000 7f5c7473a700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5c6c15ab10 0x7f5c6c0cc830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:35:47.514 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.504+0000 7f5c7473a700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c6c15c590 con 0x7f5c6c0cfe30 2023-10-14T10:35:47.515 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.504+0000 7f5c7473a700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c6c15cf60 con 0x7f5c6c15ab10 2023-10-14T10:35:47.515 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.504+0000 7f5c7473a700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c6c0aef00 con 0x7f5c6c0b1f50 2023-10-14T10:35:47.515 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.504+0000 7f5c72cd7700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5c6c15ab10 0x7f5c6c0cc830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:35:47.515 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.504+0000 7f5c724d6700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f5c6c0b1f50 0x7f5c6c15ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:35:47.516 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.504+0000 7f5c72cd7700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5c6c15ab10 0x7f5c6c0cc830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:51472/0 (socket says 172.21.15.72:51472) 2023-10-14T10:35:47.516 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.504+0000 7f5c724d6700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f5c6c0b1f50 0x7f5c6c15ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3301/0 says I am v2:172.21.15.72:54926/0 (socket says 172.21.15.72:54926) 2023-10-14T10:35:47.516 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.504+0000 7f5c72cd7700 1 -- 172.21.15.72:0/1970070782 learned_addr learned my addr 172.21.15.72:0/1970070782 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-14T10:35:47.516 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.504+0000 7f5c71cd5700 1 --2- 172.21.15.72:0/1970070782 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5c6c0cfe30 0x7f5c6c156ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:35:47.517 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.505+0000 7f5c72cd7700 1 -- 172.21.15.72:0/1970070782 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f5c6c0b1f50 msgr2=0x7f5c6c15ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:35:47.517 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.505+0000 7f5c72cd7700 1 --2- 172.21.15.72:0/1970070782 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f5c6c0b1f50 0x7f5c6c15ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:47.517 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.505+0000 7f5c72cd7700 1 -- 172.21.15.72:0/1970070782 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5c6c0cfe30 msgr2=0x7f5c6c156ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:35:47.517 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.505+0000 7f5c72cd7700 1 --2- 172.21.15.72:0/1970070782 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5c6c0cfe30 0x7f5c6c156ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:47.517 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.505+0000 7f5c72cd7700 1 -- 172.21.15.72:0/1970070782 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c64009b30 con 0x7f5c6c15ab10 2023-10-14T10:35:47.518 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.505+0000 7f5c724d6700 1 --2- 172.21.15.72:0/1970070782 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f5c6c0b1f50 0x7f5c6c15ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-14T10:35:47.518 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.505+0000 7f5c72cd7700 1 --2- 172.21.15.72:0/1970070782 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5c6c15ab10 0x7f5c6c0cc830 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5c64012000 tx=0x7f5c64004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:35:47.518 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.505+0000 7f5c62ffd700 1 -- 172.21.15.72:0/1970070782 <== mon.1 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c64004880 con 0x7f5c6c15ab10 2023-10-14T10:35:47.519 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.505+0000 7f5c62ffd700 1 -- 172.21.15.72:0/1970070782 <== mon.1 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c640099b0 con 0x7f5c6c15ab10 2023-10-14T10:35:47.519 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.505+0000 7f5c7473a700 1 -- 172.21.15.72:0/1970070782 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c6c0af1e0 con 0x7f5c6c15ab10 2023-10-14T10:35:47.519 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.505+0000 7f5c62ffd700 1 -- 172.21.15.72:0/1970070782 <== mon.1 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6401b670 con 0x7f5c6c15ab10 2023-10-14T10:35:47.519 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.505+0000 7f5c7473a700 1 -- 172.21.15.72:0/1970070782 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c6c15c220 con 0x7f5c6c15ab10 2023-10-14T10:35:47.520 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.505+0000 7f5c62ffd700 1 -- 172.21.15.72:0/1970070782 <== mon.1 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f5c6401bad0 con 0x7f5c6c15ab10 2023-10-14T10:35:47.520 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.505+0000 7f5c62ffd700 1 -- 172.21.15.72:0/1970070782 <== mon.1 v2:172.21.15.72:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f5c64024c00 con 0x7f5c6c15ab10 2023-10-14T10:35:47.520 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.506+0000 7f5c7473a700 1 -- 172.21.15.72:0/1970070782 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c6c067d20 con 0x7f5c6c15ab10 2023-10-14T10:35:47.520 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.508+0000 7f5c62ffd700 1 -- 172.21.15.72:0/1970070782 <== mon.1 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f5c64042b40 con 0x7f5c6c15ab10 2023-10-14T10:35:47.593 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.591+0000 7f5c7473a700 1 -- 172.21.15.72:0/1970070782 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd new", "uuid": "1d889d76-a646-499d-bfff-ed5bae5434ff", "id": 3} v 0) v1 -- 0x7f5c6c0b0d90 con 0x7f5c6c15ab10 2023-10-14T10:35:47.599 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.597+0000 7f5c62ffd700 1 -- 172.21.15.72:0/1970070782 <== mon.1 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "1d889d76-a646-499d-bfff-ed5bae5434ff", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f5c64031030 con 0x7f5c6c15ab10 2023-10-14T10:35:47.599 INFO:teuthology.orchestra.run.smithi072.stdout:3 2023-10-14T10:35:47.600 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.598+0000 7f5c7473a700 1 -- 172.21.15.72:0/1970070782 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5c6c15ab10 msgr2=0x7f5c6c0cc830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:35:47.600 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.598+0000 7f5c7473a700 1 --2- 172.21.15.72:0/1970070782 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5c6c15ab10 0x7f5c6c0cc830 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5c64012000 tx=0x7f5c64004ad0 comp rx=0 tx=0).stop 2023-10-14T10:35:47.601 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.598+0000 7f5c7473a700 1 -- 172.21.15.72:0/1970070782 shutdown_connections 2023-10-14T10:35:47.601 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.598+0000 7f5c7473a700 1 --2- 172.21.15.72:0/1970070782 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5c6c15ab10 0x7f5c6c0cc830 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:47.601 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.598+0000 7f5c7473a700 1 --2- 172.21.15.72:0/1970070782 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f5c6c0b1f50 0x7f5c6c15ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:47.601 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.598+0000 7f5c7473a700 1 --2- 172.21.15.72:0/1970070782 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5c6c0cfe30 0x7f5c6c156ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:47.601 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.598+0000 7f5c7473a700 1 -- 172.21.15.72:0/1970070782 >> 172.21.15.72:0/1970070782 conn(0x7f5c6c06dae0 msgr2=0x7f5c6c0cc040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:35:47.602 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.598+0000 7f5c7473a700 1 -- 172.21.15.72:0/1970070782 shutdown_connections 2023-10-14T10:35:47.602 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.598+0000 7f5c7473a700 1 -- 172.21.15.72:0/1970070782 wait complete. 2023-10-14T10:35:47.611 DEBUG:teuthology.orchestra.run.smithi072:> sudo ceph --cluster ceph osd new c3271dff-b437-4fdd-b13b-6e49a35473af 4 2023-10-14T10:35:47.732 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.730+0000 7f8fb5ba1700 1 Processor -- start 2023-10-14T10:35:47.733 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.730+0000 7f8fb5ba1700 1 -- start start 2023-10-14T10:35:47.735 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.731+0000 7f8fb5ba1700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f8fb00cd950 0x7f8fb015e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:35:47.736 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.731+0000 7f8fb5ba1700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f8fb006a840 0x7f8fb006ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:35:47.736 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.731+0000 7f8fb5ba1700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fb00cf290 con 0x7f8fb00cede0 2023-10-14T10:35:47.736 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.731+0000 7f8fb5ba1700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fb01619a0 con 0x7f8fb00cd950 2023-10-14T10:35:47.736 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.731+0000 7f8fb5ba1700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fb006b1b0 con 0x7f8fb006a840 2023-10-14T10:35:47.736 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.731+0000 7f8faffff700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f8fb006a840 0x7f8fb006ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:35:47.737 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.731+0000 7f8faffff700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f8fb006a840 0x7f8fb006ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:51476/0 (socket says 172.21.15.72:51476) 2023-10-14T10:35:47.737 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.731+0000 7f8faffff700 1 -- 172.21.15.72:0/2320757715 learned_addr learned my addr 172.21.15.72:0/2320757715 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-14T10:35:47.737 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.731+0000 7f8faf7fe700 1 --2- 172.21.15.72:0/2320757715 >> v2:172.21.15.67:3300/0 conn(0x7f8fb00cd950 0x7f8fb015e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:35:47.737 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.732+0000 7f8faffff700 1 -- 172.21.15.72:0/2320757715 >> v2:172.21.15.67:3300/0 conn(0x7f8fb00cd950 msgr2=0x7f8fb015e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:35:47.737 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.732+0000 7f8fae7fc700 1 -- 172.21.15.72:0/2320757715 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4197151568 0 0) 0x7f8fb00cf290 con 0x7f8fb00cede0 2023-10-14T10:35:47.738 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.732+0000 7f8faffff700 1 --2- 172.21.15.72:0/2320757715 >> v2:172.21.15.67:3300/0 conn(0x7f8fb00cd950 0x7f8fb015e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:47.738 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.732+0000 7f8faffff700 1 -- 172.21.15.72:0/2320757715 >> v1:172.21.15.67:6789/0 conn(0x7f8fb00cede0 legacy=0x7f8fb015f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:35:47.738 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.732+0000 7f8faffff700 1 -- 172.21.15.72:0/2320757715 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fb00b2d60 con 0x7f8fb006a840 2023-10-14T10:35:47.738 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.732+0000 7f8faffff700 1 --2- 172.21.15.72:0/2320757715 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8fb006a840 0x7f8fb006ac70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8fb015e830 tx=0x7f8fa8009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9926c8bd1e27422 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:35:47.739 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.732+0000 7f8fae7fc700 1 -- 172.21.15.72:0/2320757715 <== mon.1 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa8011830 con 0x7f8fb006a840 2023-10-14T10:35:47.739 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.732+0000 7f8fae7fc700 1 -- 172.21.15.72:0/2320757715 <== mon.1 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fa8011990 con 0x7f8fb006a840 2023-10-14T10:35:47.739 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.732+0000 7f8fae7fc700 1 -- 172.21.15.72:0/2320757715 <== mon.1 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa8011d20 con 0x7f8fb006a840 2023-10-14T10:35:47.739 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.733+0000 7f8fb5ba1700 1 -- 172.21.15.72:0/2320757715 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8fb006a840 msgr2=0x7f8fb006ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:35:47.739 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.733+0000 7f8fb5ba1700 1 --2- 172.21.15.72:0/2320757715 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8fb006a840 0x7f8fb006ac70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8fb015e830 tx=0x7f8fa8009bd0 comp rx=0 tx=0).stop 2023-10-14T10:35:47.740 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.733+0000 7f8fb5ba1700 1 -- 172.21.15.72:0/2320757715 shutdown_connections 2023-10-14T10:35:47.740 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.733+0000 7f8fb5ba1700 1 --2- 172.21.15.72:0/2320757715 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8fb006a840 0x7f8fb006ac70 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:47.740 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.733+0000 7f8fb5ba1700 1 --2- 172.21.15.72:0/2320757715 >> v2:172.21.15.67:3300/0 conn(0x7f8fb00cd950 0x7f8fb015e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:47.740 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.733+0000 7f8fb5ba1700 1 -- 172.21.15.72:0/2320757715 >> 172.21.15.72:0/2320757715 conn(0x7f8fb006dae0 msgr2=0x7f8fb006def0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:35:47.740 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.733+0000 7f8fb5ba1700 1 -- 172.21.15.72:0/2320757715 shutdown_connections 2023-10-14T10:35:47.741 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.733+0000 7f8fb5ba1700 1 -- 172.21.15.72:0/2320757715 wait complete. 2023-10-14T10:35:47.743 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.734+0000 7f8fb5ba1700 1 Processor -- start 2023-10-14T10:35:47.743 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.734+0000 7f8fb5ba1700 1 -- start start 2023-10-14T10:35:47.744 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.735+0000 7f8fb5ba1700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fb006a840 0x7f8fb015b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:35:47.744 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.735+0000 7f8fb5ba1700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f8fb00cd950 0x7f8fb00b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:35:47.744 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.735+0000 7f8fb5ba1700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8fb00cede0 0x7f8fb01f38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:35:47.744 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.735+0000 7f8fb5ba1700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb00b42a0 con 0x7f8fb006a840 2023-10-14T10:35:47.744 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.735+0000 7f8fb5ba1700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb01f4410 con 0x7f8fb00cede0 2023-10-14T10:35:47.745 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.735+0000 7f8fb5ba1700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb01f46f0 con 0x7f8fb00cd950 2023-10-14T10:35:47.745 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.735+0000 7f8faffff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8fb00cede0 0x7f8fb01f38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:35:47.745 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.735+0000 7f8faeffd700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f8fb00cd950 0x7f8fb00b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:35:47.745 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.735+0000 7f8faffff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8fb00cede0 0x7f8fb01f38e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:51482/0 (socket says 172.21.15.72:51482) 2023-10-14T10:35:47.745 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.735+0000 7f8faffff700 1 -- 172.21.15.72:0/1142503818 learned_addr learned my addr 172.21.15.72:0/1142503818 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-14T10:35:47.746 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.735+0000 7f8faf7fe700 1 --2- 172.21.15.72:0/1142503818 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fb006a840 0x7f8fb015b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:35:47.746 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.735+0000 7f8faffff700 1 -- 172.21.15.72:0/1142503818 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f8fb00cd950 msgr2=0x7f8fb00b4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:35:47.746 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.735+0000 7f8faffff700 1 --2- 172.21.15.72:0/1142503818 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f8fb00cd950 0x7f8fb00b4cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:47.746 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.735+0000 7f8faffff700 1 -- 172.21.15.72:0/1142503818 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fb006a840 msgr2=0x7f8fb015b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:35:47.746 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.735+0000 7f8faffff700 1 --2- 172.21.15.72:0/1142503818 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fb006a840 0x7f8fb015b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:47.747 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.735+0000 7f8faffff700 1 -- 172.21.15.72:0/1142503818 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fa8009550 con 0x7f8fb00cede0 2023-10-14T10:35:47.747 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.735+0000 7f8faeffd700 1 --2- 172.21.15.72:0/1142503818 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f8fb00cd950 0x7f8fb00b4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-14T10:35:47.747 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.735+0000 7f8faffff700 1 --2- 172.21.15.72:0/1142503818 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8fb00cede0 0x7f8fb01f38e0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8fb006d4f0 tx=0x7f8fa8004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:35:47.747 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.736+0000 7f8f97fff700 1 -- 172.21.15.72:0/1142503818 <== mon.1 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa8004300 con 0x7f8fb00cede0 2023-10-14T10:35:47.747 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.736+0000 7f8f97fff700 1 -- 172.21.15.72:0/1142503818 <== mon.1 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fa801f070 con 0x7f8fb00cede0 2023-10-14T10:35:47.748 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.736+0000 7f8fb5ba1700 1 -- 172.21.15.72:0/1142503818 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fb00b44b0 con 0x7f8fb00cede0 2023-10-14T10:35:47.748 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.736+0000 7f8f97fff700 1 -- 172.21.15.72:0/1142503818 <== mon.1 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa80036a0 con 0x7f8fb00cede0 2023-10-14T10:35:47.748 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.736+0000 7f8fb5ba1700 1 -- 172.21.15.72:0/1142503818 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fb01f7100 con 0x7f8fb00cede0 2023-10-14T10:35:47.748 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.736+0000 7f8f97fff700 1 -- 172.21.15.72:0/1142503818 <== mon.1 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f8fa8021030 con 0x7f8fb00cede0 2023-10-14T10:35:47.748 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.736+0000 7f8f97fff700 1 -- 172.21.15.72:0/1142503818 <== mon.1 v2:172.21.15.72:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f8fa801adc0 con 0x7f8fb00cede0 2023-10-14T10:35:47.749 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.737+0000 7f8fb5ba1700 1 -- 172.21.15.72:0/1142503818 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fb0067d20 con 0x7f8fb00cede0 2023-10-14T10:35:47.749 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.739+0000 7f8f97fff700 1 -- 172.21.15.72:0/1142503818 <== mon.1 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f8fa8015020 con 0x7f8fb00cede0 2023-10-14T10:35:47.766 INFO:tasks.ceph.mgr.x.smithi072.stderr:2023-10-14T10:35:47.761+0000 7fa493d36200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-14T10:35:47.769 INFO:tasks.ceph.mgr.y.smithi067.stderr:2023-10-14T10:35:47.768+0000 7f928a49e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-14T10:35:47.825 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.822+0000 7f8fb5ba1700 1 -- 172.21.15.72:0/1142503818 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd new", "uuid": "c3271dff-b437-4fdd-b13b-6e49a35473af", "id": 4} v 0) v1 -- 0x7f8fb00b2630 con 0x7f8fb00cede0 2023-10-14T10:35:47.830 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.828+0000 7f8f97fff700 1 -- 172.21.15.72:0/1142503818 <== mon.1 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c3271dff-b437-4fdd-b13b-6e49a35473af", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f8fa8003b00 con 0x7f8fb00cede0 2023-10-14T10:35:47.831 INFO:teuthology.orchestra.run.smithi072.stdout:4 2023-10-14T10:35:47.832 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.829+0000 7f8fb5ba1700 1 -- 172.21.15.72:0/1142503818 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8fb00cede0 msgr2=0x7f8fb01f38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:35:47.832 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.829+0000 7f8fb5ba1700 1 --2- 172.21.15.72:0/1142503818 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8fb00cede0 0x7f8fb01f38e0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8fb006d4f0 tx=0x7f8fa8004750 comp rx=0 tx=0).stop 2023-10-14T10:35:47.832 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.829+0000 7f8fb5ba1700 1 -- 172.21.15.72:0/1142503818 shutdown_connections 2023-10-14T10:35:47.833 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.829+0000 7f8fb5ba1700 1 --2- 172.21.15.72:0/1142503818 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8fb00cede0 0x7f8fb01f38e0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:47.833 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.829+0000 7f8fb5ba1700 1 --2- 172.21.15.72:0/1142503818 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fb006a840 0x7f8fb015b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:47.833 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.829+0000 7f8fb5ba1700 1 --2- 172.21.15.72:0/1142503818 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f8fb00cd950 0x7f8fb00b4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:47.833 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.829+0000 7f8fb5ba1700 1 -- 172.21.15.72:0/1142503818 >> 172.21.15.72:0/1142503818 conn(0x7f8fb006dae0 msgr2=0x7f8fb006dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:35:47.833 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.829+0000 7f8fb5ba1700 1 -- 172.21.15.72:0/1142503818 shutdown_connections 2023-10-14T10:35:47.834 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.829+0000 7f8fb5ba1700 1 -- 172.21.15.72:0/1142503818 wait complete. 2023-10-14T10:35:47.843 DEBUG:teuthology.orchestra.run.smithi072:> sudo ceph --cluster ceph osd new 0337bb18-4b68-4eff-b528-a667275e6926 5 2023-10-14T10:35:47.964 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.962+0000 7fc457e53700 1 Processor -- start 2023-10-14T10:35:47.964 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.962+0000 7fc457e53700 1 -- start start 2023-10-14T10:35:47.966 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.962+0000 7fc457e53700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fc4500ce6e0 0x7fc4500cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:35:47.967 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.963+0000 7fc457e53700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4500d23e0 con 0x7fc4500d0c70 2023-10-14T10:35:47.967 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.963+0000 7fc457e53700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4500cf190 con 0x7fc45015b740 2023-10-14T10:35:47.967 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.963+0000 7fc457e53700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc45015bb20 con 0x7fc4500ce6e0 2023-10-14T10:35:47.967 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.963+0000 7fc455bef700 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7fc45015b740 0x7fc4500d1cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:46928/0 (socket says 172.21.15.72:46928) 2023-10-14T10:35:47.968 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.963+0000 7fc455bef700 1 -- 172.21.15.72:0/3040079911 learned_addr learned my addr 172.21.15.72:0/3040079911 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-10-14T10:35:47.968 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.963+0000 7fc4563f0700 1 --2- 172.21.15.72:0/3040079911 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fc4500ce6e0 0x7fc4500cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:35:47.968 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.963+0000 7fc454bed700 1 -- 172.21.15.72:0/3040079911 <== mon.1 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2948468240 0 0) 0x7fc4500cf190 con 0x7fc45015b740 2023-10-14T10:35:47.968 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.963+0000 7fc454bed700 1 -- 172.21.15.72:0/3040079911 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc43c0035b0 con 0x7fc45015b740 2023-10-14T10:35:47.969 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.963+0000 7fc4563f0700 1 -- 172.21.15.72:0/3040079911 >> v1:172.21.15.72:6789/0 conn(0x7fc45015b740 legacy=0x7fc4500d1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:35:47.969 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.963+0000 7fc4563f0700 1 -- 172.21.15.72:0/3040079911 >> v1:172.21.15.67:6789/0 conn(0x7fc4500d0c70 legacy=0x7fc4500cdfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:35:47.969 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.963+0000 7fc454bed700 1 -- 172.21.15.72:0/3040079911 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1054840696 0 0) 0x7fc440001110 con 0x7fc4500d0c70 2023-10-14T10:35:47.970 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.963+0000 7fc4563f0700 1 -- 172.21.15.72:0/3040079911 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4500cbd40 con 0x7fc4500ce6e0 2023-10-14T10:35:47.970 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.963+0000 7fc4563f0700 1 --2- 172.21.15.72:0/3040079911 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fc4500ce6e0 0x7fc4500cf690 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc448009dc0 tx=0x7fc44800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd9f263c4094d2af server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:35:47.970 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.963+0000 7fc454bed700 1 -- 172.21.15.72:0/3040079911 <== mon.2 v2:172.21.15.72:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc44800e070 con 0x7fc4500ce6e0 2023-10-14T10:35:47.970 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.963+0000 7fc454bed700 1 -- 172.21.15.72:0/3040079911 <== mon.2 v2:172.21.15.72:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc44800bb20 con 0x7fc4500ce6e0 2023-10-14T10:35:47.970 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.963+0000 7fc454bed700 1 -- 172.21.15.72:0/3040079911 <== mon.2 v2:172.21.15.72:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc44800be90 con 0x7fc4500ce6e0 2023-10-14T10:35:47.971 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.964+0000 7fc457e53700 1 -- 172.21.15.72:0/3040079911 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fc4500ce6e0 msgr2=0x7fc4500cf690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:35:47.971 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.964+0000 7fc457e53700 1 --2- 172.21.15.72:0/3040079911 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fc4500ce6e0 0x7fc4500cf690 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc448009dc0 tx=0x7fc44800b330 comp rx=0 tx=0).stop 2023-10-14T10:35:47.971 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.964+0000 7fc457e53700 1 -- 172.21.15.72:0/3040079911 shutdown_connections 2023-10-14T10:35:47.971 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.964+0000 7fc457e53700 1 --2- 172.21.15.72:0/3040079911 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fc4500ce6e0 0x7fc4500cf690 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:47.973 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.964+0000 7fc457e53700 1 -- 172.21.15.72:0/3040079911 >> 172.21.15.72:0/3040079911 conn(0x7fc4500665a0 msgr2=0x7fc450066980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:35:47.973 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.964+0000 7fc457e53700 1 -- 172.21.15.72:0/3040079911 shutdown_connections 2023-10-14T10:35:47.974 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.964+0000 7fc457e53700 1 -- 172.21.15.72:0/3040079911 wait complete. 2023-10-14T10:35:47.977 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.965+0000 7fc457e53700 1 Processor -- start 2023-10-14T10:35:47.977 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.965+0000 7fc457e53700 1 -- start start 2023-10-14T10:35:47.977 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.966+0000 7fc457e53700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fc4500ce6e0 0x7fc4501f2880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:35:47.978 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.966+0000 7fc457e53700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc4500d0c70 0x7fc450157330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:35:47.978 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.966+0000 7fc457e53700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc45015b740 0x7fc450157870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:35:47.978 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.966+0000 7fc457e53700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc45006c780 con 0x7fc4500d0c70 2023-10-14T10:35:47.979 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.966+0000 7fc457e53700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc45006cab0 con 0x7fc45015b740 2023-10-14T10:35:47.979 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.966+0000 7fc457e53700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc45006cd90 con 0x7fc4500ce6e0 2023-10-14T10:35:47.979 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.966+0000 7fc455bef700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fc4500ce6e0 0x7fc4501f2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:35:47.979 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.966+0000 7fc4563f0700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc45015b740 0x7fc450157870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:35:47.979 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.966+0000 7fc455bef700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fc4500ce6e0 0x7fc4501f2880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3301/0 says I am v2:172.21.15.72:54948/0 (socket says 172.21.15.72:54948) 2023-10-14T10:35:47.980 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.966+0000 7fc455bef700 1 -- 172.21.15.72:0/2383339093 learned_addr learned my addr 172.21.15.72:0/2383339093 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-14T10:35:47.980 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.966+0000 7fc4553ee700 1 --2- 172.21.15.72:0/2383339093 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc4500d0c70 0x7fc450157330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:35:47.980 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.966+0000 7fc455bef700 1 -- 172.21.15.72:0/2383339093 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc45015b740 msgr2=0x7fc450157870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:35:47.981 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.966+0000 7fc455bef700 1 --2- 172.21.15.72:0/2383339093 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc45015b740 0x7fc450157870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:47.981 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.966+0000 7fc455bef700 1 -- 172.21.15.72:0/2383339093 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc4500d0c70 msgr2=0x7fc450157330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:35:47.981 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.966+0000 7fc455bef700 1 --2- 172.21.15.72:0/2383339093 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc4500d0c70 0x7fc450157330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:47.981 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.966+0000 7fc455bef700 1 -- 172.21.15.72:0/2383339093 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc448009a70 con 0x7fc4500ce6e0 2023-10-14T10:35:47.982 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.966+0000 7fc4563f0700 1 --2- 172.21.15.72:0/2383339093 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc45015b740 0x7fc450157870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-14T10:35:47.982 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.967+0000 7fc455bef700 1 --2- 172.21.15.72:0/2383339093 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fc4500ce6e0 0x7fc4501f2880 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc44c00c430 tx=0x7fc44c00c740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:35:47.982 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.967+0000 7fc4467fc700 1 -- 172.21.15.72:0/2383339093 <== mon.2 v2:172.21.15.72:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc44c005d70 con 0x7fc4500ce6e0 2023-10-14T10:35:47.982 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.967+0000 7fc4467fc700 1 -- 172.21.15.72:0/2383339093 <== mon.2 v2:172.21.15.72:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc44c00ce40 con 0x7fc4500ce6e0 2023-10-14T10:35:47.982 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.967+0000 7fc457e53700 1 -- 172.21.15.72:0/2383339093 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc45006d070 con 0x7fc4500ce6e0 2023-10-14T10:35:47.983 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.967+0000 7fc4467fc700 1 -- 172.21.15.72:0/2383339093 <== mon.2 v2:172.21.15.72:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc44c0072f0 con 0x7fc4500ce6e0 2023-10-14T10:35:47.983 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.967+0000 7fc457e53700 1 -- 172.21.15.72:0/2383339093 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc45006d870 con 0x7fc4500ce6e0 2023-10-14T10:35:47.983 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.967+0000 7fc4467fc700 1 -- 172.21.15.72:0/2383339093 <== mon.2 v2:172.21.15.72:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fc44c01c440 con 0x7fc4500ce6e0 2023-10-14T10:35:47.983 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.967+0000 7fc4467fc700 1 -- 172.21.15.72:0/2383339093 <== mon.2 v2:172.21.15.72:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7fc44c0077a0 con 0x7fc4500ce6e0 2023-10-14T10:35:47.983 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.968+0000 7fc457e53700 1 -- 172.21.15.72:0/2383339093 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc45006d590 con 0x7fc4500ce6e0 2023-10-14T10:35:47.984 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:47.970+0000 7fc4467fc700 1 -- 172.21.15.72:0/2383339093 <== mon.2 v2:172.21.15.72:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7fc45006d590 con 0x7fc4500ce6e0 2023-10-14T10:35:48.056 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.054+0000 7fc457e53700 1 -- 172.21.15.72:0/2383339093 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_command({"prefix": "osd new", "uuid": "0337bb18-4b68-4eff-b528-a667275e6926", "id": 5} v 0) v1 -- 0x7fc450067d20 con 0x7fc4500ce6e0 2023-10-14T10:35:48.061 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.059+0000 7fc4467fc700 1 -- 172.21.15.72:0/2383339093 <== mon.2 v2:172.21.15.72:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "0337bb18-4b68-4eff-b528-a667275e6926", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7fc44c017070 con 0x7fc4500ce6e0 2023-10-14T10:35:48.062 INFO:teuthology.orchestra.run.smithi072.stdout:5 2023-10-14T10:35:48.063 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.060+0000 7fc457e53700 1 -- 172.21.15.72:0/2383339093 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fc4500ce6e0 msgr2=0x7fc4501f2880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:35:48.063 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.060+0000 7fc457e53700 1 --2- 172.21.15.72:0/2383339093 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fc4500ce6e0 0x7fc4501f2880 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc44c00c430 tx=0x7fc44c00c740 comp rx=0 tx=0).stop 2023-10-14T10:35:48.063 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.060+0000 7fc457e53700 1 -- 172.21.15.72:0/2383339093 shutdown_connections 2023-10-14T10:35:48.064 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.060+0000 7fc457e53700 1 --2- 172.21.15.72:0/2383339093 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc45015b740 0x7fc450157870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:48.064 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.060+0000 7fc457e53700 1 --2- 172.21.15.72:0/2383339093 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fc4500ce6e0 0x7fc4501f2880 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:48.064 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.060+0000 7fc457e53700 1 --2- 172.21.15.72:0/2383339093 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc4500d0c70 0x7fc450157330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:48.064 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.060+0000 7fc457e53700 1 -- 172.21.15.72:0/2383339093 >> 172.21.15.72:0/2383339093 conn(0x7fc4500665a0 msgr2=0x7fc450159f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:35:48.065 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.061+0000 7fc457e53700 1 -- 172.21.15.72:0/2383339093 shutdown_connections 2023-10-14T10:35:48.065 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.061+0000 7fc457e53700 1 -- 172.21.15.72:0/2383339093 wait complete. 2023-10-14T10:35:48.075 DEBUG:teuthology.orchestra.run.smithi072:> sudo ceph --cluster ceph osd new 8e92d9cf-0c8c-46eb-89a7-db2023ef1509 6 2023-10-14T10:35:48.219 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.217+0000 7f75c74ab700 1 Processor -- start 2023-10-14T10:35:48.222 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.217+0000 7f75c74ab700 1 -- start start 2023-10-14T10:35:48.223 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.218+0000 7f75c74ab700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f75c0159eb0 0x7f75c00c9a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:35:48.223 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.218+0000 7f75c74ab700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75c015a290 con 0x7f75c00ad270 2023-10-14T10:35:48.223 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.218+0000 7f75c74ab700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75c00cb7e0 con 0x7f75c00cd210 2023-10-14T10:35:48.224 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.218+0000 7f75c74ab700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75c00cdfd0 con 0x7f75c0159eb0 2023-10-14T10:35:48.224 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.219+0000 7f75c4a46700 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f75c00cd210 0x7f75c00cba30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:46940/0 (socket says 172.21.15.72:46940) 2023-10-14T10:35:48.224 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.219+0000 7f75c4a46700 1 -- 172.21.15.72:0/1617988670 learned_addr learned my addr 172.21.15.72:0/1617988670 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-10-14T10:35:48.224 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.220+0000 7f75c5247700 1 --2- 172.21.15.72:0/1617988670 >> v2:172.21.15.72:3300/0 conn(0x7f75c0159eb0 0x7f75c00c9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:35:48.224 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.220+0000 7f75bffff700 1 -- 172.21.15.72:0/1617988670 <== mon.1 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2141429332 0 0) 0x7f75c00cb7e0 con 0x7f75c00cd210 2023-10-14T10:35:48.225 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.220+0000 7f75bffff700 1 -- 172.21.15.72:0/1617988670 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75a80035b0 con 0x7f75c00cd210 2023-10-14T10:35:48.225 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.220+0000 7f75bffff700 1 -- 172.21.15.72:0/1617988670 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 485468918 0 0) 0x7f75c015a290 con 0x7f75c00ad270 2023-10-14T10:35:48.225 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.220+0000 7f75bffff700 1 -- 172.21.15.72:0/1617988670 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75c00cb7e0 con 0x7f75c00ad270 2023-10-14T10:35:48.225 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.220+0000 7f75c5247700 1 -- 172.21.15.72:0/1617988670 >> v1:172.21.15.72:6789/0 conn(0x7f75c00cd210 legacy=0x7f75c00cba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:35:48.225 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.220+0000 7f75c5247700 1 -- 172.21.15.72:0/1617988670 >> v1:172.21.15.67:6789/0 conn(0x7f75c00ad270 legacy=0x7f75c00cc140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:35:48.226 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.220+0000 7f75c5247700 1 -- 172.21.15.72:0/1617988670 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75c00cabd0 con 0x7f75c0159eb0 2023-10-14T10:35:48.226 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.220+0000 7f75c5247700 1 --2- 172.21.15.72:0/1617988670 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f75c0159eb0 0x7f75c00c9a80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f75b4009820 tx=0x7f75b4009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=202cf2a04d26a68d server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:35:48.227 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.220+0000 7f75bffff700 1 -- 172.21.15.72:0/1617988670 <== mon.1 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75b4005e30 con 0x7f75c0159eb0 2023-10-14T10:35:48.227 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.220+0000 7f75bffff700 1 -- 172.21.15.72:0/1617988670 <== mon.1 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75b4006030 con 0x7f75c0159eb0 2023-10-14T10:35:48.227 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.221+0000 7f75bffff700 1 -- 172.21.15.72:0/1617988670 <== mon.1 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75b40063c0 con 0x7f75c0159eb0 2023-10-14T10:35:48.227 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.221+0000 7f75c74ab700 1 -- 172.21.15.72:0/1617988670 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f75c0159eb0 msgr2=0x7f75c00c9a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:35:48.228 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.221+0000 7f75c74ab700 1 --2- 172.21.15.72:0/1617988670 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f75c0159eb0 0x7f75c00c9a80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f75b4009820 tx=0x7f75b4009b30 comp rx=0 tx=0).stop 2023-10-14T10:35:48.228 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.221+0000 7f75c74ab700 1 -- 172.21.15.72:0/1617988670 shutdown_connections 2023-10-14T10:35:48.228 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.221+0000 7f75c74ab700 1 --2- 172.21.15.72:0/1617988670 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f75c0159eb0 0x7f75c00c9a80 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:48.228 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.221+0000 7f75c74ab700 1 -- 172.21.15.72:0/1617988670 >> 172.21.15.72:0/1617988670 conn(0x7f75c00cf430 msgr2=0x7f75c00cf840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:35:48.229 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.221+0000 7f75c74ab700 1 -- 172.21.15.72:0/1617988670 shutdown_connections 2023-10-14T10:35:48.229 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.222+0000 7f75c74ab700 1 -- 172.21.15.72:0/1617988670 wait complete. 2023-10-14T10:35:48.229 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.223+0000 7f75c74ab700 1 Processor -- start 2023-10-14T10:35:48.230 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.224+0000 7f75c74ab700 1 -- start start 2023-10-14T10:35:48.230 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.225+0000 7f75c74ab700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f75c00ad270 0x7f75c01f5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:35:48.231 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.225+0000 7f75c74ab700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75c00cd210 0x7f75c00c2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:35:48.231 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.225+0000 7f75c74ab700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f75c0159eb0 0x7f75c00c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:35:48.231 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.225+0000 7f75c74ab700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75c01f5d40 con 0x7f75c00cd210 2023-10-14T10:35:48.231 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.225+0000 7f75c74ab700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75c00c3300 con 0x7f75c00ad270 2023-10-14T10:35:48.231 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.225+0000 7f75c74ab700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75c00c35e0 con 0x7f75c0159eb0 2023-10-14T10:35:48.232 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.225+0000 7f75c5a48700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f75c0159eb0 0x7f75c00c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:35:48.232 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.225+0000 7f75c5247700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f75c00ad270 0x7f75c01f5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:35:48.232 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.225+0000 7f75c5a48700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f75c0159eb0 0x7f75c00c4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3301/0 says I am v2:172.21.15.72:54964/0 (socket says 172.21.15.72:54964) 2023-10-14T10:35:48.232 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.225+0000 7f75c5a48700 1 -- 172.21.15.72:0/3703955712 learned_addr learned my addr 172.21.15.72:0/3703955712 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-14T10:35:48.232 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.225+0000 7f75c4a46700 1 --2- 172.21.15.72:0/3703955712 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75c00cd210 0x7f75c00c2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:35:48.233 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.225+0000 7f75c5a48700 1 -- 172.21.15.72:0/3703955712 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f75c00ad270 msgr2=0x7f75c01f5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:35:48.233 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.225+0000 7f75c5a48700 1 --2- 172.21.15.72:0/3703955712 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f75c00ad270 0x7f75c01f5800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:48.233 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.225+0000 7f75c5a48700 1 -- 172.21.15.72:0/3703955712 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75c00cd210 msgr2=0x7f75c00c2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:35:48.234 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.225+0000 7f75c5a48700 1 --2- 172.21.15.72:0/3703955712 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75c00cd210 0x7f75c00c2ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:48.234 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.225+0000 7f75c5a48700 1 -- 172.21.15.72:0/3703955712 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75b4009430 con 0x7f75c0159eb0 2023-10-14T10:35:48.234 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.225+0000 7f75c4a46700 1 --2- 172.21.15.72:0/3703955712 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75c00cd210 0x7f75c00c2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-14T10:35:48.234 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.226+0000 7f75c5a48700 1 --2- 172.21.15.72:0/3703955712 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f75c0159eb0 0x7f75c00c4e30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f75b800c400 tx=0x7f75b800c710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:35:48.234 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.226+0000 7f75bdffb700 1 -- 172.21.15.72:0/3703955712 <== mon.2 v2:172.21.15.72:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75b80050d0 con 0x7f75c0159eb0 2023-10-14T10:35:48.235 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.226+0000 7f75bdffb700 1 -- 172.21.15.72:0/3703955712 <== mon.2 v2:172.21.15.72:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75b8005230 con 0x7f75c0159eb0 2023-10-14T10:35:48.235 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.226+0000 7f75c74ab700 1 -- 172.21.15.72:0/3703955712 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75c00c38c0 con 0x7f75c0159eb0 2023-10-14T10:35:48.235 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.226+0000 7f75bdffb700 1 -- 172.21.15.72:0/3703955712 <== mon.2 v2:172.21.15.72:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75b8003680 con 0x7f75c0159eb0 2023-10-14T10:35:48.235 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.226+0000 7f75c74ab700 1 -- 172.21.15.72:0/3703955712 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75c00c3e80 con 0x7f75c0159eb0 2023-10-14T10:35:48.235 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.227+0000 7f75bdffb700 1 -- 172.21.15.72:0/3703955712 <== mon.2 v2:172.21.15.72:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f75b8003ae0 con 0x7f75c0159eb0 2023-10-14T10:35:48.236 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.227+0000 7f75bdffb700 1 -- 172.21.15.72:0/3703955712 <== mon.2 v2:172.21.15.72:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f75b8033b40 con 0x7f75c0159eb0 2023-10-14T10:35:48.236 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.227+0000 7f75c74ab700 1 -- 172.21.15.72:0/3703955712 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75c00c40a0 con 0x7f75c0159eb0 2023-10-14T10:35:48.237 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.230+0000 7f75bdffb700 1 -- 172.21.15.72:0/3703955712 <== mon.2 v2:172.21.15.72:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f75b8039070 con 0x7f75c0159eb0 2023-10-14T10:35:48.314 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.312+0000 7f75c74ab700 1 -- 172.21.15.72:0/3703955712 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_command({"prefix": "osd new", "uuid": "8e92d9cf-0c8c-46eb-89a7-db2023ef1509", "id": 6} v 0) v1 -- 0x7f75c004ebb0 con 0x7f75c0159eb0 2023-10-14T10:35:48.322 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.320+0000 7f75bdffb700 1 -- 172.21.15.72:0/3703955712 <== mon.2 v2:172.21.15.72:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "8e92d9cf-0c8c-46eb-89a7-db2023ef1509", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f75b8033de0 con 0x7f75c0159eb0 2023-10-14T10:35:48.322 INFO:teuthology.orchestra.run.smithi072.stdout:6 2023-10-14T10:35:48.324 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.321+0000 7f75c74ab700 1 -- 172.21.15.72:0/3703955712 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f75c0159eb0 msgr2=0x7f75c00c4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:35:48.324 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.321+0000 7f75c74ab700 1 --2- 172.21.15.72:0/3703955712 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f75c0159eb0 0x7f75c00c4e30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f75b800c400 tx=0x7f75b800c710 comp rx=0 tx=0).stop 2023-10-14T10:35:48.325 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.321+0000 7f75c74ab700 1 -- 172.21.15.72:0/3703955712 shutdown_connections 2023-10-14T10:35:48.325 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.321+0000 7f75c74ab700 1 --2- 172.21.15.72:0/3703955712 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f75c0159eb0 0x7f75c00c4e30 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:48.325 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.321+0000 7f75c74ab700 1 --2- 172.21.15.72:0/3703955712 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f75c00ad270 0x7f75c01f5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:48.325 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.321+0000 7f75c74ab700 1 --2- 172.21.15.72:0/3703955712 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75c00cd210 0x7f75c00c2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:48.326 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.321+0000 7f75c74ab700 1 -- 172.21.15.72:0/3703955712 >> 172.21.15.72:0/3703955712 conn(0x7f75c00cf430 msgr2=0x7f75c006d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:35:48.326 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.321+0000 7f75c74ab700 1 -- 172.21.15.72:0/3703955712 shutdown_connections 2023-10-14T10:35:48.326 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.321+0000 7f75c74ab700 1 -- 172.21.15.72:0/3703955712 wait complete. 2023-10-14T10:35:48.335 DEBUG:teuthology.orchestra.run.smithi072:> sudo ceph --cluster ceph osd new 3fd6aa7d-50a9-45c9-9561-5902a044a119 7 2023-10-14T10:35:48.435 INFO:tasks.ceph.mgr.x.smithi072.stderr:2023-10-14T10:35:48.433+0000 7fa493d36200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-14T10:35:48.443 INFO:tasks.ceph.mgr.y.smithi067.stderr:2023-10-14T10:35:48.441+0000 7f928a49e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-14T10:35:48.459 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.457+0000 7f83340bd700 1 Processor -- start 2023-10-14T10:35:48.459 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.457+0000 7f83340bd700 1 -- start start 2023-10-14T10:35:48.461 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.458+0000 7f83340bd700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f832c06af50 0x7f832c0cede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:35:48.461 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.458+0000 7f83340bd700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f832c0b2d60 con 0x7f832c15e2f0 2023-10-14T10:35:48.461 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.458+0000 7f83340bd700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f832c0cdd30 con 0x7f832c0cd950 2023-10-14T10:35:48.462 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.458+0000 7f83340bd700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f832c1619a0 con 0x7f832c06af50 2023-10-14T10:35:48.462 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.458+0000 7f8331e59700 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f832c0cd950 0x7f832c15f570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:46952/0 (socket says 172.21.15.72:46952) 2023-10-14T10:35:48.462 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.458+0000 7f8331e59700 1 -- 172.21.15.72:0/856567664 learned_addr learned my addr 172.21.15.72:0/856567664 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-10-14T10:35:48.462 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.458+0000 7f833265a700 1 --2- 172.21.15.72:0/856567664 >> v2:172.21.15.67:3300/0 conn(0x7f832c06af50 0x7f832c0cede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:35:48.462 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.458+0000 7f8330e57700 1 -- 172.21.15.72:0/856567664 <== mon.1 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3865038257 0 0) 0x7f832c0cdd30 con 0x7f832c0cd950 2023-10-14T10:35:48.463 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.458+0000 7f8330e57700 1 -- 172.21.15.72:0/856567664 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83180035b0 con 0x7f832c0cd950 2023-10-14T10:35:48.463 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.458+0000 7f833265a700 1 -- 172.21.15.72:0/856567664 >> v1:172.21.15.72:6789/0 conn(0x7f832c0cd950 legacy=0x7f832c15f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:35:48.463 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.458+0000 7f833265a700 1 -- 172.21.15.72:0/856567664 >> v1:172.21.15.67:6789/0 conn(0x7f832c15e2f0 legacy=0x7f832c06a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:35:48.463 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.458+0000 7f8330e57700 1 -- 172.21.15.72:0/856567664 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2299158338 0 0) 0x7f832c0b2d60 con 0x7f832c15e2f0 2023-10-14T10:35:48.463 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.459+0000 7f833265a700 1 -- 172.21.15.72:0/856567664 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f832c15e920 con 0x7f832c06af50 2023-10-14T10:35:48.464 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.459+0000 7f833265a700 1 --2- 172.21.15.72:0/856567664 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f832c06af50 0x7f832c0cede0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8324009d90 tx=0x7f832400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c34d2856f00f2bd server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:35:48.464 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.459+0000 7f8330e57700 1 -- 172.21.15.72:0/856567664 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f832400e070 con 0x7f832c06af50 2023-10-14T10:35:48.464 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.459+0000 7f8330e57700 1 -- 172.21.15.72:0/856567664 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f832400bb20 con 0x7f832c06af50 2023-10-14T10:35:48.464 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.459+0000 7f8330e57700 1 -- 172.21.15.72:0/856567664 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f832400bed0 con 0x7f832c06af50 2023-10-14T10:35:48.467 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.459+0000 7f83340bd700 1 -- 172.21.15.72:0/856567664 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f832c06af50 msgr2=0x7f832c0cede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:35:48.468 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.459+0000 7f83340bd700 1 --2- 172.21.15.72:0/856567664 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f832c06af50 0x7f832c0cede0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8324009d90 tx=0x7f832400b330 comp rx=0 tx=0).stop 2023-10-14T10:35:48.468 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.459+0000 7f83340bd700 1 -- 172.21.15.72:0/856567664 shutdown_connections 2023-10-14T10:35:48.468 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.459+0000 7f83340bd700 1 --2- 172.21.15.72:0/856567664 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f832c06af50 0x7f832c0cede0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:48.468 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.459+0000 7f83340bd700 1 -- 172.21.15.72:0/856567664 >> 172.21.15.72:0/856567664 conn(0x7f832c06dae0 msgr2=0x7f832c06def0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:35:48.469 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.460+0000 7f83340bd700 1 -- 172.21.15.72:0/856567664 shutdown_connections 2023-10-14T10:35:48.469 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.460+0000 7f83340bd700 1 -- 172.21.15.72:0/856567664 wait complete. 2023-10-14T10:35:48.469 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.461+0000 7f83340bd700 1 Processor -- start 2023-10-14T10:35:48.469 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.461+0000 7f83340bd700 1 -- start start 2023-10-14T10:35:48.470 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.461+0000 7f83340bd700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f832c06af50 0x7f832c15a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:35:48.470 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.461+0000 7f83340bd700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f832c0cd950 0x7f832c0b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:35:48.470 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.461+0000 7f83340bd700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f832c15e2f0 0x7f832c156d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:35:48.470 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.461+0000 7f83340bd700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f832c0b57f0 con 0x7f832c06af50 2023-10-14T10:35:48.471 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.461+0000 7f83340bd700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f832c0b5a20 con 0x7f832c15e2f0 2023-10-14T10:35:48.471 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.461+0000 7f83340bd700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f832c0b3bc0 con 0x7f832c0cd950 2023-10-14T10:35:48.471 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.462+0000 7f8331658700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f832c0cd950 0x7f832c0b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:35:48.471 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.462+0000 7f8331e59700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f832c06af50 0x7f832c15a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:35:48.472 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.462+0000 7f833265a700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f832c15e2f0 0x7f832c156d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:35:48.472 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.462+0000 7f8331658700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f832c0cd950 0x7f832c0b52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3301/0 says I am v2:172.21.15.72:54974/0 (socket says 172.21.15.72:54974) 2023-10-14T10:35:48.472 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.462+0000 7f8331658700 1 -- 172.21.15.72:0/391945281 learned_addr learned my addr 172.21.15.72:0/391945281 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-14T10:35:48.472 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.462+0000 7f8331658700 1 -- 172.21.15.72:0/391945281 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f832c15e2f0 msgr2=0x7f832c156d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:35:48.472 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.462+0000 7f8331658700 1 --2- 172.21.15.72:0/391945281 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f832c15e2f0 0x7f832c156d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:48.473 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.462+0000 7f8331658700 1 -- 172.21.15.72:0/391945281 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f832c06af50 msgr2=0x7f832c15a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:35:48.473 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.462+0000 7f8331658700 1 --2- 172.21.15.72:0/391945281 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f832c06af50 0x7f832c15a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:48.473 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.462+0000 7f8331658700 1 -- 172.21.15.72:0/391945281 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8324009a70 con 0x7f832c0cd950 2023-10-14T10:35:48.473 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.462+0000 7f8331e59700 1 --2- 172.21.15.72:0/391945281 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f832c06af50 0x7f832c15a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-14T10:35:48.474 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.462+0000 7f8331658700 1 --2- 172.21.15.72:0/391945281 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f832c0cd950 0x7f832c0b52b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f831c00c4d0 tx=0x7f831c00c7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:35:48.474 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.462+0000 7f83227fc700 1 -- 172.21.15.72:0/391945281 <== mon.2 v2:172.21.15.72:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f831c00cea0 con 0x7f832c0cd950 2023-10-14T10:35:48.474 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.462+0000 7f83227fc700 1 -- 172.21.15.72:0/391945281 <== mon.2 v2:172.21.15.72:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f831c0054e0 con 0x7f832c0cd950 2023-10-14T10:35:48.474 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.462+0000 7f83340bd700 1 -- 172.21.15.72:0/391945281 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f832c0b3ea0 con 0x7f832c0cd950 2023-10-14T10:35:48.475 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.462+0000 7f83227fc700 1 -- 172.21.15.72:0/391945281 <== mon.2 v2:172.21.15.72:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f831c003910 con 0x7f832c0cd950 2023-10-14T10:35:48.475 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.462+0000 7f83340bd700 1 -- 172.21.15.72:0/391945281 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f832c0b4460 con 0x7f832c0cd950 2023-10-14T10:35:48.475 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.462+0000 7f83227fc700 1 -- 172.21.15.72:0/391945281 <== mon.2 v2:172.21.15.72:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f831c01c440 con 0x7f832c0cd950 2023-10-14T10:35:48.476 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.462+0000 7f83227fc700 1 -- 172.21.15.72:0/391945281 <== mon.2 v2:172.21.15.72:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f831c012a60 con 0x7f832c0cd950 2023-10-14T10:35:48.476 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.463+0000 7f83340bd700 1 -- 172.21.15.72:0/391945281 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f832c06e8e0 con 0x7f832c0cd950 2023-10-14T10:35:48.476 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.465+0000 7f83227fc700 1 -- 172.21.15.72:0/391945281 <== mon.2 v2:172.21.15.72:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f831c017070 con 0x7f832c0cd950 2023-10-14T10:35:48.507 INFO:tasks.ceph.mgr.x.smithi072.stderr:2023-10-14T10:35:48.505+0000 7fa493d36200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-14T10:35:48.515 INFO:tasks.ceph.mgr.y.smithi067.stderr:2023-10-14T10:35:48.513+0000 7f928a49e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-14T10:35:48.551 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.549+0000 7f83340bd700 1 -- 172.21.15.72:0/391945281 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_command({"prefix": "osd new", "uuid": "3fd6aa7d-50a9-45c9-9561-5902a044a119", "id": 7} v 0) v1 -- 0x7f832c067d20 con 0x7f832c0cd950 2023-10-14T10:35:48.561 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.559+0000 7f83227fc700 1 -- 172.21.15.72:0/391945281 <== mon.2 v2:172.21.15.72:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "3fd6aa7d-50a9-45c9-9561-5902a044a119", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f831c012400 con 0x7f832c0cd950 2023-10-14T10:35:48.561 INFO:teuthology.orchestra.run.smithi072.stdout:7 2023-10-14T10:35:48.563 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.560+0000 7f83340bd700 1 -- 172.21.15.72:0/391945281 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f832c0cd950 msgr2=0x7f832c0b52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:35:48.563 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.560+0000 7f83340bd700 1 --2- 172.21.15.72:0/391945281 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f832c0cd950 0x7f832c0b52b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f831c00c4d0 tx=0x7f831c00c7e0 comp rx=0 tx=0).stop 2023-10-14T10:35:48.563 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.560+0000 7f83340bd700 1 -- 172.21.15.72:0/391945281 shutdown_connections 2023-10-14T10:35:48.563 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.560+0000 7f83340bd700 1 --2- 172.21.15.72:0/391945281 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f832c15e2f0 0x7f832c156d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:48.564 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.560+0000 7f83340bd700 1 --2- 172.21.15.72:0/391945281 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f832c06af50 0x7f832c15a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:48.564 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.560+0000 7f83340bd700 1 --2- 172.21.15.72:0/391945281 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f832c0cd950 0x7f832c0b52b0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:48.564 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.560+0000 7f83340bd700 1 -- 172.21.15.72:0/391945281 >> 172.21.15.72:0/391945281 conn(0x7f832c06dae0 msgr2=0x7f832c15fd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:35:48.564 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.560+0000 7f83340bd700 1 -- 172.21.15.72:0/391945281 shutdown_connections 2023-10-14T10:35:48.565 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-14T10:35:48.560+0000 7f83340bd700 1 -- 172.21.15.72:0/391945281 wait complete. 2023-10-14T10:35:48.574 INFO:tasks.ceph.osd.0:Restarting daemon 2023-10-14T10:35:48.574 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-10-14T10:35:48.576 INFO:tasks.ceph.osd.0:Started 2023-10-14T10:35:48.577 INFO:tasks.ceph.osd.1:Restarting daemon 2023-10-14T10:35:48.577 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-10-14T10:35:48.579 INFO:tasks.ceph.osd.1:Started 2023-10-14T10:35:48.579 INFO:tasks.ceph.osd.2:Restarting daemon 2023-10-14T10:35:48.580 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-10-14T10:35:48.582 INFO:tasks.ceph.osd.2:Started 2023-10-14T10:35:48.582 INFO:tasks.ceph.osd.3:Restarting daemon 2023-10-14T10:35:48.582 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-10-14T10:35:48.585 INFO:tasks.ceph.osd.3:Started 2023-10-14T10:35:48.585 INFO:tasks.ceph.osd.4:Restarting daemon 2023-10-14T10:35:48.586 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-10-14T10:35:48.588 INFO:tasks.ceph.osd.4:Started 2023-10-14T10:35:48.588 INFO:tasks.ceph.osd.5:Restarting daemon 2023-10-14T10:35:48.589 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-10-14T10:35:48.590 INFO:tasks.ceph.osd.5:Started 2023-10-14T10:35:48.591 INFO:tasks.ceph.osd.6:Restarting daemon 2023-10-14T10:35:48.591 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-10-14T10:35:48.593 INFO:tasks.ceph.osd.6:Started 2023-10-14T10:35:48.594 INFO:tasks.ceph.osd.7:Restarting daemon 2023-10-14T10:35:48.594 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-10-14T10:35:48.596 INFO:tasks.ceph.osd.7:Started 2023-10-14T10:35:48.597 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-14T10:35:48.751 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:48.749+0000 7f8f1f2fa700 1 Processor -- start 2023-10-14T10:35:48.752 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:48.749+0000 7f8f1f2fa700 1 -- start start 2023-10-14T10:35:48.754 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:48.750+0000 7f8f1f2fa700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f8f180cd950 0x7f8f1815e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:35:48.755 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:48.750+0000 7f8f1f2fa700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f8f1806a840 0x7f8f1806acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:35:48.755 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:48.750+0000 7f8f1f2fa700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f180cf290 con 0x7f8f180cede0 2023-10-14T10:35:48.755 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:48.750+0000 7f8f1f2fa700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f181619a0 con 0x7f8f180cd950 2023-10-14T10:35:48.755 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:48.750+0000 7f8f1f2fa700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f1806b1f0 con 0x7f8f1806a840 2023-10-14T10:35:48.756 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:48.750+0000 7f8f1d897700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f8f1806a840 0x7f8f1806acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:35:48.756 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:48.750+0000 7f8f1d096700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f8f180cd950 0x7f8f1815e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:35:48.756 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:48.750+0000 7f8f1c895700 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f8f180cede0 0x7f8f1815f570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.67:59368/0 (socket says 172.21.15.67:59368) 2023-10-14T10:35:48.756 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:48.750+0000 7f8f1c895700 1 -- 172.21.15.67:0/2527341453 learned_addr learned my addr 172.21.15.67:0/2527341453 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:35:48.757 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:48.751+0000 7f8f1d897700 1 -- 172.21.15.67:0/2527341453 >> v2:172.21.15.72:3300/0 conn(0x7f8f180cd950 msgr2=0x7f8f1815e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:35:48.757 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:48.751+0000 7f8f1d897700 1 --2- 172.21.15.67:0/2527341453 >> v2:172.21.15.72:3300/0 conn(0x7f8f180cd950 0x7f8f1815e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:48.757 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:48.751+0000 7f8f1d897700 1 -- 172.21.15.67:0/2527341453 >> v1:172.21.15.72:6789/0 conn(0x7f8f180cede0 legacy=0x7f8f1815f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:35:48.758 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:48.751+0000 7f8f1d897700 1 -- 172.21.15.67:0/2527341453 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f180b2d60 con 0x7f8f1806a840 2023-10-14T10:35:48.758 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:48.751+0000 7f8f1d897700 1 --2- 172.21.15.67:0/2527341453 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f8f1806a840 0x7f8f1806acb0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f8f10009840 tx=0x7f8f10009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89c7980df68376eb server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:35:48.758 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:48.751+0000 7f8f17fff700 1 -- 172.21.15.67:0/2527341453 <== mon.2 v2:172.21.15.72:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f10011830 con 0x7f8f1806a840 2023-10-14T10:35:48.758 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:48.751+0000 7f8f17fff700 1 -- 172.21.15.67:0/2527341453 <== mon.2 v2:172.21.15.72:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f10011990 con 0x7f8f1806a840 2023-10-14T10:35:48.759 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:48.751+0000 7f8f17fff700 1 -- 172.21.15.67:0/2527341453 <== mon.2 v2:172.21.15.72:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f10011d20 con 0x7f8f1806a840 2023-10-14T10:35:48.759 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:48.752+0000 7f8f1f2fa700 1 -- 172.21.15.67:0/2527341453 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f8f1806a840 msgr2=0x7f8f1806acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:35:48.759 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:48.752+0000 7f8f1f2fa700 1 --2- 172.21.15.67:0/2527341453 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f8f1806a840 0x7f8f1806acb0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f8f10009840 tx=0x7f8f10009b50 comp rx=0 tx=0).stop 2023-10-14T10:35:48.759 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:48.752+0000 7f8f1f2fa700 1 -- 172.21.15.67:0/2527341453 shutdown_connections 2023-10-14T10:35:48.760 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:48.752+0000 7f8f1f2fa700 1 --2- 172.21.15.67:0/2527341453 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f8f1806a840 0x7f8f1806acb0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:48.760 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:48.752+0000 7f8f1f2fa700 1 --2- 172.21.15.67:0/2527341453 >> v2:172.21.15.72:3300/0 conn(0x7f8f180cd950 0x7f8f1815e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:48.760 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:48.752+0000 7f8f1f2fa700 1 -- 172.21.15.67:0/2527341453 >> 172.21.15.67:0/2527341453 conn(0x7f8f1806dae0 msgr2=0x7f8f1806def0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:35:48.760 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:48.752+0000 7f8f1f2fa700 1 -- 172.21.15.67:0/2527341453 shutdown_connections 2023-10-14T10:35:48.760 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:48.752+0000 7f8f1f2fa700 1 -- 172.21.15.67:0/2527341453 wait complete. 2023-10-14T10:35:48.763 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:48.753+0000 7f8f1f2fa700 1 Processor -- start 2023-10-14T10:35:48.763 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:48.753+0000 7f8f1f2fa700 1 -- start start 2023-10-14T10:35:48.764 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:48.754+0000 7f8f1f2fa700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8f1806a840 0x7f8f180b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:35:48.764 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:48.754+0000 7f8f1f2fa700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f8f180cd950 0x7f8f180cc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:35:48.764 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:48.754+0000 7f8f1f2fa700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8f180cede0 0x7f8f180cca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:35:48.764 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:48.754+0000 7f8f1f2fa700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f180b4440 con 0x7f8f1806a840 2023-10-14T10:35:48.764 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:48.754+0000 7f8f1f2fa700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f180b5f70 con 0x7f8f180cede0 2023-10-14T10:35:48.765 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:48.754+0000 7f8f1f2fa700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f180b55a0 con 0x7f8f180cd950 2023-10-14T10:35:48.765 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:48.755+0000 7f8f1d096700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8f1806a840 0x7f8f180b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:35:48.765 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:48.755+0000 7f8f1d096700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8f1806a840 0x7f8f180b5a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43364/0 (socket says 172.21.15.67:43364) 2023-10-14T10:35:48.765 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:48.755+0000 7f8f1d096700 1 -- 172.21.15.67:0/392338950 learned_addr learned my addr 172.21.15.67:0/392338950 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:35:48.765 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:48.755+0000 7f8f1c895700 1 --2- 172.21.15.67:0/392338950 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f8f180cd950 0x7f8f180cc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:35:48.766 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:48.755+0000 7f8f1d897700 1 --2- 172.21.15.67:0/392338950 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8f180cede0 0x7f8f180cca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:35:48.766 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:48.755+0000 7f8f1d096700 1 -- 172.21.15.67:0/392338950 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f8f180cd950 msgr2=0x7f8f180cc4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:35:48.766 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:48.755+0000 7f8f1d096700 1 --2- 172.21.15.67:0/392338950 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f8f180cd950 0x7f8f180cc4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:48.766 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:48.755+0000 7f8f1d096700 1 -- 172.21.15.67:0/392338950 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8f180cede0 msgr2=0x7f8f180cca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:35:48.766 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:48.755+0000 7f8f1d096700 1 --2- 172.21.15.67:0/392338950 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8f180cede0 0x7f8f180cca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:48.767 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:48.755+0000 7f8f1d096700 1 -- 172.21.15.67:0/392338950 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f100094a0 con 0x7f8f1806a840 2023-10-14T10:35:48.767 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:48.755+0000 7f8f1d096700 1 --2- 172.21.15.67:0/392338950 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8f1806a840 0x7f8f180b5a30 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8f08009db0 tx=0x7f8f0800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:35:48.767 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:48.755+0000 7f8f15ffb700 1 -- 172.21.15.67:0/392338950 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f08009c20 con 0x7f8f1806a840 2023-10-14T10:35:48.767 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:48.755+0000 7f8f15ffb700 1 -- 172.21.15.67:0/392338950 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f0800f040 con 0x7f8f1806a840 2023-10-14T10:35:48.767 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:48.755+0000 7f8f1f2fa700 1 -- 172.21.15.67:0/392338950 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f180b4c70 con 0x7f8f1806a840 2023-10-14T10:35:48.768 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:48.755+0000 7f8f15ffb700 1 -- 172.21.15.67:0/392338950 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f08014480 con 0x7f8f1806a840 2023-10-14T10:35:48.768 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:48.755+0000 7f8f1f2fa700 1 -- 172.21.15.67:0/392338950 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f180b5230 con 0x7f8f1806a840 2023-10-14T10:35:48.768 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:48.756+0000 7f8f15ffb700 1 -- 172.21.15.67:0/392338950 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f8f080148e0 con 0x7f8f1806a840 2023-10-14T10:35:48.768 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:48.756+0000 7f8f15ffb700 1 -- 172.21.15.67:0/392338950 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f8f08013ab0 con 0x7f8f1806a840 2023-10-14T10:35:48.768 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:48.756+0000 7f8f1f2fa700 1 -- 172.21.15.67:0/392338950 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f18067d20 con 0x7f8f1806a840 2023-10-14T10:35:48.769 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:48.759+0000 7f8f15ffb700 1 -- 172.21.15.67:0/392338950 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f8f08018020 con 0x7f8f1806a840 2023-10-14T10:35:48.831 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:48.829+0000 7f8f1f2fa700 1 -- 172.21.15.67:0/392338950 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8f180b2100 con 0x7f8f1806a840 2023-10-14T10:35:48.832 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:48.830+0000 7f8f15ffb700 1 -- 172.21.15.67:0/392338950 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f8f08013d10 con 0x7f8f1806a840 2023-10-14T10:35:48.833 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-14T10:35:48.833 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":10,"fsid":"37f9ab46-3082-4b71-8b24-08d39897b008","created":"2023-10-14T10:35:46.067554+0000","modified":"2023-10-14T10:35:48.551490+0000","last_up_change":"0.000000","last_in_change":"2023-10-14T10:35:48.551490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"2b59a07e-11ef-440c-b766-9f64ac885838","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"6f813bd6-fa0b-487d-a204-431eb1276a60","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"6af42c6a-1b59-4669-9c04-b6876788446d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"1d889d76-a646-499d-bfff-ed5bae5434ff","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"c3271dff-b437-4fdd-b13b-6e49a35473af","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"0337bb18-4b68-4eff-b528-a667275e6926","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"8e92d9cf-0c8c-46eb-89a7-db2023ef1509","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"3fd6aa7d-50a9-45c9-9561-5902a044a119","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-14T10:35:48.834 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:48.831+0000 7f8f1f2fa700 1 -- 172.21.15.67:0/392338950 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8f1806a840 msgr2=0x7f8f180b5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:35:48.834 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:48.831+0000 7f8f1f2fa700 1 --2- 172.21.15.67:0/392338950 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8f1806a840 0x7f8f180b5a30 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8f08009db0 tx=0x7f8f0800c3a0 comp rx=0 tx=0).stop 2023-10-14T10:35:48.834 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:48.831+0000 7f8f1f2fa700 1 -- 172.21.15.67:0/392338950 shutdown_connections 2023-10-14T10:35:48.835 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:48.831+0000 7f8f1f2fa700 1 --2- 172.21.15.67:0/392338950 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8f180cede0 0x7f8f180cca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:48.835 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:48.831+0000 7f8f1f2fa700 1 --2- 172.21.15.67:0/392338950 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8f1806a840 0x7f8f180b5a30 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:48.835 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:48.831+0000 7f8f1f2fa700 1 --2- 172.21.15.67:0/392338950 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f8f180cd950 0x7f8f180cc4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:48.835 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:48.831+0000 7f8f1f2fa700 1 -- 172.21.15.67:0/392338950 >> 172.21.15.67:0/392338950 conn(0x7f8f1806dae0 msgr2=0x7f8f180b18d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:35:48.836 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:48.831+0000 7f8f1f2fa700 1 -- 172.21.15.67:0/392338950 shutdown_connections 2023-10-14T10:35:48.836 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:48.831+0000 7f8f1f2fa700 1 -- 172.21.15.67:0/392338950 wait complete. 2023-10-14T10:35:48.845 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-10-14T10:35:48.846 INFO:tasks.ceph:Waiting for OSDs to come up 2023-10-14T10:35:48.939 INFO:tasks.ceph.osd.0.smithi067.stderr:2023-10-14T10:35:48.937+0000 7effb54da640 -1 Falling back to public interface 2023-10-14T10:35:49.208 INFO:tasks.ceph.osd.4.smithi072.stderr:2023-10-14T10:35:49.206+0000 7f6939f89640 -1 Falling back to public interface 2023-10-14T10:35:49.209 INFO:tasks.ceph.osd.5.smithi072.stderr:2023-10-14T10:35:49.206+0000 7f9ed1763640 -1 Falling back to public interface 2023-10-14T10:35:49.209 INFO:tasks.ceph.osd.4.smithi072.stderr:2023-10-14T10:35:49.206+0000 7f6931347700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6800/0: (98) Address already in use 2023-10-14T10:35:49.213 INFO:tasks.ceph.osd.6.smithi072.stderr:2023-10-14T10:35:49.211+0000 7f7869701640 -1 Falling back to public interface 2023-10-14T10:35:49.216 INFO:tasks.ceph.osd.2.smithi067.stderr:2023-10-14T10:35:49.214+0000 7f8726455640 -1 Falling back to public interface 2023-10-14T10:35:49.218 INFO:tasks.ceph.osd.7.smithi072.stderr:2023-10-14T10:35:49.216+0000 7ff6fd69b640 -1 Falling back to public interface 2023-10-14T10:35:49.219 INFO:tasks.ceph.osd.1.smithi067.stderr:2023-10-14T10:35:49.217+0000 7fbf6ffd6640 -1 Falling back to public interface 2023-10-14T10:35:49.220 INFO:tasks.ceph.osd.3.smithi067.stderr:2023-10-14T10:35:49.218+0000 7f7191506640 -1 Falling back to public interface 2023-10-14T10:35:49.246 INFO:tasks.ceph.mgr.x.smithi072.stderr:2023-10-14T10:35:49.244+0000 7fa493d36200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-14T10:35:49.296 INFO:tasks.ceph.mgr.y.smithi067.stderr:2023-10-14T10:35:49.294+0000 7f928a49e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-14T10:35:49.322 INFO:tasks.ceph.mgr.x.smithi072.stderr:2023-10-14T10:35:49.320+0000 7fa493d36200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-14T10:35:49.377 INFO:tasks.ceph.mgr.y.smithi067.stderr:2023-10-14T10:35:49.375+0000 7f928a49e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-14T10:35:49.461 INFO:tasks.ceph.mgr.x.smithi072.stderr:2023-10-14T10:35:49.459+0000 7fa493d36200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-14T10:35:49.524 INFO:tasks.ceph.mgr.y.smithi067.stderr:2023-10-14T10:35:49.522+0000 7f928a49e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-14T10:35:49.652 DEBUG:teuthology.orchestra.run.smithi067:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-14T10:35:49.696 INFO:tasks.ceph.mgr.x.smithi072.stderr:2023-10-14T10:35:49.694+0000 7fa493d36200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-14T10:35:49.768 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:49.766+0000 7f91e6432700 1 Processor -- start 2023-10-14T10:35:49.768 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:49.766+0000 7f91e6432700 1 -- start start 2023-10-14T10:35:49.773 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:49.767+0000 7f91e6432700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f91e00cb140 0x7f91e006b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:35:49.773 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:49.767+0000 7f91e6432700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f91e006be20 0x7f91e00cd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:35:49.773 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:49.767+0000 7f91e6432700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91e013f5b0 con 0x7f91e013ced0 2023-10-14T10:35:49.773 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:49.767+0000 7f91e6432700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91e0157d60 con 0x7f91e006be20 2023-10-14T10:35:49.774 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:49.767+0000 7f91e6432700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91e00cf580 con 0x7f91e00cb140 2023-10-14T10:35:49.774 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:49.767+0000 7f91dffff700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f91e013ced0 0x7f91e00cee70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:51138/0 (socket says 172.21.15.67:51138) 2023-10-14T10:35:49.774 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:49.767+0000 7f91dffff700 1 -- 172.21.15.67:0/4219242997 learned_addr learned my addr 172.21.15.67:0/4219242997 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:35:49.774 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:49.767+0000 7f91df7fe700 1 --2- 172.21.15.67:0/4219242997 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f91e00cb140 0x7f91e006b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:35:49.775 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:49.767+0000 7f91e49cf700 1 --2- 172.21.15.67:0/4219242997 >> v2:172.21.15.72:3300/0 conn(0x7f91e006be20 0x7f91e00cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:35:49.775 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:49.768+0000 7f91deffd700 1 -- 172.21.15.67:0/4219242997 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3189709435 0 0) 0x7f91e013f5b0 con 0x7f91e013ced0 2023-10-14T10:35:49.775 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:49.768+0000 7f91deffd700 1 -- 172.21.15.67:0/4219242997 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91c80035b0 con 0x7f91e013ced0 2023-10-14T10:35:49.775 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:49.768+0000 7f91e49cf700 1 -- 172.21.15.67:0/4219242997 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f91e00cb140 msgr2=0x7f91e006b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:35:49.775 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:49.768+0000 7f91e49cf700 1 --2- 172.21.15.67:0/4219242997 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f91e00cb140 0x7f91e006b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:49.775 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:49.768+0000 7f91e49cf700 1 -- 172.21.15.67:0/4219242997 >> v1:172.21.15.67:6789/0 conn(0x7f91e013ced0 legacy=0x7f91e00cee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:35:49.776 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:49.768+0000 7f91e49cf700 1 -- 172.21.15.67:0/4219242997 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91e013f800 con 0x7f91e006be20 2023-10-14T10:35:49.776 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:49.768+0000 7f91df7fe700 1 --2- 172.21.15.67:0/4219242997 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f91e00cb140 0x7f91e006b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-14T10:35:49.776 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:49.768+0000 7f91e49cf700 1 --2- 172.21.15.67:0/4219242997 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f91e006be20 0x7f91e00cd920 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f91d8009d90 tx=0x7f91d800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1441b87dfc6824da server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:35:49.776 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:49.768+0000 7f91deffd700 1 -- 172.21.15.67:0/4219242997 <== mon.1 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d800e070 con 0x7f91e006be20 2023-10-14T10:35:49.776 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:49.768+0000 7f91deffd700 1 -- 172.21.15.67:0/4219242997 <== mon.1 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91d800bb20 con 0x7f91e006be20 2023-10-14T10:35:49.776 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:49.768+0000 7f91deffd700 1 -- 172.21.15.67:0/4219242997 <== mon.1 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d8012430 con 0x7f91e006be20 2023-10-14T10:35:49.777 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:49.769+0000 7f91e6432700 1 -- 172.21.15.67:0/4219242997 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f91e006be20 msgr2=0x7f91e00cd920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:35:49.777 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:49.769+0000 7f91e6432700 1 --2- 172.21.15.67:0/4219242997 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f91e006be20 0x7f91e00cd920 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f91d8009d90 tx=0x7f91d800b330 comp rx=0 tx=0).stop 2023-10-14T10:35:49.777 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:49.769+0000 7f91e6432700 1 -- 172.21.15.67:0/4219242997 shutdown_connections 2023-10-14T10:35:49.777 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:49.769+0000 7f91e6432700 1 --2- 172.21.15.67:0/4219242997 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f91e006be20 0x7f91e00cd920 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:49.778 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:49.769+0000 7f91e6432700 1 --2- 172.21.15.67:0/4219242997 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f91e00cb140 0x7f91e006b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:49.778 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:49.769+0000 7f91e6432700 1 -- 172.21.15.67:0/4219242997 >> 172.21.15.67:0/4219242997 conn(0x7f91e0071210 msgr2=0x7f91e0071620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:35:49.778 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:49.769+0000 7f91e6432700 1 -- 172.21.15.67:0/4219242997 shutdown_connections 2023-10-14T10:35:49.778 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:49.769+0000 7f91e6432700 1 -- 172.21.15.67:0/4219242997 wait complete. 2023-10-14T10:35:49.779 INFO:tasks.ceph.mgr.y.smithi067.stderr:2023-10-14T10:35:49.770+0000 7f928a49e200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-14T10:35:49.782 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:49.770+0000 7f91e6432700 1 Processor -- start 2023-10-14T10:35:49.782 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:49.771+0000 7f91e6432700 1 -- start start 2023-10-14T10:35:49.783 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:49.772+0000 7f91e6432700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f91e006be20 0x7f91e015b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:35:49.783 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:49.772+0000 7f91e6432700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f91e00cb140 0x7f91e006f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:35:49.783 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:49.772+0000 7f91e6432700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f91e013ced0 0x7f91e013c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:35:49.784 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:49.772+0000 7f91e6432700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91e00c8890 con 0x7f91e013ced0 2023-10-14T10:35:49.784 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:49.772+0000 7f91e6432700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91e013ca80 con 0x7f91e006be20 2023-10-14T10:35:49.784 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:49.772+0000 7f91e6432700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91e006dbd0 con 0x7f91e00cb140 2023-10-14T10:35:49.785 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:49.772+0000 7f91e49cf700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f91e013ced0 0x7f91e013c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:35:49.785 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:49.772+0000 7f91e49cf700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f91e013ced0 0x7f91e013c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43380/0 (socket says 172.21.15.67:43380) 2023-10-14T10:35:49.785 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:49.772+0000 7f91e49cf700 1 -- 172.21.15.67:0/2314208827 learned_addr learned my addr 172.21.15.67:0/2314208827 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:35:49.785 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:49.772+0000 7f91dffff700 1 --2- 172.21.15.67:0/2314208827 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f91e006be20 0x7f91e015b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:35:49.786 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:49.772+0000 7f91df7fe700 1 --2- 172.21.15.67:0/2314208827 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f91e00cb140 0x7f91e006f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:35:49.786 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:49.772+0000 7f91e49cf700 1 -- 172.21.15.67:0/2314208827 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f91e00cb140 msgr2=0x7f91e006f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:35:49.786 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:49.772+0000 7f91e49cf700 1 --2- 172.21.15.67:0/2314208827 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f91e00cb140 0x7f91e006f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:49.786 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:49.772+0000 7f91e49cf700 1 -- 172.21.15.67:0/2314208827 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f91e006be20 msgr2=0x7f91e015b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:35:49.787 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:49.772+0000 7f91e49cf700 1 --2- 172.21.15.67:0/2314208827 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f91e006be20 0x7f91e015b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:49.787 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:49.772+0000 7f91e49cf700 1 -- 172.21.15.67:0/2314208827 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91d40094a0 con 0x7f91e013ced0 2023-10-14T10:35:49.787 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:49.772+0000 7f91e49cf700 1 --2- 172.21.15.67:0/2314208827 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f91e013ced0 0x7f91e013c540 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f91d8014040 tx=0x7f91d8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:35:49.787 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:49.772+0000 7f91dcff9700 1 -- 172.21.15.67:0/2314208827 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d80129f0 con 0x7f91e013ced0 2023-10-14T10:35:49.787 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:49.772+0000 7f91dcff9700 1 -- 172.21.15.67:0/2314208827 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91d8020070 con 0x7f91e013ced0 2023-10-14T10:35:49.787 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:49.772+0000 7f91e6432700 1 -- 172.21.15.67:0/2314208827 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91d8009a70 con 0x7f91e013ced0 2023-10-14T10:35:49.788 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:49.772+0000 7f91dcff9700 1 -- 172.21.15.67:0/2314208827 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d8012db0 con 0x7f91e013ced0 2023-10-14T10:35:49.788 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:49.773+0000 7f91e6432700 1 -- 172.21.15.67:0/2314208827 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91e006e190 con 0x7f91e013ced0 2023-10-14T10:35:49.788 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:49.773+0000 7f91dcff9700 1 -- 172.21.15.67:0/2314208827 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f91d8004340 con 0x7f91e013ced0 2023-10-14T10:35:49.788 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:49.773+0000 7f91dcff9700 1 -- 172.21.15.67:0/2314208827 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f91d8022070 con 0x7f91e013ced0 2023-10-14T10:35:49.788 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:49.773+0000 7f91e6432700 1 -- 172.21.15.67:0/2314208827 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91e00c85f0 con 0x7f91e013ced0 2023-10-14T10:35:49.789 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:49.776+0000 7f91dcff9700 1 -- 172.21.15.67:0/2314208827 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f91e00c85f0 con 0x7f91e013ced0 2023-10-14T10:35:49.849 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:49.847+0000 7f91e6432700 1 -- 172.21.15.67:0/2314208827 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f91e00c89d0 con 0x7f91e013ced0 2023-10-14T10:35:49.850 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:49.847+0000 7f91dcff9700 1 -- 172.21.15.67:0/2314208827 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f91d80047d0 con 0x7f91e013ced0 2023-10-14T10:35:49.851 INFO:teuthology.misc.health.smithi067.stdout: 2023-10-14T10:35:49.851 INFO:teuthology.misc.health.smithi067.stdout:{"epoch":10,"fsid":"37f9ab46-3082-4b71-8b24-08d39897b008","created":"2023-10-14T10:35:46.067554+0000","modified":"2023-10-14T10:35:48.551490+0000","last_up_change":"0.000000","last_in_change":"2023-10-14T10:35:48.551490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"2b59a07e-11ef-440c-b766-9f64ac885838","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"6f813bd6-fa0b-487d-a204-431eb1276a60","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"6af42c6a-1b59-4669-9c04-b6876788446d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"1d889d76-a646-499d-bfff-ed5bae5434ff","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"c3271dff-b437-4fdd-b13b-6e49a35473af","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"0337bb18-4b68-4eff-b528-a667275e6926","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"8e92d9cf-0c8c-46eb-89a7-db2023ef1509","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"3fd6aa7d-50a9-45c9-9561-5902a044a119","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-14T10:35:49.852 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:49.848+0000 7f91e6432700 1 -- 172.21.15.67:0/2314208827 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f91e013ced0 msgr2=0x7f91e013c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:35:49.852 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:49.848+0000 7f91e6432700 1 --2- 172.21.15.67:0/2314208827 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f91e013ced0 0x7f91e013c540 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f91d8014040 tx=0x7f91d8004ad0 comp rx=0 tx=0).stop 2023-10-14T10:35:49.853 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:49.848+0000 7f91e6432700 1 -- 172.21.15.67:0/2314208827 shutdown_connections 2023-10-14T10:35:49.853 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:49.848+0000 7f91e6432700 1 --2- 172.21.15.67:0/2314208827 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f91e013ced0 0x7f91e013c540 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:49.853 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:49.848+0000 7f91e6432700 1 --2- 172.21.15.67:0/2314208827 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f91e006be20 0x7f91e015b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:49.853 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:49.848+0000 7f91e6432700 1 --2- 172.21.15.67:0/2314208827 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f91e00cb140 0x7f91e006f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:49.853 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:49.848+0000 7f91e6432700 1 -- 172.21.15.67:0/2314208827 >> 172.21.15.67:0/2314208827 conn(0x7f91e0071210 msgr2=0x7f91e0151710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:35:49.854 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:49.849+0000 7f91e6432700 1 -- 172.21.15.67:0/2314208827 shutdown_connections 2023-10-14T10:35:49.854 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:49.849+0000 7f91e6432700 1 -- 172.21.15.67:0/2314208827 wait complete. 2023-10-14T10:35:49.862 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-10-14T10:35:49.906 INFO:tasks.ceph.mgr.x.smithi072.stderr:2023-10-14T10:35:49.904+0000 7fa493d36200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-14T10:35:49.995 INFO:tasks.ceph.mgr.y.smithi067.stderr:2023-10-14T10:35:49.993+0000 7f928a49e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-14T10:35:50.037 INFO:tasks.ceph.mgr.x.smithi072.stderr:2023-10-14T10:35:50.035+0000 7fa493d36200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-14T10:35:50.136 INFO:tasks.ceph.mgr.y.smithi067.stderr:2023-10-14T10:35:50.134+0000 7f928a49e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-14T10:35:50.258 INFO:tasks.ceph.mgr.x.smithi072.stderr:2023-10-14T10:35:50.256+0000 7fa493d36200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-14T10:35:50.366 INFO:tasks.ceph.mgr.y.smithi067.stderr:2023-10-14T10:35:50.364+0000 7f928a49e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-14T10:35:50.413 INFO:tasks.ceph.mgr.x.smithi072.stderr:2023-10-14T10:35:50.411+0000 7fa493d36200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-14T10:35:50.527 INFO:tasks.ceph.mgr.x.smithi072.stderr:2023-10-14T10:35:50.525+0000 7fa493d36200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-14T10:35:50.528 INFO:tasks.ceph.mgr.y.smithi067.stderr:2023-10-14T10:35:50.526+0000 7f928a49e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-14T10:35:50.650 INFO:tasks.ceph.mgr.y.smithi067.stderr:2023-10-14T10:35:50.648+0000 7f928a49e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-14T10:35:50.849 INFO:tasks.ceph.mgr.x.smithi072.stderr:2023-10-14T10:35:50.847+0000 7fa493d36200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-14T10:35:50.950 INFO:tasks.ceph.mgr.x.smithi072.stderr:2023-10-14T10:35:50.948+0000 7fa493d36200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-14T10:35:50.985 INFO:tasks.ceph.mgr.y.smithi067.stderr:2023-10-14T10:35:50.983+0000 7f928a49e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-14T10:35:51.090 INFO:tasks.ceph.mgr.y.smithi067.stderr:2023-10-14T10:35:51.088+0000 7f928a49e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-14T10:35:51.136 INFO:tasks.ceph.mgr.x.smithi072.stderr:2023-10-14T10:35:51.134+0000 7fa493d36200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-14T10:35:51.285 INFO:tasks.ceph.mgr.y.smithi067.stderr:2023-10-14T10:35:51.283+0000 7f928a49e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-14T10:35:51.734 INFO:tasks.ceph.mgr.x.smithi072.stderr:2023-10-14T10:35:51.732+0000 7fa493d36200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-14T10:35:51.932 INFO:tasks.ceph.mgr.y.smithi067.stderr:2023-10-14T10:35:51.930+0000 7f928a49e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-14T10:35:52.464 INFO:tasks.ceph.osd.0.smithi067.stderr:2023-10-14T10:35:52.462+0000 7effb54da640 -1 osd.0 0 log_to_monitors true 2023-10-14T10:35:52.754 INFO:tasks.ceph.osd.6.smithi072.stderr:2023-10-14T10:35:52.752+0000 7f7869701640 -1 osd.6 0 log_to_monitors true 2023-10-14T10:35:52.763 INFO:tasks.ceph.mgr.x.smithi072.stderr:2023-10-14T10:35:52.761+0000 7fa44bcc4700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-10-14T10:35:52.764 INFO:tasks.ceph.osd.3.smithi067.stderr:2023-10-14T10:35:52.762+0000 7f7191506640 -1 osd.3 0 log_to_monitors true 2023-10-14T10:35:52.766 INFO:tasks.ceph.mgr.x.smithi072.stderr:2023-10-14T10:35:52.764+0000 7fa44bcc4700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-10-14T10:35:52.775 INFO:tasks.ceph.osd.5.smithi072.stderr:2023-10-14T10:35:52.773+0000 7f9ed1763640 -1 osd.5 0 log_to_monitors true 2023-10-14T10:35:52.776 INFO:tasks.ceph.osd.4.smithi072.stderr:2023-10-14T10:35:52.773+0000 7f6939f89640 -1 osd.4 0 log_to_monitors true 2023-10-14T10:35:52.777 INFO:tasks.ceph.osd.7.smithi072.stderr:2023-10-14T10:35:52.775+0000 7ff6fd69b640 -1 osd.7 0 log_to_monitors true 2023-10-14T10:35:52.785 INFO:tasks.ceph.osd.1.smithi067.stderr:2023-10-14T10:35:52.784+0000 7fbf6ffd6640 -1 osd.1 0 log_to_monitors true 2023-10-14T10:35:52.838 INFO:tasks.ceph.osd.2.smithi067.stderr:2023-10-14T10:35:52.836+0000 7f8726455640 -1 osd.2 0 log_to_monitors true 2023-10-14T10:35:55.643 INFO:tasks.ceph.osd.3.smithi067.stderr:2023-10-14T10:35:55.641+0000 7f71850bd700 -1 osd.3 0 waiting for initial osdmap 2023-10-14T10:35:55.656 INFO:tasks.ceph.osd.1.smithi067.stderr:2023-10-14T10:35:55.654+0000 7fbf63b8d700 -1 osd.1 0 waiting for initial osdmap 2023-10-14T10:35:55.657 INFO:tasks.ceph.osd.3.smithi067.stderr:2023-10-14T10:35:55.656+0000 7f717d498700 -1 osd.3 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-14T10:35:55.667 INFO:tasks.ceph.osd.0.smithi067.stderr:2023-10-14T10:35:55.665+0000 7effa9091700 -1 osd.0 0 waiting for initial osdmap 2023-10-14T10:35:55.670 INFO:tasks.ceph.osd.1.smithi067.stderr:2023-10-14T10:35:55.669+0000 7fbf5bf68700 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-14T10:35:55.683 INFO:tasks.ceph.osd.0.smithi067.stderr:2023-10-14T10:35:55.681+0000 7effa146c700 -1 osd.0 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-14T10:35:55.871 INFO:tasks.ceph.osd.5.smithi072.stderr:2023-10-14T10:35:55.870+0000 7f9ec531a700 -1 osd.5 0 waiting for initial osdmap 2023-10-14T10:35:55.885 INFO:tasks.ceph.osd.7.smithi072.stderr:2023-10-14T10:35:55.884+0000 7ff6f1252700 -1 osd.7 0 waiting for initial osdmap 2023-10-14T10:35:55.888 INFO:tasks.ceph.osd.4.smithi072.stderr:2023-10-14T10:35:55.887+0000 7f692db40700 -1 osd.4 0 waiting for initial osdmap 2023-10-14T10:35:55.890 INFO:tasks.ceph.osd.5.smithi072.stderr:2023-10-14T10:35:55.889+0000 7f9ebd6f5700 -1 osd.5 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-14T10:35:55.902 INFO:tasks.ceph.osd.7.smithi072.stderr:2023-10-14T10:35:55.901+0000 7ff6e962d700 -1 osd.7 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-14T10:35:55.906 INFO:tasks.ceph.osd.6.smithi072.stderr:2023-10-14T10:35:55.905+0000 7f785d2b8700 -1 osd.6 0 waiting for initial osdmap 2023-10-14T10:35:55.907 INFO:tasks.ceph.osd.4.smithi072.stderr:2023-10-14T10:35:55.906+0000 7f6925f1b700 -1 osd.4 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-14T10:35:55.923 INFO:tasks.ceph.osd.6.smithi072.stderr:2023-10-14T10:35:55.922+0000 7f7855693700 -1 osd.6 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-14T10:35:56.018 INFO:tasks.ceph.osd.2.smithi067.stderr:2023-10-14T10:35:56.016+0000 7f871a00c700 -1 osd.2 0 waiting for initial osdmap 2023-10-14T10:35:56.032 INFO:tasks.ceph.osd.2.smithi067.stderr:2023-10-14T10:35:56.031+0000 7f87123e7700 -1 osd.2 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-14T10:35:56.668 DEBUG:teuthology.orchestra.run.smithi067:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-14T10:35:56.793 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:56.791+0000 7fd476b8b700 1 Processor -- start 2023-10-14T10:35:56.794 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:56.791+0000 7fd476b8b700 1 -- start start 2023-10-14T10:35:56.802 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:56.792+0000 7fd476b8b700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fd470158210 0x7fd4700b3200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:35:56.802 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:56.792+0000 7fd476b8b700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fd47015ece0 0x7fd4700b3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:35:56.802 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:56.792+0000 7fd476b8b700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd4700b4290 0x7fd4700b46f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:35:56.803 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:56.792+0000 7fd476b8b700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd47015dff0 con 0x7fd4700b4290 2023-10-14T10:35:56.803 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:56.792+0000 7fd476b8b700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4700b4c30 con 0x7fd470158210 2023-10-14T10:35:56.803 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:56.792+0000 7fd476b8b700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4700afab0 con 0x7fd47015ece0 2023-10-14T10:35:56.803 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:56.792+0000 7fd475128700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd4700b4290 0x7fd4700b46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:35:56.803 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:56.792+0000 7fd475128700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd4700b4290 0x7fd4700b46f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43548/0 (socket says 172.21.15.67:43548) 2023-10-14T10:35:56.804 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:56.792+0000 7fd475128700 1 -- 172.21.15.67:0/4294489279 learned_addr learned my addr 172.21.15.67:0/4294489279 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:35:56.804 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:56.793+0000 7fd46ffff700 1 --2- 172.21.15.67:0/4294489279 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fd47015ece0 0x7fd4700b3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:35:56.804 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:56.793+0000 7fd474927700 1 --2- 172.21.15.67:0/4294489279 >> v2:172.21.15.72:3300/0 conn(0x7fd470158210 0x7fd4700b3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:35:56.804 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:56.793+0000 7fd475128700 1 -- 172.21.15.67:0/4294489279 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fd47015ece0 msgr2=0x7fd4700b3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:35:56.804 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:56.793+0000 7fd475128700 1 --2- 172.21.15.67:0/4294489279 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fd47015ece0 0x7fd4700b3740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:56.805 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:56.793+0000 7fd475128700 1 -- 172.21.15.67:0/4294489279 >> v2:172.21.15.72:3300/0 conn(0x7fd470158210 msgr2=0x7fd4700b3200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:35:56.805 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:56.793+0000 7fd475128700 1 --2- 172.21.15.67:0/4294489279 >> v2:172.21.15.72:3300/0 conn(0x7fd470158210 0x7fd4700b3200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:56.805 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:56.793+0000 7fd475128700 1 -- 172.21.15.67:0/4294489279 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4700b3d70 con 0x7fd4700b4290 2023-10-14T10:35:56.805 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:56.793+0000 7fd474927700 1 --2- 172.21.15.67:0/4294489279 >> v2:172.21.15.72:3300/0 conn(0x7fd470158210 0x7fd4700b3200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-14T10:35:56.805 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:56.793+0000 7fd475128700 1 --2- 172.21.15.67:0/4294489279 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4700b4290 0x7fd4700b46f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd470075c90 tx=0x7fd46800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25a6e2f9c863f28a server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:35:56.806 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:56.793+0000 7fd46f7fe700 1 -- 172.21.15.67:0/4294489279 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd46800e070 con 0x7fd4700b4290 2023-10-14T10:35:56.806 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:56.793+0000 7fd46f7fe700 1 -- 172.21.15.67:0/4294489279 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd46800b9c0 con 0x7fd4700b4290 2023-10-14T10:35:56.806 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:56.793+0000 7fd46f7fe700 1 -- 172.21.15.67:0/4294489279 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd46800bd70 con 0x7fd4700b4290 2023-10-14T10:35:56.806 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:56.793+0000 7fd476b8b700 1 -- 172.21.15.67:0/4294489279 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4700b4290 msgr2=0x7fd4700b46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:35:56.806 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:56.793+0000 7fd476b8b700 1 --2- 172.21.15.67:0/4294489279 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4700b4290 0x7fd4700b46f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd470075c90 tx=0x7fd46800b330 comp rx=0 tx=0).stop 2023-10-14T10:35:56.807 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:56.793+0000 7fd476b8b700 1 -- 172.21.15.67:0/4294489279 shutdown_connections 2023-10-14T10:35:56.807 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:56.793+0000 7fd476b8b700 1 --2- 172.21.15.67:0/4294489279 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4700b4290 0x7fd4700b46f0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:56.807 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:56.793+0000 7fd476b8b700 1 --2- 172.21.15.67:0/4294489279 >> v2:172.21.15.72:3300/0 conn(0x7fd470158210 0x7fd4700b3200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:56.807 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:56.793+0000 7fd476b8b700 1 --2- 172.21.15.67:0/4294489279 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fd47015ece0 0x7fd4700b3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:56.807 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:56.794+0000 7fd476b8b700 1 -- 172.21.15.67:0/4294489279 >> 172.21.15.67:0/4294489279 conn(0x7fd4700cce30 msgr2=0x7fd4700cd210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:35:56.808 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:56.794+0000 7fd476b8b700 1 -- 172.21.15.67:0/4294489279 shutdown_connections 2023-10-14T10:35:56.808 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:56.794+0000 7fd476b8b700 1 -- 172.21.15.67:0/4294489279 wait complete. 2023-10-14T10:35:56.808 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:56.795+0000 7fd476b8b700 1 Processor -- start 2023-10-14T10:35:56.808 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:56.795+0000 7fd476b8b700 1 -- start start 2023-10-14T10:35:56.808 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:56.796+0000 7fd476b8b700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4700b4290 0x7fd4700b2940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:35:56.809 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:56.796+0000 7fd476b8b700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd470158210 0x7fd470152b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:35:56.809 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:56.796+0000 7fd476b8b700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fd47015ece0 0x7fd4700afe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:35:56.809 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:56.796+0000 7fd476b8b700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd470153060 con 0x7fd4700b4290 2023-10-14T10:35:56.809 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:56.796+0000 7fd476b8b700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4700c5500 con 0x7fd470158210 2023-10-14T10:35:56.810 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:56.796+0000 7fd476b8b700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4700cbca0 con 0x7fd47015ece0 2023-10-14T10:35:56.810 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:56.796+0000 7fd474927700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4700b4290 0x7fd4700b2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:35:56.810 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:56.796+0000 7fd474927700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4700b4290 0x7fd4700b2940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43550/0 (socket says 172.21.15.67:43550) 2023-10-14T10:35:56.811 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:56.796+0000 7fd474927700 1 -- 172.21.15.67:0/643149141 learned_addr learned my addr 172.21.15.67:0/643149141 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:35:56.811 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:56.796+0000 7fd475128700 1 --2- 172.21.15.67:0/643149141 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fd47015ece0 0x7fd4700afe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:35:56.811 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:56.796+0000 7fd46ffff700 1 --2- 172.21.15.67:0/643149141 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd470158210 0x7fd470152b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:35:56.811 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:56.796+0000 7fd474927700 1 -- 172.21.15.67:0/643149141 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fd47015ece0 msgr2=0x7fd4700afe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:35:56.811 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:56.796+0000 7fd474927700 1 --2- 172.21.15.67:0/643149141 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fd47015ece0 0x7fd4700afe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:56.811 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:56.796+0000 7fd474927700 1 -- 172.21.15.67:0/643149141 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd470158210 msgr2=0x7fd470152b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:35:56.812 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:56.796+0000 7fd474927700 1 --2- 172.21.15.67:0/643149141 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd470158210 0x7fd470152b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:56.812 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:56.796+0000 7fd474927700 1 -- 172.21.15.67:0/643149141 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd468009b30 con 0x7fd4700b4290 2023-10-14T10:35:56.812 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:56.796+0000 7fd474927700 1 --2- 172.21.15.67:0/643149141 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4700b4290 0x7fd4700b2940 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd46400c750 tx=0x7fd46400ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:35:56.812 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:56.796+0000 7fd46d7fa700 1 -- 172.21.15.67:0/643149141 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd464005c20 con 0x7fd4700b4290 2023-10-14T10:35:56.812 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:56.796+0000 7fd46d7fa700 1 -- 172.21.15.67:0/643149141 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd464005d80 con 0x7fd4700b4290 2023-10-14T10:35:56.813 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:56.796+0000 7fd46d7fa700 1 -- 172.21.15.67:0/643149141 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd464011840 con 0x7fd4700b4290 2023-10-14T10:35:56.813 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:56.796+0000 7fd476b8b700 1 -- 172.21.15.67:0/643149141 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4700cb090 con 0x7fd4700b4290 2023-10-14T10:35:56.813 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:56.796+0000 7fd476b8b700 1 -- 172.21.15.67:0/643149141 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4700cb650 con 0x7fd4700b4290 2023-10-14T10:35:56.813 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:56.797+0000 7fd476b8b700 1 -- 172.21.15.67:0/643149141 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd470074310 con 0x7fd4700b4290 2023-10-14T10:35:56.813 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:56.798+0000 7fd46d7fa700 1 -- 172.21.15.67:0/643149141 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 4) v1 ==== 89173+0+0 (secure 0 0 0) 0x7fd4640119a0 con 0x7fd4700b4290 2023-10-14T10:35:56.814 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:56.798+0000 7fd46d7fa700 1 --2- 172.21.15.67:0/643149141 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fd45406ae70 0x7fd45406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:35:56.814 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:56.798+0000 7fd46d7fa700 1 -- 172.21.15.67:0/643149141 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 4471+0+0 (secure 0 0 0) 0x7fd4640890d0 con 0x7fd4700b4290 2023-10-14T10:35:56.814 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:56.798+0000 7fd46ffff700 1 --2- 172.21.15.67:0/643149141 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fd45406ae70 0x7fd45406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:35:56.814 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:56.799+0000 7fd46ffff700 1 --2- 172.21.15.67:0/643149141 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fd45406ae70 0x7fd45406d330 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd4700b2e80 tx=0x7fd45c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:35:56.815 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:56.803+0000 7fd46d7fa700 1 -- 172.21.15.67:0/643149141 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd464054730 con 0x7fd4700b4290 2023-10-14T10:35:57.004 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:57.002+0000 7fd476b8b700 1 -- 172.21.15.67:0/643149141 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd470074d70 con 0x7fd4700b4290 2023-10-14T10:35:57.005 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:57.003+0000 7fd46d7fa700 1 -- 172.21.15.67:0/643149141 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) v1 ==== 74+0+10860 (secure 0 0 0) 0x7fd464057d50 con 0x7fd4700b4290 2023-10-14T10:35:57.005 INFO:teuthology.misc.health.smithi067.stdout: 2023-10-14T10:35:57.005 INFO:teuthology.misc.health.smithi067.stdout:{"epoch":14,"fsid":"37f9ab46-3082-4b71-8b24-08d39897b008","created":"2023-10-14T10:35:46.067554+0000","modified":"2023-10-14T10:35:56.668142+0000","last_up_change":"2023-10-14T10:35:56.668142+0000","last_in_change":"2023-10-14T10:35:48.551490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"2b59a07e-11ef-440c-b766-9f64ac885838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6801","nonce":1287255499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6803","nonce":1287255499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6807","nonce":1287255499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6805","nonce":1287255499}]},"public_addr":"172.21.15.67:6801/1287255499","cluster_addr":"172.21.15.67:6803/1287255499","heartbeat_back_addr":"172.21.15.67:6807/1287255499","heartbeat_front_addr":"172.21.15.67:6805/1287255499","state":["exists","up"]},{"osd":1,"uuid":"6f813bd6-fa0b-487d-a204-431eb1276a60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6818","nonce":2435538787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6822","nonce":2435538787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6830","nonce":2435538787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6826","nonce":2435538787}]},"public_addr":"172.21.15.67:6818/2435538787","cluster_addr":"172.21.15.67:6822/2435538787","heartbeat_back_addr":"172.21.15.67:6830/2435538787","heartbeat_front_addr":"172.21.15.67:6826/2435538787","state":["exists","up"]},{"osd":2,"uuid":"6af42c6a-1b59-4669-9c04-b6876788446d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6809","nonce":2827491791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6811","nonce":2827491791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6815","nonce":2827491791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6813","nonce":2827491791}]},"public_addr":"172.21.15.67:6809/2827491791","cluster_addr":"172.21.15.67:6811/2827491791","heartbeat_back_addr":"172.21.15.67:6815/2827491791","heartbeat_front_addr":"172.21.15.67:6813/2827491791","state":["exists","up"]},{"osd":3,"uuid":"1d889d76-a646-499d-bfff-ed5bae5434ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6819","nonce":1848001176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6821","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6823","nonce":1848001176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6831","nonce":1848001176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6825","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6827","nonce":1848001176}]},"public_addr":"172.21.15.67:6819/1848001176","cluster_addr":"172.21.15.67:6823/1848001176","heartbeat_back_addr":"172.21.15.67:6831/1848001176","heartbeat_front_addr":"172.21.15.67:6827/1848001176","state":["exists","up"]},{"osd":4,"uuid":"c3271dff-b437-4fdd-b13b-6e49a35473af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6804","nonce":1891230785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6808","nonce":1891230785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6815","nonce":1891230785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6811","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6813","nonce":1891230785}]},"public_addr":"172.21.15.72:6804/1891230785","cluster_addr":"172.21.15.72:6808/1891230785","heartbeat_back_addr":"172.21.15.72:6815/1891230785","heartbeat_front_addr":"172.21.15.72:6813/1891230785","state":["exists","up"]},{"osd":5,"uuid":"0337bb18-4b68-4eff-b528-a667275e6926","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6801","nonce":569686991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6803","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6805","nonce":569686991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6812","nonce":569686991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6807","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6809","nonce":569686991}]},"public_addr":"172.21.15.72:6801/569686991","cluster_addr":"172.21.15.72:6805/569686991","heartbeat_back_addr":"172.21.15.72:6812/569686991","heartbeat_front_addr":"172.21.15.72:6809/569686991","state":["exists","up"]},{"osd":6,"uuid":"8e92d9cf-0c8c-46eb-89a7-db2023ef1509","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6817","nonce":3767685934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6819","nonce":3767685934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6823","nonce":3767685934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6821","nonce":3767685934}]},"public_addr":"172.21.15.72:6817/3767685934","cluster_addr":"172.21.15.72:6819/3767685934","heartbeat_back_addr":"172.21.15.72:6823/3767685934","heartbeat_front_addr":"172.21.15.72:6821/3767685934","state":["exists","up"]},{"osd":7,"uuid":"3fd6aa7d-50a9-45c9-9561-5902a044a119","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6825","nonce":3567465926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6827","nonce":3567465926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6831","nonce":3567465926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6829","nonce":3567465926}]},"public_addr":"172.21.15.72:6825/3567465926","cluster_addr":"172.21.15.72:6827/3567465926","heartbeat_back_addr":"172.21.15.72:6831/3567465926","heartbeat_front_addr":"172.21.15.72:6829/3567465926","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-14T10:35:57.007 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:57.005+0000 7fd476b8b700 1 -- 172.21.15.67:0/643149141 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fd45406ae70 msgr2=0x7fd45406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:35:57.008 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:57.005+0000 7fd476b8b700 1 --2- 172.21.15.67:0/643149141 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fd45406ae70 0x7fd45406d330 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd4700b2e80 tx=0x7fd45c008040 comp rx=0 tx=0).stop 2023-10-14T10:35:57.008 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:57.005+0000 7fd476b8b700 1 -- 172.21.15.67:0/643149141 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4700b4290 msgr2=0x7fd4700b2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:35:57.008 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:57.005+0000 7fd476b8b700 1 --2- 172.21.15.67:0/643149141 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4700b4290 0x7fd4700b2940 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd46400c750 tx=0x7fd46400ca60 comp rx=0 tx=0).stop 2023-10-14T10:35:57.008 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:57.005+0000 7fd476b8b700 1 -- 172.21.15.67:0/643149141 shutdown_connections 2023-10-14T10:35:57.008 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:57.005+0000 7fd476b8b700 1 --2- 172.21.15.67:0/643149141 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fd45406ae70 0x7fd45406d330 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:57.009 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:57.005+0000 7fd476b8b700 1 --2- 172.21.15.67:0/643149141 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fd47015ece0 0x7fd4700afe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:57.009 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:57.005+0000 7fd476b8b700 1 --2- 172.21.15.67:0/643149141 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4700b4290 0x7fd4700b2940 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:57.009 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:57.005+0000 7fd476b8b700 1 --2- 172.21.15.67:0/643149141 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd470158210 0x7fd470152b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:57.009 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:57.005+0000 7fd476b8b700 1 -- 172.21.15.67:0/643149141 >> 172.21.15.67:0/643149141 conn(0x7fd4700cce30 msgr2=0x7fd47015f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:35:57.010 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:57.005+0000 7fd476b8b700 1 -- 172.21.15.67:0/643149141 shutdown_connections 2023-10-14T10:35:57.010 INFO:teuthology.misc.health.smithi067.stderr:2023-10-14T10:35:57.005+0000 7fd476b8b700 1 -- 172.21.15.67:0/643149141 wait complete. 2023-10-14T10:35:57.019 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-10-14T10:35:57.019 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-10-14T10:35:57.019 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-14T10:35:57.020 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-14T10:35:57.022 INFO:tasks.ceph.mds.a:Started 2023-10-14T10:35:57.022 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-14T10:35:57.023 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-14T10:35:57.025 INFO:tasks.ceph.mds.c:Started 2023-10-14T10:35:57.025 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-14T10:35:57.025 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-14T10:35:57.028 INFO:tasks.ceph.mds.b:Started 2023-10-14T10:35:57.028 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-14T10:35:57.029 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-14T10:35:57.031 INFO:tasks.ceph.mds.d:Started 2023-10-14T10:35:57.031 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-10-14T10:35:57.032 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-14T10:35:57.104 INFO:tasks.ceph.mds.c.smithi067.stdout:starting mds.c at 2023-10-14T10:35:57.104 INFO:tasks.ceph.mds.a.smithi067.stdout:starting mds.a at 2023-10-14T10:35:57.118 INFO:tasks.ceph.mds.b.smithi072.stdout:starting mds.b at 2023-10-14T10:35:57.119 INFO:tasks.ceph.mds.d.smithi072.stdout:starting mds.d at 2023-10-14T10:35:57.164 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.162+0000 7f94c2060700 1 Processor -- start 2023-10-14T10:35:57.165 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.162+0000 7f94c2060700 1 -- start start 2023-10-14T10:35:57.167 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.163+0000 7f94c2060700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f94bc160e30 0x7f94bc0d4200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:35:57.167 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.163+0000 7f94c2060700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f94bc0d08a0 0x7f94bc0ce3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:35:57.167 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.163+0000 7f94c2060700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94bc0b6610 con 0x7f94bc161210 2023-10-14T10:35:57.168 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.163+0000 7f94c2060700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94bc0d1da0 con 0x7f94bc0d08a0 2023-10-14T10:35:57.168 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.163+0000 7f94c2060700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94bc0d2080 con 0x7f94bc160e30 2023-10-14T10:35:57.168 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.163+0000 7f94bbfff700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f94bc0d08a0 0x7f94bc0ce3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:35:57.168 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.163+0000 7f94bbfff700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f94bc0d08a0 0x7f94bc0ce3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43618/0 (socket says 172.21.15.67:43618) 2023-10-14T10:35:57.168 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.163+0000 7f94bbfff700 1 -- 172.21.15.67:0/3715770513 learned_addr learned my addr 172.21.15.67:0/3715770513 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:35:57.169 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.163+0000 7f94bb7fe700 1 --2- 172.21.15.67:0/3715770513 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f94bc160e30 0x7f94bc0d4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:35:57.169 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.163+0000 7f94bbfff700 1 -- 172.21.15.67:0/3715770513 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f94bc160e30 msgr2=0x7f94bc0d4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:35:57.169 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.163+0000 7f94bbfff700 1 --2- 172.21.15.67:0/3715770513 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f94bc160e30 0x7f94bc0d4200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:57.169 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.163+0000 7f94bbfff700 1 -- 172.21.15.67:0/3715770513 >> v1:172.21.15.72:6789/0 conn(0x7f94bc161210 legacy=0x7f94bc0d0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:35:57.169 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.163+0000 7f94bbfff700 1 -- 172.21.15.67:0/3715770513 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94bc0d2360 con 0x7f94bc0d08a0 2023-10-14T10:35:57.170 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.164+0000 7f94bbfff700 1 --2- 172.21.15.67:0/3715770513 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94bc0d08a0 0x7f94bc0ce3d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f94b4009e50 tx=0x7f94b400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b493b233c6f54f0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:35:57.170 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.164+0000 7f94ba7fc700 1 -- 172.21.15.67:0/3715770513 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b400e070 con 0x7f94bc0d08a0 2023-10-14T10:35:57.170 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.164+0000 7f94ba7fc700 1 -- 172.21.15.67:0/3715770513 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94b400b9c0 con 0x7f94bc0d08a0 2023-10-14T10:35:57.170 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.164+0000 7f94ba7fc700 1 -- 172.21.15.67:0/3715770513 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b400bd70 con 0x7f94bc0d08a0 2023-10-14T10:35:57.170 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.164+0000 7f94c2060700 1 -- 172.21.15.67:0/3715770513 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94bc0d08a0 msgr2=0x7f94bc0ce3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:35:57.171 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.164+0000 7f94c2060700 1 --2- 172.21.15.67:0/3715770513 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94bc0d08a0 0x7f94bc0ce3d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f94b4009e50 tx=0x7f94b400b330 comp rx=0 tx=0).stop 2023-10-14T10:35:57.171 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.164+0000 7f94c2060700 1 -- 172.21.15.67:0/3715770513 shutdown_connections 2023-10-14T10:35:57.171 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.164+0000 7f94c2060700 1 --2- 172.21.15.67:0/3715770513 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94bc0d08a0 0x7f94bc0ce3d0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:57.171 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.164+0000 7f94c2060700 1 --2- 172.21.15.67:0/3715770513 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f94bc160e30 0x7f94bc0d4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:57.171 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.164+0000 7f94c2060700 1 -- 172.21.15.67:0/3715770513 >> 172.21.15.67:0/3715770513 conn(0x7f94bc09cb40 msgr2=0x7f94bc0d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:35:57.172 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.164+0000 7f94c2060700 1 -- 172.21.15.67:0/3715770513 shutdown_connections 2023-10-14T10:35:57.172 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.164+0000 7f94c2060700 1 -- 172.21.15.67:0/3715770513 wait complete. 2023-10-14T10:35:57.175 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.166+0000 7f94c2060700 1 Processor -- start 2023-10-14T10:35:57.175 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.166+0000 7f94c2060700 1 -- start start 2023-10-14T10:35:57.175 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.166+0000 7f94c2060700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f94bc0d08a0 0x7f94bc14e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:35:57.176 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.166+0000 7f94c2060700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f94bc160e30 0x7f94bc0ccf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:35:57.176 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.166+0000 7f94c2060700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94bc161210 0x7f94bc0cd4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:35:57.176 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.166+0000 7f94c2060700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94bc15b430 con 0x7f94bc161210 2023-10-14T10:35:57.176 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.166+0000 7f94c2060700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94bc0c8c70 con 0x7f94bc160e30 2023-10-14T10:35:57.176 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.166+0000 7f94c2060700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94bc0cc1c0 con 0x7f94bc0d08a0 2023-10-14T10:35:57.177 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.167+0000 7f94bbfff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94bc161210 0x7f94bc0cd4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:35:57.177 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.167+0000 7f94bbfff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94bc161210 0x7f94bc0cd4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43624/0 (socket says 172.21.15.67:43624) 2023-10-14T10:35:57.177 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.167+0000 7f94bbfff700 1 -- 172.21.15.67:0/3459392220 learned_addr learned my addr 172.21.15.67:0/3459392220 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:35:57.177 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.167+0000 7f94bb7fe700 1 --2- 172.21.15.67:0/3459392220 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f94bc0d08a0 0x7f94bc14e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:35:57.178 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.167+0000 7f94baffd700 1 --2- 172.21.15.67:0/3459392220 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f94bc160e30 0x7f94bc0ccf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:35:57.178 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.167+0000 7f94bbfff700 1 -- 172.21.15.67:0/3459392220 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f94bc0d08a0 msgr2=0x7f94bc14e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:35:57.178 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.167+0000 7f94bbfff700 1 --2- 172.21.15.67:0/3459392220 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f94bc0d08a0 0x7f94bc14e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:57.178 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.167+0000 7f94bbfff700 1 -- 172.21.15.67:0/3459392220 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f94bc160e30 msgr2=0x7f94bc0ccf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:35:57.178 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.167+0000 7f94bbfff700 1 --2- 172.21.15.67:0/3459392220 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f94bc160e30 0x7f94bc0ccf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:57.179 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.167+0000 7f94bbfff700 1 -- 172.21.15.67:0/3459392220 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94b4009b30 con 0x7f94bc161210 2023-10-14T10:35:57.179 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.167+0000 7f94bbfff700 1 --2- 172.21.15.67:0/3459392220 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94bc161210 0x7f94bc0cd4c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f94bc06b470 tx=0x7f94b4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:35:57.179 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.167+0000 7f94b3fff700 1 -- 172.21.15.67:0/3459392220 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b4004800 con 0x7f94bc161210 2023-10-14T10:35:57.179 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.167+0000 7f94b3fff700 1 -- 172.21.15.67:0/3459392220 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94b4004960 con 0x7f94bc161210 2023-10-14T10:35:57.179 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.167+0000 7f94c2060700 1 -- 172.21.15.67:0/3459392220 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94bc0cc4a0 con 0x7f94bc161210 2023-10-14T10:35:57.180 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.167+0000 7f94b3fff700 1 -- 172.21.15.67:0/3459392220 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b401c680 con 0x7f94bc161210 2023-10-14T10:35:57.180 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.167+0000 7f94c2060700 1 -- 172.21.15.67:0/3459392220 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94bc0cca60 con 0x7f94bc161210 2023-10-14T10:35:57.180 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.168+0000 7f94c2060700 1 -- 172.21.15.67:0/3459392220 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94bc067d20 con 0x7f94bc161210 2023-10-14T10:35:57.180 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.169+0000 7f94b3fff700 1 -- 172.21.15.67:0/3459392220 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 4) v1 ==== 89173+0+0 (secure 0 0 0) 0x7f94b401c7e0 con 0x7f94bc161210 2023-10-14T10:35:57.180 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.169+0000 7f94b3fff700 1 --2- 172.21.15.67:0/3459392220 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f949c06ae70 0x7f949c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:35:57.180 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.169+0000 7f94b3fff700 1 -- 172.21.15.67:0/3459392220 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 4471+0+0 (secure 0 0 0) 0x7f94b408aba0 con 0x7f94bc161210 2023-10-14T10:35:57.181 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.169+0000 7f94bb7fe700 1 --2- 172.21.15.67:0/3459392220 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f949c06ae70 0x7f949c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:35:57.181 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.170+0000 7f94bb7fe700 1 --2- 172.21.15.67:0/3459392220 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f949c06ae70 0x7f949c06d330 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f94ac006fd0 tx=0x7f94ac008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:35:57.181 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.174+0000 7f94b3fff700 1 -- 172.21.15.67:0/3459392220 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f94b4056290 con 0x7f94bc161210 2023-10-14T10:35:57.377 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.375+0000 7f94c2060700 1 -- 172.21.15.67:0/3459392220 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f94bc0c6e90 con 0x7f94bc161210 2023-10-14T10:35:57.378 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.376+0000 7f94b3fff700 1 -- 172.21.15.67:0/3459392220 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) v1 ==== 74+0+10860 (secure 0 0 0) 0x7f94b40598b0 con 0x7f94bc161210 2023-10-14T10:35:57.378 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-14T10:35:57.379 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":14,"fsid":"37f9ab46-3082-4b71-8b24-08d39897b008","created":"2023-10-14T10:35:46.067554+0000","modified":"2023-10-14T10:35:56.668142+0000","last_up_change":"2023-10-14T10:35:56.668142+0000","last_in_change":"2023-10-14T10:35:48.551490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"2b59a07e-11ef-440c-b766-9f64ac885838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6801","nonce":1287255499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6803","nonce":1287255499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6807","nonce":1287255499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6805","nonce":1287255499}]},"public_addr":"172.21.15.67:6801/1287255499","cluster_addr":"172.21.15.67:6803/1287255499","heartbeat_back_addr":"172.21.15.67:6807/1287255499","heartbeat_front_addr":"172.21.15.67:6805/1287255499","state":["exists","up"]},{"osd":1,"uuid":"6f813bd6-fa0b-487d-a204-431eb1276a60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6818","nonce":2435538787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6822","nonce":2435538787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6830","nonce":2435538787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6826","nonce":2435538787}]},"public_addr":"172.21.15.67:6818/2435538787","cluster_addr":"172.21.15.67:6822/2435538787","heartbeat_back_addr":"172.21.15.67:6830/2435538787","heartbeat_front_addr":"172.21.15.67:6826/2435538787","state":["exists","up"]},{"osd":2,"uuid":"6af42c6a-1b59-4669-9c04-b6876788446d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6809","nonce":2827491791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6811","nonce":2827491791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6815","nonce":2827491791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6813","nonce":2827491791}]},"public_addr":"172.21.15.67:6809/2827491791","cluster_addr":"172.21.15.67:6811/2827491791","heartbeat_back_addr":"172.21.15.67:6815/2827491791","heartbeat_front_addr":"172.21.15.67:6813/2827491791","state":["exists","up"]},{"osd":3,"uuid":"1d889d76-a646-499d-bfff-ed5bae5434ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6819","nonce":1848001176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6821","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6823","nonce":1848001176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6831","nonce":1848001176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6825","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6827","nonce":1848001176}]},"public_addr":"172.21.15.67:6819/1848001176","cluster_addr":"172.21.15.67:6823/1848001176","heartbeat_back_addr":"172.21.15.67:6831/1848001176","heartbeat_front_addr":"172.21.15.67:6827/1848001176","state":["exists","up"]},{"osd":4,"uuid":"c3271dff-b437-4fdd-b13b-6e49a35473af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6804","nonce":1891230785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6808","nonce":1891230785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6815","nonce":1891230785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6811","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6813","nonce":1891230785}]},"public_addr":"172.21.15.72:6804/1891230785","cluster_addr":"172.21.15.72:6808/1891230785","heartbeat_back_addr":"172.21.15.72:6815/1891230785","heartbeat_front_addr":"172.21.15.72:6813/1891230785","state":["exists","up"]},{"osd":5,"uuid":"0337bb18-4b68-4eff-b528-a667275e6926","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6801","nonce":569686991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6803","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6805","nonce":569686991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6812","nonce":569686991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6807","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6809","nonce":569686991}]},"public_addr":"172.21.15.72:6801/569686991","cluster_addr":"172.21.15.72:6805/569686991","heartbeat_back_addr":"172.21.15.72:6812/569686991","heartbeat_front_addr":"172.21.15.72:6809/569686991","state":["exists","up"]},{"osd":6,"uuid":"8e92d9cf-0c8c-46eb-89a7-db2023ef1509","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6817","nonce":3767685934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6819","nonce":3767685934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6823","nonce":3767685934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6821","nonce":3767685934}]},"public_addr":"172.21.15.72:6817/3767685934","cluster_addr":"172.21.15.72:6819/3767685934","heartbeat_back_addr":"172.21.15.72:6823/3767685934","heartbeat_front_addr":"172.21.15.72:6821/3767685934","state":["exists","up"]},{"osd":7,"uuid":"3fd6aa7d-50a9-45c9-9561-5902a044a119","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6825","nonce":3567465926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6827","nonce":3567465926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6831","nonce":3567465926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6829","nonce":3567465926}]},"public_addr":"172.21.15.72:6825/3567465926","cluster_addr":"172.21.15.72:6827/3567465926","heartbeat_back_addr":"172.21.15.72:6831/3567465926","heartbeat_front_addr":"172.21.15.72:6829/3567465926","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-14T10:35:57.381 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.378+0000 7f94c2060700 1 -- 172.21.15.67:0/3459392220 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f949c06ae70 msgr2=0x7f949c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:35:57.381 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.378+0000 7f94c2060700 1 --2- 172.21.15.67:0/3459392220 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f949c06ae70 0x7f949c06d330 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f94ac006fd0 tx=0x7f94ac008040 comp rx=0 tx=0).stop 2023-10-14T10:35:57.381 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.378+0000 7f94c2060700 1 -- 172.21.15.67:0/3459392220 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94bc161210 msgr2=0x7f94bc0cd4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:35:57.381 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.378+0000 7f94c2060700 1 --2- 172.21.15.67:0/3459392220 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94bc161210 0x7f94bc0cd4c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f94bc06b470 tx=0x7f94b4004ad0 comp rx=0 tx=0).stop 2023-10-14T10:35:57.381 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.378+0000 7f94c2060700 1 -- 172.21.15.67:0/3459392220 shutdown_connections 2023-10-14T10:35:57.382 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.378+0000 7f94c2060700 1 --2- 172.21.15.67:0/3459392220 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f949c06ae70 0x7f949c06d330 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:57.382 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.378+0000 7f94c2060700 1 --2- 172.21.15.67:0/3459392220 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94bc161210 0x7f94bc0cd4c0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:57.382 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.378+0000 7f94c2060700 1 --2- 172.21.15.67:0/3459392220 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f94bc0d08a0 0x7f94bc14e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:57.382 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.378+0000 7f94c2060700 1 --2- 172.21.15.67:0/3459392220 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f94bc160e30 0x7f94bc0ccf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:57.382 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.378+0000 7f94c2060700 1 -- 172.21.15.67:0/3459392220 >> 172.21.15.67:0/3459392220 conn(0x7f94bc09cb40 msgr2=0x7f94bc160080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:35:57.383 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.378+0000 7f94c2060700 1 -- 172.21.15.67:0/3459392220 shutdown_connections 2023-10-14T10:35:57.383 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.378+0000 7f94c2060700 1 -- 172.21.15.67:0/3459392220 wait complete. 2023-10-14T10:35:57.392 INFO:tasks.cephfs.filesystem.ceph_manager:[] 2023-10-14T10:35:57.393 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-14T10:35:57.519 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.516+0000 7fac75ec7700 1 Processor -- start 2023-10-14T10:35:57.519 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.516+0000 7fac75ec7700 1 -- start start 2023-10-14T10:35:57.521 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.517+0000 7fac75ec7700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fac700cfe30 0x7fac700cbf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:35:57.521 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.517+0000 7fac75ec7700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac7015e7a0 con 0x7fac7015f1f0 2023-10-14T10:35:57.521 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.517+0000 7fac75ec7700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac700b29a0 con 0x7fac7015ab10 2023-10-14T10:35:57.521 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.517+0000 7fac75ec7700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac700b2c80 con 0x7fac700cfe30 2023-10-14T10:35:57.522 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.517+0000 7fac6ffff700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fac700cfe30 0x7fac700cbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:35:57.522 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.517+0000 7fac6effd700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fac7015f1f0 0x7fac700cf720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:51414/0 (socket says 172.21.15.67:51414) 2023-10-14T10:35:57.522 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.517+0000 7fac6effd700 1 -- 172.21.15.67:0/4145735177 learned_addr learned my addr 172.21.15.67:0/4145735177 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:35:57.522 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.518+0000 7fac6ffff700 1 -- 172.21.15.67:0/4145735177 >> v1:172.21.15.72:6789/0 conn(0x7fac7015ab10 legacy=0x7fac700b1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:35:57.523 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.518+0000 7fac6ffff700 1 -- 172.21.15.67:0/4145735177 >> v1:172.21.15.67:6789/0 conn(0x7fac7015f1f0 legacy=0x7fac700cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:35:57.523 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.518+0000 7fac6ffff700 1 -- 172.21.15.67:0/4145735177 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac7015e9f0 con 0x7fac700cfe30 2023-10-14T10:35:57.523 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.518+0000 7fac6ffff700 1 --2- 172.21.15.67:0/4145735177 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fac700cfe30 0x7fac700cbf40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fac68009d90 tx=0x7fac6800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f18955e718ce724 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:35:57.523 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.518+0000 7fac6e7fc700 1 -- 172.21.15.67:0/4145735177 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac6800e070 con 0x7fac700cfe30 2023-10-14T10:35:57.523 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.518+0000 7fac6e7fc700 1 -- 172.21.15.67:0/4145735177 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac6800bbd0 con 0x7fac700cfe30 2023-10-14T10:35:57.524 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.518+0000 7fac6e7fc700 1 -- 172.21.15.67:0/4145735177 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac68012430 con 0x7fac700cfe30 2023-10-14T10:35:57.524 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.518+0000 7fac75ec7700 1 -- 172.21.15.67:0/4145735177 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fac700cfe30 msgr2=0x7fac700cbf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:35:57.524 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.518+0000 7fac75ec7700 1 --2- 172.21.15.67:0/4145735177 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fac700cfe30 0x7fac700cbf40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fac68009d90 tx=0x7fac6800b330 comp rx=0 tx=0).stop 2023-10-14T10:35:57.524 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.519+0000 7fac75ec7700 1 -- 172.21.15.67:0/4145735177 shutdown_connections 2023-10-14T10:35:57.524 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.519+0000 7fac75ec7700 1 --2- 172.21.15.67:0/4145735177 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fac700cfe30 0x7fac700cbf40 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:57.524 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.519+0000 7fac75ec7700 1 -- 172.21.15.67:0/4145735177 >> 172.21.15.67:0/4145735177 conn(0x7fac7006dae0 msgr2=0x7fac7006def0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:35:57.525 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.519+0000 7fac75ec7700 1 -- 172.21.15.67:0/4145735177 shutdown_connections 2023-10-14T10:35:57.525 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.519+0000 7fac75ec7700 1 -- 172.21.15.67:0/4145735177 wait complete. 2023-10-14T10:35:57.528 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.520+0000 7fac75ec7700 1 Processor -- start 2023-10-14T10:35:57.528 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.520+0000 7fac75ec7700 1 -- start start 2023-10-14T10:35:57.528 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.521+0000 7fac75ec7700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fac700cfe30 0x7fac7014c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:35:57.528 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.521+0000 7fac75ec7700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fac7015ab10 0x7fac700b1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:35:57.529 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.521+0000 7fac75ec7700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fac7015f1f0 0x7fac700b1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:35:57.529 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.521+0000 7fac75ec7700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac700c3fe0 con 0x7fac700cfe30 2023-10-14T10:35:57.529 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.521+0000 7fac75ec7700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac700c4b60 con 0x7fac7015f1f0 2023-10-14T10:35:57.529 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.521+0000 7fac75ec7700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac700c4ff0 con 0x7fac7015ab10 2023-10-14T10:35:57.529 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.521+0000 7fac6f7fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fac700cfe30 0x7fac7014c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:35:57.530 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.521+0000 7fac6f7fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fac700cfe30 0x7fac7014c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43650/0 (socket says 172.21.15.67:43650) 2023-10-14T10:35:57.530 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.521+0000 7fac6f7fe700 1 -- 172.21.15.67:0/2426542321 learned_addr learned my addr 172.21.15.67:0/2426542321 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:35:57.530 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.521+0000 7fac6ffff700 1 --2- 172.21.15.67:0/2426542321 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fac7015f1f0 0x7fac700b1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:35:57.530 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.521+0000 7fac6effd700 1 --2- 172.21.15.67:0/2426542321 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fac7015ab10 0x7fac700b1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:35:57.530 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.521+0000 7fac6f7fe700 1 -- 172.21.15.67:0/2426542321 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fac7015ab10 msgr2=0x7fac700b1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:35:57.531 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.521+0000 7fac6f7fe700 1 --2- 172.21.15.67:0/2426542321 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fac7015ab10 0x7fac700b1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:57.531 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.521+0000 7fac6f7fe700 1 -- 172.21.15.67:0/2426542321 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fac7015f1f0 msgr2=0x7fac700b1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:35:57.531 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.521+0000 7fac6f7fe700 1 --2- 172.21.15.67:0/2426542321 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fac7015f1f0 0x7fac700b1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:57.531 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.521+0000 7fac6f7fe700 1 -- 172.21.15.67:0/2426542321 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac68009a70 con 0x7fac700cfe30 2023-10-14T10:35:57.531 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.521+0000 7fac6f7fe700 1 --2- 172.21.15.67:0/2426542321 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fac700cfe30 0x7fac7014c800 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fac6000c510 tx=0x7fac6000c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:35:57.532 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.521+0000 7fac57fff700 1 -- 172.21.15.67:0/2426542321 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac600052d0 con 0x7fac700cfe30 2023-10-14T10:35:57.532 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.521+0000 7fac57fff700 1 -- 172.21.15.67:0/2426542321 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac60005430 con 0x7fac700cfe30 2023-10-14T10:35:57.532 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.521+0000 7fac75ec7700 1 -- 172.21.15.67:0/2426542321 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac700c4230 con 0x7fac700cfe30 2023-10-14T10:35:57.532 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.521+0000 7fac57fff700 1 -- 172.21.15.67:0/2426542321 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac600038c0 con 0x7fac700cfe30 2023-10-14T10:35:57.532 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.522+0000 7fac75ec7700 1 -- 172.21.15.67:0/2426542321 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac700c47f0 con 0x7fac700cfe30 2023-10-14T10:35:57.532 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.522+0000 7fac75ec7700 1 -- 172.21.15.67:0/2426542321 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac700c91a0 con 0x7fac700cfe30 2023-10-14T10:35:57.533 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.523+0000 7fac57fff700 1 -- 172.21.15.67:0/2426542321 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 4) v1 ==== 89173+0+0 (secure 0 0 0) 0x7fac60003a20 con 0x7fac700cfe30 2023-10-14T10:35:57.533 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.523+0000 7fac57fff700 1 --2- 172.21.15.67:0/2426542321 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fac5c06aea0 0x7fac5c06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:35:57.533 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.523+0000 7fac57fff700 1 -- 172.21.15.67:0/2426542321 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 4471+0+0 (secure 0 0 0) 0x7fac60088380 con 0x7fac700cfe30 2023-10-14T10:35:57.533 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.523+0000 7fac6effd700 1 --2- 172.21.15.67:0/2426542321 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fac5c06aea0 0x7fac5c06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:35:57.533 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.524+0000 7fac6effd700 1 --2- 172.21.15.67:0/2426542321 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fac5c06aea0 0x7fac5c06d360 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fac70159400 tx=0x7fac64009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:35:57.534 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.528+0000 7fac57fff700 1 -- 172.21.15.67:0/2426542321 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fac600539e0 con 0x7fac700cfe30 2023-10-14T10:35:57.778 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.776+0000 7fac75ec7700 1 -- 172.21.15.67:0/2426542321 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fac700c72b0 con 0x7fac700cfe30 2023-10-14T10:35:57.778 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.776+0000 7fac57fff700 1 -- 172.21.15.67:0/2426542321 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1324 (secure 0 0 0) 0x7fac60017540 con 0x7fac700cfe30 2023-10-14T10:35:57.778 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-14T10:35:57.779 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6834/3936231765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":3936231765},{"type":"v1","addr":"172.21.15.67:6834","nonce":3936231765}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-10-14T10:35:57.781 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.778+0000 7fac75ec7700 1 -- 172.21.15.67:0/2426542321 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fac5c06aea0 msgr2=0x7fac5c06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:35:57.781 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.778+0000 7fac75ec7700 1 --2- 172.21.15.67:0/2426542321 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fac5c06aea0 0x7fac5c06d360 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fac70159400 tx=0x7fac64009040 comp rx=0 tx=0).stop 2023-10-14T10:35:57.781 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.778+0000 7fac75ec7700 1 -- 172.21.15.67:0/2426542321 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fac700cfe30 msgr2=0x7fac7014c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:35:57.781 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.778+0000 7fac75ec7700 1 --2- 172.21.15.67:0/2426542321 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fac700cfe30 0x7fac7014c800 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fac6000c510 tx=0x7fac6000c820 comp rx=0 tx=0).stop 2023-10-14T10:35:57.782 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.778+0000 7fac75ec7700 1 -- 172.21.15.67:0/2426542321 shutdown_connections 2023-10-14T10:35:57.782 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.778+0000 7fac75ec7700 1 --2- 172.21.15.67:0/2426542321 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fac5c06aea0 0x7fac5c06d360 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:57.782 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.778+0000 7fac75ec7700 1 --2- 172.21.15.67:0/2426542321 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fac7015f1f0 0x7fac700b1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:57.782 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.778+0000 7fac75ec7700 1 --2- 172.21.15.67:0/2426542321 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fac700cfe30 0x7fac7014c800 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:57.782 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.778+0000 7fac75ec7700 1 --2- 172.21.15.67:0/2426542321 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fac7015ab10 0x7fac700b1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:57.783 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.778+0000 7fac75ec7700 1 -- 172.21.15.67:0/2426542321 >> 172.21.15.67:0/2426542321 conn(0x7fac7006dae0 msgr2=0x7fac700b3030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:35:57.783 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.778+0000 7fac75ec7700 1 -- 172.21.15.67:0/2426542321 shutdown_connections 2023-10-14T10:35:57.783 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:57.778+0000 7fac75ec7700 1 -- 172.21.15.67:0/2426542321 wait complete. 2023-10-14T10:35:57.783 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 2 2023-10-14T10:35:59.793 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-14T10:35:59.932 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:59.930+0000 7f663aec1700 1 Processor -- start 2023-10-14T10:35:59.932 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:59.930+0000 7f663aec1700 1 -- start start 2023-10-14T10:35:59.935 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:59.931+0000 7f663aec1700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f663415e2f0 0x7f66340cede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:35:59.935 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:59.931+0000 7f663aec1700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f663406a840 0x7f663406ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:35:59.935 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:59.931+0000 7f663aec1700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f663406b1a0 con 0x7f66340cd950 2023-10-14T10:35:59.935 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:59.931+0000 7f663aec1700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66341619a0 con 0x7f663415e2f0 2023-10-14T10:35:59.935 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:59.931+0000 7f663aec1700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66340b2d60 con 0x7f663406a840 2023-10-14T10:35:59.936 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:59.931+0000 7f6633fff700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f663415e2f0 0x7f66340cede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:35:59.936 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:59.931+0000 7f6638c5d700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f66340cd950 0x7f663415f570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:51446/0 (socket says 172.21.15.67:51446) 2023-10-14T10:35:59.936 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:59.931+0000 7f6638c5d700 1 -- 172.21.15.67:0/2253868972 learned_addr learned my addr 172.21.15.67:0/2253868972 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:35:59.936 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:59.931+0000 7f663945e700 1 --2- 172.21.15.67:0/2253868972 >> v2:172.21.15.72:3300/0 conn(0x7f663406a840 0x7f663406ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:35:59.936 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:59.931+0000 7f66337fe700 1 -- 172.21.15.67:0/2253868972 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4010421684 0 0) 0x7f663406b1a0 con 0x7f66340cd950 2023-10-14T10:35:59.937 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:59.931+0000 7f66337fe700 1 -- 172.21.15.67:0/2253868972 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66240035b0 con 0x7f66340cd950 2023-10-14T10:35:59.937 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:59.931+0000 7f6633fff700 1 -- 172.21.15.67:0/2253868972 >> v2:172.21.15.72:3300/0 conn(0x7f663406a840 msgr2=0x7f663406ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:35:59.937 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:59.931+0000 7f6633fff700 1 --2- 172.21.15.67:0/2253868972 >> v2:172.21.15.72:3300/0 conn(0x7f663406a840 0x7f663406ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:59.937 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:59.931+0000 7f6633fff700 1 -- 172.21.15.67:0/2253868972 >> v1:172.21.15.67:6789/0 conn(0x7f66340cd950 legacy=0x7f663415f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:35:59.937 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:59.931+0000 7f6633fff700 1 -- 172.21.15.67:0/2253868972 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66340cdd30 con 0x7f663415e2f0 2023-10-14T10:35:59.937 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:59.931+0000 7f6633fff700 1 --2- 172.21.15.67:0/2253868972 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f663415e2f0 0x7f66340cede0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f66340afed0 tx=0x7f6620009b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b0e190108df1ab1 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:35:59.938 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:59.932+0000 7f66337fe700 1 -- 172.21.15.67:0/2253868972 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6620005e30 con 0x7f663415e2f0 2023-10-14T10:35:59.938 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:59.932+0000 7f66337fe700 1 -- 172.21.15.67:0/2253868972 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6620006030 con 0x7f663415e2f0 2023-10-14T10:35:59.938 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:59.932+0000 7f66337fe700 1 -- 172.21.15.67:0/2253868972 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66200063c0 con 0x7f663415e2f0 2023-10-14T10:35:59.938 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:59.932+0000 7f663aec1700 1 -- 172.21.15.67:0/2253868972 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f663415e2f0 msgr2=0x7f66340cede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:35:59.939 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:59.932+0000 7f663aec1700 1 --2- 172.21.15.67:0/2253868972 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f663415e2f0 0x7f66340cede0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f66340afed0 tx=0x7f6620009b00 comp rx=0 tx=0).stop 2023-10-14T10:35:59.939 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:59.932+0000 7f663aec1700 1 -- 172.21.15.67:0/2253868972 shutdown_connections 2023-10-14T10:35:59.939 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:59.932+0000 7f663aec1700 1 --2- 172.21.15.67:0/2253868972 >> v2:172.21.15.72:3300/0 conn(0x7f663406a840 0x7f663406ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:59.941 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:59.932+0000 7f663aec1700 1 --2- 172.21.15.67:0/2253868972 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f663415e2f0 0x7f66340cede0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:59.941 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:59.932+0000 7f663aec1700 1 -- 172.21.15.67:0/2253868972 >> 172.21.15.67:0/2253868972 conn(0x7f663406dae0 msgr2=0x7f663406def0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:35:59.941 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:59.932+0000 7f663aec1700 1 -- 172.21.15.67:0/2253868972 shutdown_connections 2023-10-14T10:35:59.942 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:59.932+0000 7f663aec1700 1 -- 172.21.15.67:0/2253868972 wait complete. 2023-10-14T10:35:59.955 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:59.933+0000 7f663aec1700 1 Processor -- start 2023-10-14T10:35:59.956 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:59.933+0000 7f663aec1700 1 -- start start 2023-10-14T10:35:59.956 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:59.934+0000 7f663aec1700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f663406a840 0x7f66340c7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:35:59.957 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:59.934+0000 7f663aec1700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f66340cd950 0x7f66340c9f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:35:59.957 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:59.934+0000 7f663aec1700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f663415e2f0 0x7f66340aca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:35:59.957 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:59.934+0000 7f663aec1700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66340c4000 con 0x7f66340cd950 2023-10-14T10:35:59.958 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:59.934+0000 7f663aec1700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66340ca440 con 0x7f663415e2f0 2023-10-14T10:35:59.958 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:59.934+0000 7f663aec1700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66340c4b10 con 0x7f663406a840 2023-10-14T10:35:59.958 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:59.934+0000 7f6633fff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f66340cd950 0x7f66340c9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:35:59.959 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:59.934+0000 7f6633fff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f66340cd950 0x7f66340c9f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43686/0 (socket says 172.21.15.67:43686) 2023-10-14T10:35:59.959 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:59.934+0000 7f6633fff700 1 -- 172.21.15.67:0/1087984707 learned_addr learned my addr 172.21.15.67:0/1087984707 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:35:59.960 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:59.935+0000 7f663945e700 1 --2- 172.21.15.67:0/1087984707 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f663415e2f0 0x7f66340aca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:35:59.960 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:59.935+0000 7f6638c5d700 1 --2- 172.21.15.67:0/1087984707 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f663406a840 0x7f66340c7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:35:59.960 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:59.935+0000 7f6633fff700 1 -- 172.21.15.67:0/1087984707 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f663406a840 msgr2=0x7f66340c7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:35:59.961 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:59.935+0000 7f6633fff700 1 --2- 172.21.15.67:0/1087984707 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f663406a840 0x7f66340c7fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:59.961 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:59.935+0000 7f6633fff700 1 -- 172.21.15.67:0/1087984707 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f663415e2f0 msgr2=0x7f66340aca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:35:59.962 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:59.935+0000 7f6633fff700 1 --2- 172.21.15.67:0/1087984707 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f663415e2f0 0x7f66340aca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:35:59.962 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:59.935+0000 7f6633fff700 1 -- 172.21.15.67:0/1087984707 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6620009430 con 0x7f66340cd950 2023-10-14T10:35:59.962 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:59.935+0000 7f663945e700 1 --2- 172.21.15.67:0/1087984707 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f663415e2f0 0x7f66340aca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-14T10:35:59.963 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:59.935+0000 7f6633fff700 1 --2- 172.21.15.67:0/1087984707 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f66340cd950 0x7f66340c9f00 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6620000c00 tx=0x7f6620004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:35:59.963 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:59.935+0000 7f66317fa700 1 -- 172.21.15.67:0/1087984707 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6620006970 con 0x7f66340cd950 2023-10-14T10:35:59.963 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:59.935+0000 7f66317fa700 1 -- 172.21.15.67:0/1087984707 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6620020070 con 0x7f66340cd950 2023-10-14T10:35:59.964 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:59.935+0000 7f66317fa700 1 -- 172.21.15.67:0/1087984707 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6620004260 con 0x7f66340cd950 2023-10-14T10:35:59.964 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:59.935+0000 7f663aec1700 1 -- 172.21.15.67:0/1087984707 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66340acf90 con 0x7f66340cd950 2023-10-14T10:35:59.965 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:59.935+0000 7f663aec1700 1 -- 172.21.15.67:0/1087984707 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66340c44c0 con 0x7f66340cd950 2023-10-14T10:35:59.965 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:59.936+0000 7f663aec1700 1 -- 172.21.15.67:0/1087984707 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6634067d20 con 0x7f66340cd950 2023-10-14T10:35:59.965 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:59.937+0000 7f66317fa700 1 -- 172.21.15.67:0/1087984707 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 4) v1 ==== 89173+0+0 (secure 0 0 0) 0x7f6620022030 con 0x7f66340cd950 2023-10-14T10:35:59.966 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:59.937+0000 7f66317fa700 1 --2- 172.21.15.67:0/1087984707 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f661806ae70 0x7f661806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:35:59.966 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:59.938+0000 7f66317fa700 1 -- 172.21.15.67:0/1087984707 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f662008aa70 con 0x7f66340cd950 2023-10-14T10:35:59.967 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:59.938+0000 7f6638c5d700 1 --2- 172.21.15.67:0/1087984707 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f661806ae70 0x7f661806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:35:59.967 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:59.938+0000 7f6638c5d700 1 --2- 172.21.15.67:0/1087984707 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f661806ae70 0x7f661806d330 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f66280011e0 tx=0x7f6628009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:35:59.967 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:35:59.942+0000 7f66317fa700 1 -- 172.21.15.67:0/1087984707 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6620056030 con 0x7f66340cd950 2023-10-14T10:36:00.195 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.193+0000 7f663aec1700 1 -- 172.21.15.67:0/1087984707 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f66340b2630 con 0x7f66340cd950 2023-10-14T10:36:00.196 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.194+0000 7f66317fa700 1 -- 172.21.15.67:0/1087984707 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3688 (secure 0 0 0) 0x7f6620059650 con 0x7f66340cd950 2023-10-14T10:36:00.197 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-14T10:36:00.197 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4217,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6836/838298018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":838298018},{"type":"v1","addr":"172.21.15.72:6836","nonce":838298018}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4255,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/1687192726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6833","nonce":1687192726},{"type":"v1","addr":"172.21.15.67:6835","nonce":1687192726}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6834/3936231765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":3936231765},{"type":"v1","addr":"172.21.15.67:6834","nonce":3936231765}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4267,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/757500960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6835","nonce":757500960},{"type":"v1","addr":"172.21.15.72:6837","nonce":757500960}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2023-10-14T10:36:00.199 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.196+0000 7f663aec1700 1 -- 172.21.15.67:0/1087984707 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f661806ae70 msgr2=0x7f661806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:00.199 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.196+0000 7f663aec1700 1 --2- 172.21.15.67:0/1087984707 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f661806ae70 0x7f661806d330 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f66280011e0 tx=0x7f6628009040 comp rx=0 tx=0).stop 2023-10-14T10:36:00.200 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.196+0000 7f663aec1700 1 -- 172.21.15.67:0/1087984707 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f66340cd950 msgr2=0x7f66340c9f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:00.200 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.196+0000 7f663aec1700 1 --2- 172.21.15.67:0/1087984707 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f66340cd950 0x7f66340c9f00 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6620000c00 tx=0x7f6620004750 comp rx=0 tx=0).stop 2023-10-14T10:36:00.200 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.196+0000 7f663aec1700 1 -- 172.21.15.67:0/1087984707 shutdown_connections 2023-10-14T10:36:00.200 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.196+0000 7f663aec1700 1 --2- 172.21.15.67:0/1087984707 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f661806ae70 0x7f661806d330 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:00.200 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.196+0000 7f663aec1700 1 --2- 172.21.15.67:0/1087984707 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f663415e2f0 0x7f66340aca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:00.201 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.196+0000 7f663aec1700 1 --2- 172.21.15.67:0/1087984707 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f663406a840 0x7f66340c7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:00.201 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.196+0000 7f663aec1700 1 --2- 172.21.15.67:0/1087984707 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f66340cd950 0x7f66340c9f00 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:00.201 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.196+0000 7f663aec1700 1 -- 172.21.15.67:0/1087984707 >> 172.21.15.67:0/1087984707 conn(0x7f663406dae0 msgr2=0x7f663415da80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:00.201 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.196+0000 7f663aec1700 1 -- 172.21.15.67:0/1087984707 shutdown_connections 2023-10-14T10:36:00.202 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.197+0000 7f663aec1700 1 -- 172.21.15.67:0/1087984707 wait complete. 2023-10-14T10:36:00.202 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 3 2023-10-14T10:36:00.212 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-14T10:36:00.338 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.336+0000 7f76ff62b700 1 Processor -- start 2023-10-14T10:36:00.338 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.336+0000 7f76ff62b700 1 -- start start 2023-10-14T10:36:00.341 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.337+0000 7f76ff62b700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f76f80cd950 0x7f76f815e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:00.341 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.337+0000 7f76ff62b700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f76f80cede0 0x7f76f815f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:00.341 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.337+0000 7f76ff62b700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76f80cf290 con 0x7f76f806a840 2023-10-14T10:36:00.341 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.337+0000 7f76ff62b700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76f81619a0 con 0x7f76f80cede0 2023-10-14T10:36:00.342 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.337+0000 7f76ff62b700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76f80b2d60 con 0x7f76f80cd950 2023-10-14T10:36:00.342 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.337+0000 7f76fcbc6700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f76f80cede0 0x7f76f815f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:00.342 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.337+0000 7f76fcbc6700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f76f80cede0 0x7f76f815f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43694/0 (socket says 172.21.15.67:43694) 2023-10-14T10:36:00.342 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.337+0000 7f76fcbc6700 1 -- 172.21.15.67:0/2204093800 learned_addr learned my addr 172.21.15.67:0/2204093800 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:00.342 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.337+0000 7f76fd3c7700 1 --2- 172.21.15.67:0/2204093800 >> v2:172.21.15.72:3300/0 conn(0x7f76f80cd950 0x7f76f815e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:00.343 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.337+0000 7f76fcbc6700 1 -- 172.21.15.67:0/2204093800 >> v2:172.21.15.72:3300/0 conn(0x7f76f80cd950 msgr2=0x7f76f815e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:00.343 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.337+0000 7f76fcbc6700 1 --2- 172.21.15.67:0/2204093800 >> v2:172.21.15.72:3300/0 conn(0x7f76f80cd950 0x7f76f815e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:00.343 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.337+0000 7f76fcbc6700 1 -- 172.21.15.67:0/2204093800 >> v1:172.21.15.72:6789/0 conn(0x7f76f806a840 legacy=0x7f76f8161df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:00.343 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.337+0000 7f76fcbc6700 1 -- 172.21.15.67:0/2204093800 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76f80cdd30 con 0x7f76f80cede0 2023-10-14T10:36:00.343 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.338+0000 7f76fcbc6700 1 --2- 172.21.15.67:0/2204093800 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76f80cede0 0x7f76f815f570 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f76f8161ae0 tx=0x7f76f4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddda720be9848f86 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:00.344 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.338+0000 7f76effff700 1 -- 172.21.15.67:0/2204093800 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f4005c20 con 0x7f76f80cede0 2023-10-14T10:36:00.344 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.338+0000 7f76effff700 1 -- 172.21.15.67:0/2204093800 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76f4005da0 con 0x7f76f80cede0 2023-10-14T10:36:00.344 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.338+0000 7f76effff700 1 -- 172.21.15.67:0/2204093800 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f4004270 con 0x7f76f80cede0 2023-10-14T10:36:00.344 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.338+0000 7f76ff62b700 1 -- 172.21.15.67:0/2204093800 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76f80cede0 msgr2=0x7f76f815f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:00.344 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.338+0000 7f76ff62b700 1 --2- 172.21.15.67:0/2204093800 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76f80cede0 0x7f76f815f570 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f76f8161ae0 tx=0x7f76f4009bd0 comp rx=0 tx=0).stop 2023-10-14T10:36:00.345 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.338+0000 7f76ff62b700 1 -- 172.21.15.67:0/2204093800 shutdown_connections 2023-10-14T10:36:00.345 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.338+0000 7f76ff62b700 1 --2- 172.21.15.67:0/2204093800 >> v2:172.21.15.72:3300/0 conn(0x7f76f80cd950 0x7f76f815e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:00.345 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.338+0000 7f76ff62b700 1 --2- 172.21.15.67:0/2204093800 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76f80cede0 0x7f76f815f570 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:00.345 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.338+0000 7f76ff62b700 1 -- 172.21.15.67:0/2204093800 >> 172.21.15.67:0/2204093800 conn(0x7f76f806dae0 msgr2=0x7f76f806def0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:00.345 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.338+0000 7f76ff62b700 1 -- 172.21.15.67:0/2204093800 shutdown_connections 2023-10-14T10:36:00.346 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.338+0000 7f76ff62b700 1 -- 172.21.15.67:0/2204093800 wait complete. 2023-10-14T10:36:00.348 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.340+0000 7f76ff62b700 1 Processor -- start 2023-10-14T10:36:00.349 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.340+0000 7f76ff62b700 1 -- start start 2023-10-14T10:36:00.349 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.341+0000 7f76ff62b700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f76f806a840 0x7f76f80b49f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:00.349 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.341+0000 7f76ff62b700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76f80cd950 0x7f76f80c5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:00.349 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.341+0000 7f76ff62b700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f76f80cede0 0x7f76f81f3390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:00.349 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.341+0000 7f76ff62b700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76f81f38d0 con 0x7f76f80cd950 2023-10-14T10:36:00.350 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.341+0000 7f76ff62b700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76f81f3b60 con 0x7f76f806a840 2023-10-14T10:36:00.350 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.341+0000 7f76ff62b700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76f80b5040 con 0x7f76f80cede0 2023-10-14T10:36:00.350 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.341+0000 7f76fcbc6700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76f80cd950 0x7f76f80c5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:00.350 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.341+0000 7f76fcbc6700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76f80cd950 0x7f76f80c5a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43700/0 (socket says 172.21.15.67:43700) 2023-10-14T10:36:00.350 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.341+0000 7f76fcbc6700 1 -- 172.21.15.67:0/1305321899 learned_addr learned my addr 172.21.15.67:0/1305321899 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:00.351 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.341+0000 7f76fd3c7700 1 --2- 172.21.15.67:0/1305321899 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f76f806a840 0x7f76f80b49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:00.351 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.341+0000 7f76fdbc8700 1 --2- 172.21.15.67:0/1305321899 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f76f80cede0 0x7f76f81f3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:00.351 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.341+0000 7f76fcbc6700 1 -- 172.21.15.67:0/1305321899 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f76f80cede0 msgr2=0x7f76f81f3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:00.351 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.341+0000 7f76fcbc6700 1 --2- 172.21.15.67:0/1305321899 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f76f80cede0 0x7f76f81f3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:00.351 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.341+0000 7f76fcbc6700 1 -- 172.21.15.67:0/1305321899 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f76f806a840 msgr2=0x7f76f80b49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:00.352 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.341+0000 7f76fcbc6700 1 --2- 172.21.15.67:0/1305321899 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f76f806a840 0x7f76f80b49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:00.352 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.341+0000 7f76fcbc6700 1 -- 172.21.15.67:0/1305321899 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76f4009500 con 0x7f76f80cd950 2023-10-14T10:36:00.352 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.341+0000 7f76fcbc6700 1 --2- 172.21.15.67:0/1305321899 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76f80cd950 0x7f76f80c5a60 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f76f80cf420 tx=0x7f76f4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:00.352 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.342+0000 7f76edffb700 1 -- 172.21.15.67:0/1305321899 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f40038a0 con 0x7f76f80cd950 2023-10-14T10:36:00.352 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.342+0000 7f76edffb700 1 -- 172.21.15.67:0/1305321899 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76f4003a00 con 0x7f76f80cd950 2023-10-14T10:36:00.353 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.342+0000 7f76ff62b700 1 -- 172.21.15.67:0/1305321899 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76f80b5320 con 0x7f76f80cd950 2023-10-14T10:36:00.353 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.342+0000 7f76edffb700 1 -- 172.21.15.67:0/1305321899 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f4003db0 con 0x7f76f80cd950 2023-10-14T10:36:00.353 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.342+0000 7f76ff62b700 1 -- 172.21.15.67:0/1305321899 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76f80b58e0 con 0x7f76f80cd950 2023-10-14T10:36:00.353 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.343+0000 7f76ff62b700 1 -- 172.21.15.67:0/1305321899 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76f81f3d10 con 0x7f76f80cd950 2023-10-14T10:36:00.353 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.343+0000 7f76edffb700 1 -- 172.21.15.67:0/1305321899 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 4) v1 ==== 89173+0+0 (secure 0 0 0) 0x7f76f4020020 con 0x7f76f80cd950 2023-10-14T10:36:00.353 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.344+0000 7f76edffb700 1 --2- 172.21.15.67:0/1305321899 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f76dc06ae70 0x7f76dc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:00.354 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.344+0000 7f76edffb700 1 -- 172.21.15.67:0/1305321899 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f76f4089ff0 con 0x7f76f80cd950 2023-10-14T10:36:00.354 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.344+0000 7f76fd3c7700 1 --2- 172.21.15.67:0/1305321899 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f76dc06ae70 0x7f76dc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:00.354 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.344+0000 7f76fd3c7700 1 --2- 172.21.15.67:0/1305321899 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f76dc06ae70 0x7f76dc06d330 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f76e8006b20 tx=0x7f76e8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:00.354 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.348+0000 7f76edffb700 1 -- 172.21.15.67:0/1305321899 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f76f81f3d10 con 0x7f76f80cd950 2023-10-14T10:36:00.556 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.554+0000 7f76ff62b700 1 -- 172.21.15.67:0/1305321899 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f76f81f3d10 con 0x7f76f80cd950 2023-10-14T10:36:00.557 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.555+0000 7f76edffb700 1 -- 172.21.15.67:0/1305321899 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12815 (secure 0 0 0) 0x7f76f81f3d10 con 0x7f76f80cd950 2023-10-14T10:36:00.558 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-14T10:36:00.558 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":17,"fsid":"37f9ab46-3082-4b71-8b24-08d39897b008","created":"2023-10-14T10:35:46.067554+0000","modified":"2023-10-14T10:35:59.680080+0000","last_up_change":"2023-10-14T10:35:56.668142+0000","last_in_change":"2023-10-14T10:35:48.551490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-14T10:35:57.809566+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2b59a07e-11ef-440c-b766-9f64ac885838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6801","nonce":1287255499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6803","nonce":1287255499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6807","nonce":1287255499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6805","nonce":1287255499}]},"public_addr":"172.21.15.67:6801/1287255499","cluster_addr":"172.21.15.67:6803/1287255499","heartbeat_back_addr":"172.21.15.67:6807/1287255499","heartbeat_front_addr":"172.21.15.67:6805/1287255499","state":["exists","up"]},{"osd":1,"uuid":"6f813bd6-fa0b-487d-a204-431eb1276a60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6818","nonce":2435538787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6822","nonce":2435538787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6830","nonce":2435538787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6826","nonce":2435538787}]},"public_addr":"172.21.15.67:6818/2435538787","cluster_addr":"172.21.15.67:6822/2435538787","heartbeat_back_addr":"172.21.15.67:6830/2435538787","heartbeat_front_addr":"172.21.15.67:6826/2435538787","state":["exists","up"]},{"osd":2,"uuid":"6af42c6a-1b59-4669-9c04-b6876788446d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6809","nonce":2827491791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6811","nonce":2827491791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6815","nonce":2827491791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6813","nonce":2827491791}]},"public_addr":"172.21.15.67:6809/2827491791","cluster_addr":"172.21.15.67:6811/2827491791","heartbeat_back_addr":"172.21.15.67:6815/2827491791","heartbeat_front_addr":"172.21.15.67:6813/2827491791","state":["exists","up"]},{"osd":3,"uuid":"1d889d76-a646-499d-bfff-ed5bae5434ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6819","nonce":1848001176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6821","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6823","nonce":1848001176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6831","nonce":1848001176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6825","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6827","nonce":1848001176}]},"public_addr":"172.21.15.67:6819/1848001176","cluster_addr":"172.21.15.67:6823/1848001176","heartbeat_back_addr":"172.21.15.67:6831/1848001176","heartbeat_front_addr":"172.21.15.67:6827/1848001176","state":["exists","up"]},{"osd":4,"uuid":"c3271dff-b437-4fdd-b13b-6e49a35473af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6804","nonce":1891230785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6808","nonce":1891230785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6815","nonce":1891230785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6811","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6813","nonce":1891230785}]},"public_addr":"172.21.15.72:6804/1891230785","cluster_addr":"172.21.15.72:6808/1891230785","heartbeat_back_addr":"172.21.15.72:6815/1891230785","heartbeat_front_addr":"172.21.15.72:6813/1891230785","state":["exists","up"]},{"osd":5,"uuid":"0337bb18-4b68-4eff-b528-a667275e6926","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6801","nonce":569686991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6803","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6805","nonce":569686991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6812","nonce":569686991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6807","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6809","nonce":569686991}]},"public_addr":"172.21.15.72:6801/569686991","cluster_addr":"172.21.15.72:6805/569686991","heartbeat_back_addr":"172.21.15.72:6812/569686991","heartbeat_front_addr":"172.21.15.72:6809/569686991","state":["exists","up"]},{"osd":6,"uuid":"8e92d9cf-0c8c-46eb-89a7-db2023ef1509","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6817","nonce":3767685934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6819","nonce":3767685934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6823","nonce":3767685934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6821","nonce":3767685934}]},"public_addr":"172.21.15.72:6817/3767685934","cluster_addr":"172.21.15.72:6819/3767685934","heartbeat_back_addr":"172.21.15.72:6823/3767685934","heartbeat_front_addr":"172.21.15.72:6821/3767685934","state":["exists","up"]},{"osd":7,"uuid":"3fd6aa7d-50a9-45c9-9561-5902a044a119","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6825","nonce":3567465926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6827","nonce":3567465926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6831","nonce":3567465926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6829","nonce":3567465926}]},"public_addr":"172.21.15.72:6825/3567465926","cluster_addr":"172.21.15.72:6827/3567465926","heartbeat_back_addr":"172.21.15.72:6831/3567465926","heartbeat_front_addr":"172.21.15.72:6829/3567465926","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.511466+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.795107+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.858674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.728895+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.794842+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.750064+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.732122+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.765185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-14T10:36:00.560 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.557+0000 7f76ff62b700 1 -- 172.21.15.67:0/1305321899 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f76dc06ae70 msgr2=0x7f76dc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:00.560 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.557+0000 7f76ff62b700 1 --2- 172.21.15.67:0/1305321899 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f76dc06ae70 0x7f76dc06d330 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f76e8006b20 tx=0x7f76e8008040 comp rx=0 tx=0).stop 2023-10-14T10:36:00.560 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.557+0000 7f76ff62b700 1 -- 172.21.15.67:0/1305321899 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76f80cd950 msgr2=0x7f76f80c5a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:00.560 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.557+0000 7f76ff62b700 1 --2- 172.21.15.67:0/1305321899 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76f80cd950 0x7f76f80c5a60 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f76f80cf420 tx=0x7f76f4004750 comp rx=0 tx=0).stop 2023-10-14T10:36:00.560 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.557+0000 7f76ff62b700 1 -- 172.21.15.67:0/1305321899 shutdown_connections 2023-10-14T10:36:00.561 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.557+0000 7f76ff62b700 1 --2- 172.21.15.67:0/1305321899 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f76dc06ae70 0x7f76dc06d330 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:00.561 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.557+0000 7f76ff62b700 1 --2- 172.21.15.67:0/1305321899 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f76f80cede0 0x7f76f81f3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:00.561 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.557+0000 7f76ff62b700 1 --2- 172.21.15.67:0/1305321899 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f76f806a840 0x7f76f80b49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:00.561 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.557+0000 7f76ff62b700 1 --2- 172.21.15.67:0/1305321899 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76f80cd950 0x7f76f80c5a60 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:00.562 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.557+0000 7f76ff62b700 1 -- 172.21.15.67:0/1305321899 >> 172.21.15.67:0/1305321899 conn(0x7f76f806dae0 msgr2=0x7f76f8160640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:00.562 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.557+0000 7f76ff62b700 1 -- 172.21.15.67:0/1305321899 shutdown_connections 2023-10-14T10:36:00.562 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.557+0000 7f76ff62b700 1 -- 172.21.15.67:0/1305321899 wait complete. 2023-10-14T10:36:00.571 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-14T10:35:57.809566+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-14T10:36:00.572 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-14T10:36:00.702 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.699+0000 7f006dac8700 1 Processor -- start 2023-10-14T10:36:00.702 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.699+0000 7f006dac8700 1 -- start start 2023-10-14T10:36:00.704 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.700+0000 7f006dac8700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f00680cd950 0x7f006815e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:00.704 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.700+0000 7f006dac8700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f006806a840 0x7f006806acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:00.705 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.700+0000 7f006dac8700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00680cf260 con 0x7f00680cede0 2023-10-14T10:36:00.705 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.700+0000 7f006dac8700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00681619a0 con 0x7f006806a840 2023-10-14T10:36:00.705 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.700+0000 7f006dac8700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f006806b1f0 con 0x7f00680cd950 2023-10-14T10:36:00.705 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.701+0000 7f0066ffd700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f00680cede0 0x7f006815f570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:51474/0 (socket says 172.21.15.67:51474) 2023-10-14T10:36:00.705 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.701+0000 7f0066ffd700 1 -- 172.21.15.67:0/322274806 learned_addr learned my addr 172.21.15.67:0/322274806 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:36:00.705 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.701+0000 7f0067fff700 1 --2- 172.21.15.67:0/322274806 >> v2:172.21.15.72:3300/0 conn(0x7f006806a840 0x7f006806acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:00.706 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.701+0000 7f00677fe700 1 --2- 172.21.15.67:0/322274806 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f00680cd950 0x7f006815e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:00.706 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.701+0000 7f00677fe700 1 -- 172.21.15.67:0/322274806 >> v2:172.21.15.72:3300/0 conn(0x7f006806a840 msgr2=0x7f006806acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:00.706 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.701+0000 7f00677fe700 1 --2- 172.21.15.67:0/322274806 >> v2:172.21.15.72:3300/0 conn(0x7f006806a840 0x7f006806acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:00.706 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.701+0000 7f00677fe700 1 -- 172.21.15.67:0/322274806 >> v1:172.21.15.67:6789/0 conn(0x7f00680cede0 legacy=0x7f006815f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:00.707 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.701+0000 7f00677fe700 1 -- 172.21.15.67:0/322274806 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00680b2d60 con 0x7f00680cd950 2023-10-14T10:36:00.707 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.701+0000 7f0067fff700 1 --2- 172.21.15.67:0/322274806 >> v2:172.21.15.72:3300/0 conn(0x7f006806a840 0x7f006806acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-14T10:36:00.707 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.701+0000 7f00677fe700 1 --2- 172.21.15.67:0/322274806 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f00680cd950 0x7f006815e2f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f0058009840 tx=0x7f0058009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e16bbc58ca897471 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:00.707 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.701+0000 7f00667fc700 1 -- 172.21.15.67:0/322274806 <== mon.2 v2:172.21.15.72:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0058011830 con 0x7f00680cd950 2023-10-14T10:36:00.707 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.701+0000 7f00667fc700 1 -- 172.21.15.67:0/322274806 <== mon.2 v2:172.21.15.72:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0058011990 con 0x7f00680cd950 2023-10-14T10:36:00.708 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.701+0000 7f00667fc700 1 -- 172.21.15.67:0/322274806 <== mon.2 v2:172.21.15.72:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0058011d20 con 0x7f00680cd950 2023-10-14T10:36:00.708 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.702+0000 7f006dac8700 1 -- 172.21.15.67:0/322274806 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f00680cd950 msgr2=0x7f006815e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:00.708 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.702+0000 7f006dac8700 1 --2- 172.21.15.67:0/322274806 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f00680cd950 0x7f006815e2f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f0058009840 tx=0x7f0058009b50 comp rx=0 tx=0).stop 2023-10-14T10:36:00.708 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.702+0000 7f006dac8700 1 -- 172.21.15.67:0/322274806 shutdown_connections 2023-10-14T10:36:00.708 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.702+0000 7f006dac8700 1 --2- 172.21.15.67:0/322274806 >> v2:172.21.15.72:3300/0 conn(0x7f006806a840 0x7f006806acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:00.709 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.702+0000 7f006dac8700 1 --2- 172.21.15.67:0/322274806 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f00680cd950 0x7f006815e2f0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:00.709 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.702+0000 7f006dac8700 1 -- 172.21.15.67:0/322274806 >> 172.21.15.67:0/322274806 conn(0x7f006806dae0 msgr2=0x7f006806def0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:00.710 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.702+0000 7f006dac8700 1 -- 172.21.15.67:0/322274806 shutdown_connections 2023-10-14T10:36:00.710 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.702+0000 7f006dac8700 1 -- 172.21.15.67:0/322274806 wait complete. 2023-10-14T10:36:00.713 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.704+0000 7f006dac8700 1 Processor -- start 2023-10-14T10:36:00.713 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.704+0000 7f006dac8700 1 -- start start 2023-10-14T10:36:00.714 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.705+0000 7f006dac8700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f006806a840 0x7f0068154bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:00.714 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.705+0000 7f006dac8700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f00680cd950 0x7f00680c9940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:00.714 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.705+0000 7f006dac8700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f00680cede0 0x7f00681538f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:00.714 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.705+0000 7f006dac8700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00680c9e80 con 0x7f006806a840 2023-10-14T10:36:00.714 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.705+0000 7f006dac8700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00681552f0 con 0x7f00680cd950 2023-10-14T10:36:00.715 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.705+0000 7f006dac8700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00681555d0 con 0x7f00680cede0 2023-10-14T10:36:00.715 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.705+0000 7f00677fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f006806a840 0x7f0068154bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:00.715 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.705+0000 7f00677fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f006806a840 0x7f0068154bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43710/0 (socket says 172.21.15.67:43710) 2023-10-14T10:36:00.715 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.705+0000 7f00677fe700 1 -- 172.21.15.67:0/2387488974 learned_addr learned my addr 172.21.15.67:0/2387488974 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:00.715 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.705+0000 7f0066ffd700 1 --2- 172.21.15.67:0/2387488974 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f00680cd950 0x7f00680c9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:00.716 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.705+0000 7f0067fff700 1 --2- 172.21.15.67:0/2387488974 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f00680cede0 0x7f00681538f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:00.716 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.705+0000 7f0067fff700 1 -- 172.21.15.67:0/2387488974 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f00680cd950 msgr2=0x7f00680c9940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:00.716 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.705+0000 7f0067fff700 1 --2- 172.21.15.67:0/2387488974 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f00680cd950 0x7f00680c9940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:00.716 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.705+0000 7f0067fff700 1 -- 172.21.15.67:0/2387488974 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f006806a840 msgr2=0x7f0068154bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:00.716 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.705+0000 7f0067fff700 1 --2- 172.21.15.67:0/2387488974 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f006806a840 0x7f0068154bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:00.717 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.705+0000 7f0067fff700 1 -- 172.21.15.67:0/2387488974 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0060009a70 con 0x7f00680cede0 2023-10-14T10:36:00.717 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.706+0000 7f0067fff700 1 --2- 172.21.15.67:0/2387488974 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f00680cede0 0x7f00681538f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f006000efc0 tx=0x7f006000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:00.717 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.706+0000 7f004ffff700 1 -- 172.21.15.67:0/2387488974 <== mon.2 v2:172.21.15.72:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f006000ca90 con 0x7f00680cede0 2023-10-14T10:36:00.717 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.706+0000 7f004ffff700 1 -- 172.21.15.67:0/2387488974 <== mon.2 v2:172.21.15.72:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f006000f040 con 0x7f00680cede0 2023-10-14T10:36:00.717 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.706+0000 7f006dac8700 1 -- 172.21.15.67:0/2387488974 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00580094a0 con 0x7f00680cede0 2023-10-14T10:36:00.717 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.706+0000 7f004ffff700 1 -- 172.21.15.67:0/2387488974 <== mon.2 v2:172.21.15.72:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f006000ce30 con 0x7f00680cede0 2023-10-14T10:36:00.718 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.706+0000 7f006dac8700 1 -- 172.21.15.67:0/2387488974 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0068155b90 con 0x7f00680cede0 2023-10-14T10:36:00.718 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.707+0000 7f004ffff700 1 -- 172.21.15.67:0/2387488974 <== mon.2 v2:172.21.15.72:3301/0 4 ==== mgrmap(e 4) v1 ==== 89173+0+0 (secure 0 0 0) 0x7f0060014480 con 0x7f00680cede0 2023-10-14T10:36:00.718 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.707+0000 7f006dac8700 1 -- 172.21.15.67:0/2387488974 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f006806acb0 con 0x7f00680cede0 2023-10-14T10:36:00.718 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.708+0000 7f004ffff700 1 --2- 172.21.15.67:0/2387488974 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f005406ae70 0x7f005406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:00.718 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.708+0000 7f004ffff700 1 -- 172.21.15.67:0/2387488974 <== mon.2 v2:172.21.15.72:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f0060089770 con 0x7f00680cede0 2023-10-14T10:36:00.719 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.708+0000 7f00677fe700 1 --2- 172.21.15.67:0/2387488974 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f005406ae70 0x7f005406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:00.719 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.708+0000 7f00677fe700 1 --2- 172.21.15.67:0/2387488974 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f005406ae70 0x7f005406d330 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0058009b90 tx=0x7f0058008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:00.719 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.713+0000 7f004ffff700 1 -- 172.21.15.67:0/2387488974 <== mon.2 v2:172.21.15.72:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0060054c50 con 0x7f00680cede0 2023-10-14T10:36:00.926 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.924+0000 7f006dac8700 1 -- 172.21.15.67:0/2387488974 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0068067d20 con 0x7f00680cede0 2023-10-14T10:36:00.927 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.925+0000 7f004ffff700 1 -- 172.21.15.67:0/2387488974 <== mon.2 v2:172.21.15.72:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v18) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0060009cc0 con 0x7f00680cede0 2023-10-14T10:36:00.927 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2023-10-14T10:36:00.929 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.927+0000 7f006dac8700 1 -- 172.21.15.67:0/2387488974 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f005406ae70 msgr2=0x7f005406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:00.930 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.927+0000 7f006dac8700 1 --2- 172.21.15.67:0/2387488974 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f005406ae70 0x7f005406d330 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0058009b90 tx=0x7f0058008030 comp rx=0 tx=0).stop 2023-10-14T10:36:00.930 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.927+0000 7f006dac8700 1 -- 172.21.15.67:0/2387488974 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f00680cede0 msgr2=0x7f00681538f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:00.930 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.927+0000 7f006dac8700 1 --2- 172.21.15.67:0/2387488974 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f00680cede0 0x7f00681538f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f006000efc0 tx=0x7f006000c3a0 comp rx=0 tx=0).stop 2023-10-14T10:36:00.931 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.927+0000 7f006dac8700 1 -- 172.21.15.67:0/2387488974 shutdown_connections 2023-10-14T10:36:00.931 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.927+0000 7f006dac8700 1 --2- 172.21.15.67:0/2387488974 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f005406ae70 0x7f005406d330 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:00.931 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.927+0000 7f006dac8700 1 --2- 172.21.15.67:0/2387488974 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f00680cede0 0x7f00681538f0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:00.931 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.927+0000 7f006dac8700 1 --2- 172.21.15.67:0/2387488974 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f006806a840 0x7f0068154bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:00.932 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.927+0000 7f006dac8700 1 --2- 172.21.15.67:0/2387488974 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f00680cd950 0x7f00680c9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:00.932 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.927+0000 7f006dac8700 1 -- 172.21.15.67:0/2387488974 >> 172.21.15.67:0/2387488974 conn(0x7f006806dae0 msgr2=0x7f00680cce80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:00.932 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.927+0000 7f006dac8700 1 -- 172.21.15.67:0/2387488974 shutdown_connections 2023-10-14T10:36:00.932 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:00.927+0000 7f006dac8700 1 -- 172.21.15.67:0/2387488974 wait complete. 2023-10-14T10:36:00.942 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-14T10:36:01.066 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.064+0000 7f844be1f700 1 Processor -- start 2023-10-14T10:36:01.067 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.065+0000 7f844be1f700 1 -- start start 2023-10-14T10:36:01.070 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.065+0000 7f844be1f700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f844415ab10 0x7f844415f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:01.070 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.065+0000 7f844be1f700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f844415e7a0 con 0x7f84440cfe30 2023-10-14T10:36:01.070 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.065+0000 7f844be1f700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f844415f730 con 0x7f84440b1f50 2023-10-14T10:36:01.071 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.065+0000 7f844be1f700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84440b29a0 con 0x7f844415ab10 2023-10-14T10:36:01.071 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.065+0000 7f844a3bc700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f84440cfe30 0x7f84440cbf40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:51490/0 (socket says 172.21.15.67:51490) 2023-10-14T10:36:01.071 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.065+0000 7f844a3bc700 1 -- 172.21.15.67:0/3530588243 learned_addr learned my addr 172.21.15.67:0/3530588243 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:36:01.071 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.066+0000 7f8449bbb700 1 --2- 172.21.15.67:0/3530588243 >> v2:172.21.15.72:3300/0 conn(0x7f844415ab10 0x7f844415f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:01.071 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.066+0000 7f8448bb9700 1 -- 172.21.15.67:0/3530588243 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1221715630 0 0) 0x7f844415e7a0 con 0x7f84440cfe30 2023-10-14T10:36:01.072 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.066+0000 7f8448bb9700 1 -- 172.21.15.67:0/3530588243 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84300035b0 con 0x7f84440cfe30 2023-10-14T10:36:01.072 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.066+0000 7f8448bb9700 1 -- 172.21.15.67:0/3530588243 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2552486145 0 0) 0x7f84300035b0 con 0x7f84440cfe30 2023-10-14T10:36:01.072 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.066+0000 7f8448bb9700 1 -- 172.21.15.67:0/3530588243 >> v2:172.21.15.72:3300/0 conn(0x7f844415ab10 msgr2=0x7f844415f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:01.072 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.066+0000 7f8448bb9700 1 --2- 172.21.15.67:0/3530588243 >> v2:172.21.15.72:3300/0 conn(0x7f844415ab10 0x7f844415f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:01.073 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.066+0000 7f8448bb9700 1 -- 172.21.15.67:0/3530588243 >> v1:172.21.15.72:6789/0 conn(0x7f84440b1f50 legacy=0x7f84440cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:01.073 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.066+0000 7f8448bb9700 1 -- 172.21.15.67:0/3530588243 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84440b2c80 con 0x7f84440cfe30 2023-10-14T10:36:01.073 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.066+0000 7f8448bb9700 1 -- 172.21.15.67:0/3530588243 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3180043755 0 0) 0x7f843c0019f0 con 0x7f84440cfe30 2023-10-14T10:36:01.073 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.066+0000 7f8448bb9700 1 -- 172.21.15.67:0/3530588243 >> v1:172.21.15.67:6789/0 conn(0x7f84440cfe30 legacy=0x7f84440cbf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:01.073 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.066+0000 7f8448bb9700 1 --2- 172.21.15.67:0/3530588243 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8430003a00 0x7f8430003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:01.074 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.066+0000 7f8448bb9700 1 --2- 172.21.15.67:0/3530588243 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f8430004850 0x7f8430004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:01.074 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.066+0000 7f8448bb9700 1 --2- 172.21.15.67:0/3530588243 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8430005210 0x7f843000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:01.074 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.066+0000 7f8448bb9700 1 -- 172.21.15.67:0/3530588243 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84440b29a0 con 0x7f8430003a00 2023-10-14T10:36:01.074 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.066+0000 7f8448bb9700 1 -- 172.21.15.67:0/3530588243 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f843000c0b0 con 0x7f8430005210 2023-10-14T10:36:01.074 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.066+0000 7f8448bb9700 1 -- 172.21.15.67:0/3530588243 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f843000c270 con 0x7f8430004850 2023-10-14T10:36:01.075 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.066+0000 7f8449bbb700 1 --2- 172.21.15.67:0/3530588243 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8430003a00 0x7f8430003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:01.075 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.066+0000 7f844a3bc700 1 --2- 172.21.15.67:0/3530588243 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8430005210 0x7f843000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:01.075 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.066+0000 7f84493ba700 1 --2- 172.21.15.67:0/3530588243 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f8430004850 0x7f8430004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:01.075 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.066+0000 7f8449bbb700 1 -- 172.21.15.67:0/3530588243 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f8430004850 msgr2=0x7f8430004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:01.075 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.066+0000 7f8449bbb700 1 --2- 172.21.15.67:0/3530588243 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f8430004850 0x7f8430004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:01.076 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.066+0000 7f8449bbb700 1 -- 172.21.15.67:0/3530588243 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8430005210 msgr2=0x7f843000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:01.076 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.066+0000 7f8449bbb700 1 --2- 172.21.15.67:0/3530588243 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8430005210 0x7f843000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:01.076 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.066+0000 7f8449bbb700 1 -- 172.21.15.67:0/3530588243 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84300035b0 con 0x7f8430003a00 2023-10-14T10:36:01.076 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.066+0000 7f844a3bc700 1 --2- 172.21.15.67:0/3530588243 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8430005210 0x7f843000ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-14T10:36:01.076 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.067+0000 7f844a3bc700 1 -- 172.21.15.67:0/3530588243 reap_dead start 2023-10-14T10:36:01.076 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.067+0000 7f8449bbb700 1 --2- 172.21.15.67:0/3530588243 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8430003a00 0x7f8430003e20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8434009b60 tx=0x7f8430005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=427f3b0a540086f server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:01.077 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.067+0000 7f8448bb9700 1 -- 172.21.15.67:0/3530588243 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f843400e6c0 con 0x7f8430003a00 2023-10-14T10:36:01.077 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.067+0000 7f8448bb9700 1 -- 172.21.15.67:0/3530588243 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f843400e820 con 0x7f8430003a00 2023-10-14T10:36:01.077 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.067+0000 7f8448bb9700 1 -- 172.21.15.67:0/3530588243 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f843400eb90 con 0x7f8430003a00 2023-10-14T10:36:01.077 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.067+0000 7f844be1f700 1 -- 172.21.15.67:0/3530588243 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8430003a00 msgr2=0x7f8430003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:01.077 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.067+0000 7f844be1f700 1 --2- 172.21.15.67:0/3530588243 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8430003a00 0x7f8430003e20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8434009b60 tx=0x7f8430005880 comp rx=0 tx=0).stop 2023-10-14T10:36:01.078 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.067+0000 7f844be1f700 1 -- 172.21.15.67:0/3530588243 shutdown_connections 2023-10-14T10:36:01.078 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.067+0000 7f844be1f700 1 --2- 172.21.15.67:0/3530588243 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8430003a00 0x7f8430003e20 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:01.078 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.067+0000 7f844be1f700 1 -- 172.21.15.67:0/3530588243 >> 172.21.15.67:0/3530588243 conn(0x7f844406dae0 msgr2=0x7f844406def0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:01.078 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.067+0000 7f844be1f700 1 -- 172.21.15.67:0/3530588243 shutdown_connections 2023-10-14T10:36:01.079 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.067+0000 7f844be1f700 1 -- 172.21.15.67:0/3530588243 wait complete. 2023-10-14T10:36:01.082 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.069+0000 7f844be1f700 1 Processor -- start 2023-10-14T10:36:01.082 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.069+0000 7f844be1f700 1 -- start start 2023-10-14T10:36:01.082 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.069+0000 7f844be1f700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8430003a00 0x7f844415ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:01.082 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.069+0000 7f844be1f700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f84441f6bb0 0x7f8444157c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:01.083 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.069+0000 7f844be1f700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84440c7e20 0x7f84440c82a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:01.083 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.069+0000 7f844be1f700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84441f7b50 con 0x7f84440c7e20 2023-10-14T10:36:01.083 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.069+0000 7f844be1f700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84440acb50 con 0x7f8430003a00 2023-10-14T10:36:01.083 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.069+0000 7f844be1f700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f844415d7e0 con 0x7f84441f6bb0 2023-10-14T10:36:01.084 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.069+0000 7f844a3bc700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84440c7e20 0x7f84440c82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:01.084 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.070+0000 7f844a3bc700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84440c7e20 0x7f84440c82a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43734/0 (socket says 172.21.15.67:43734) 2023-10-14T10:36:01.084 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.070+0000 7f844a3bc700 1 -- 172.21.15.67:0/1863135300 learned_addr learned my addr 172.21.15.67:0/1863135300 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:01.084 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.070+0000 7f84493ba700 1 --2- 172.21.15.67:0/1863135300 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f84441f6bb0 0x7f8444157c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:01.084 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.070+0000 7f8449bbb700 1 --2- 172.21.15.67:0/1863135300 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8430003a00 0x7f844415ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:01.085 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.070+0000 7f844a3bc700 1 -- 172.21.15.67:0/1863135300 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f84441f6bb0 msgr2=0x7f8444157c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:01.085 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.070+0000 7f844a3bc700 1 --2- 172.21.15.67:0/1863135300 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f84441f6bb0 0x7f8444157c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:01.085 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.070+0000 7f844a3bc700 1 -- 172.21.15.67:0/1863135300 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8430003a00 msgr2=0x7f844415ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:01.085 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.070+0000 7f844a3bc700 1 --2- 172.21.15.67:0/1863135300 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8430003a00 0x7f844415ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:01.085 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.070+0000 7f844a3bc700 1 -- 172.21.15.67:0/1863135300 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84441f7030 con 0x7f84440c7e20 2023-10-14T10:36:01.086 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.070+0000 7f844a3bc700 1 --2- 172.21.15.67:0/1863135300 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84440c7e20 0x7f84440c82a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f844415e700 tx=0x7f843c00f870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:01.086 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.070+0000 7f843a7fc700 1 -- 172.21.15.67:0/1863135300 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f843c006310 con 0x7f84440c7e20 2023-10-14T10:36:01.086 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.070+0000 7f843a7fc700 1 -- 172.21.15.67:0/1863135300 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f843c006470 con 0x7f84440c7e20 2023-10-14T10:36:01.086 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.070+0000 7f843a7fc700 1 -- 172.21.15.67:0/1863135300 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f843c006820 con 0x7f84440c7e20 2023-10-14T10:36:01.087 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.070+0000 7f844be1f700 1 -- 172.21.15.67:0/1863135300 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84441f72b0 con 0x7f84440c7e20 2023-10-14T10:36:01.087 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.070+0000 7f844be1f700 1 -- 172.21.15.67:0/1863135300 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84441f7870 con 0x7f84440c7e20 2023-10-14T10:36:01.087 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.071+0000 7f844be1f700 1 -- 172.21.15.67:0/1863135300 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84440ac8b0 con 0x7f84440c7e20 2023-10-14T10:36:01.087 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.072+0000 7f843a7fc700 1 -- 172.21.15.67:0/1863135300 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 4) v1 ==== 89173+0+0 (secure 0 0 0) 0x7f843c01c460 con 0x7f84440c7e20 2023-10-14T10:36:01.087 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.072+0000 7f843a7fc700 1 --2- 172.21.15.67:0/1863135300 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f842806ae70 0x7f842806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:01.088 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.072+0000 7f843a7fc700 1 -- 172.21.15.67:0/1863135300 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f843c08a060 con 0x7f84440c7e20 2023-10-14T10:36:01.088 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.072+0000 7f8449bbb700 1 --2- 172.21.15.67:0/1863135300 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f842806ae70 0x7f842806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:01.088 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.073+0000 7f8449bbb700 1 --2- 172.21.15.67:0/1863135300 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f842806ae70 0x7f842806d330 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f84340090b0 tx=0x7f843401f040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:01.088 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.077+0000 7f843a7fc700 1 -- 172.21.15.67:0/1863135300 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f84440ac8b0 con 0x7f84440c7e20 2023-10-14T10:36:01.281 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.278+0000 7f844be1f700 1 -- 172.21.15.67:0/1863135300 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f84440acc90 con 0x7f84440c7e20 2023-10-14T10:36:01.281 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.279+0000 7f843a7fc700 1 -- 172.21.15.67:0/1863135300 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v18) v1 ==== 84+0+65 (secure 0 0 0) 0x7f84440acc90 con 0x7f84440c7e20 2023-10-14T10:36:01.281 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-14T10:36:01.282 INFO:teuthology.orchestra.run.smithi067.stdout:[ 2023-10-14T10:36:01.282 INFO:teuthology.orchestra.run.smithi067.stdout: { 2023-10-14T10:36:01.282 INFO:teuthology.orchestra.run.smithi067.stdout: "poolnum": 1, 2023-10-14T10:36:01.282 INFO:teuthology.orchestra.run.smithi067.stdout: "poolname": ".mgr" 2023-10-14T10:36:01.283 INFO:teuthology.orchestra.run.smithi067.stdout: } 2023-10-14T10:36:01.283 INFO:teuthology.orchestra.run.smithi067.stdout:] 2023-10-14T10:36:01.284 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.281+0000 7f844be1f700 1 -- 172.21.15.67:0/1863135300 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f842806ae70 msgr2=0x7f842806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:01.284 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.281+0000 7f844be1f700 1 --2- 172.21.15.67:0/1863135300 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f842806ae70 0x7f842806d330 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f84340090b0 tx=0x7f843401f040 comp rx=0 tx=0).stop 2023-10-14T10:36:01.284 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.281+0000 7f844be1f700 1 -- 172.21.15.67:0/1863135300 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84440c7e20 msgr2=0x7f84440c82a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:01.285 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.281+0000 7f844be1f700 1 --2- 172.21.15.67:0/1863135300 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84440c7e20 0x7f84440c82a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f844415e700 tx=0x7f843c00f870 comp rx=0 tx=0).stop 2023-10-14T10:36:01.285 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.281+0000 7f844be1f700 1 -- 172.21.15.67:0/1863135300 shutdown_connections 2023-10-14T10:36:01.285 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.281+0000 7f844be1f700 1 --2- 172.21.15.67:0/1863135300 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f842806ae70 0x7f842806d330 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:01.285 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.281+0000 7f844be1f700 1 --2- 172.21.15.67:0/1863135300 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84440c7e20 0x7f84440c82a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:01.285 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.281+0000 7f844be1f700 1 --2- 172.21.15.67:0/1863135300 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8430003a00 0x7f844415ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:01.286 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.281+0000 7f844be1f700 1 --2- 172.21.15.67:0/1863135300 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f84441f6bb0 0x7f8444157c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:01.286 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.281+0000 7f844be1f700 1 -- 172.21.15.67:0/1863135300 >> 172.21.15.67:0/1863135300 conn(0x7f844406dae0 msgr2=0x7f84440cd870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:01.286 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.281+0000 7f844be1f700 1 -- 172.21.15.67:0/1863135300 shutdown_connections 2023-10-14T10:36:01.286 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.281+0000 7f844be1f700 1 -- 172.21.15.67:0/1863135300 wait complete. 2023-10-14T10:36:01.295 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-14T10:36:01.295 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-14T10:36:01.424 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.422+0000 7f17a0902700 1 Processor -- start 2023-10-14T10:36:01.424 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.422+0000 7f17a0902700 1 -- start start 2023-10-14T10:36:01.427 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.423+0000 7f17a0902700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f179c06d320 0x7f179c15c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:01.427 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.423+0000 7f17a0902700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f179c06d700 0x7f179c071210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:01.427 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.423+0000 7f17a0902700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f179c071750 0x7f179c15f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:01.428 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.423+0000 7f17a0902700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f179c071c60 con 0x7f179c06d320 2023-10-14T10:36:01.428 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.423+0000 7f17a0902700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f179c071f40 con 0x7f179c06d700 2023-10-14T10:36:01.428 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.423+0000 7f17a0902700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f179c15bfb0 con 0x7f179c071750 2023-10-14T10:36:01.428 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.423+0000 7f179a59c700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f179c06d320 0x7f179c15c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:01.429 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.423+0000 7f179a59c700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f179c06d320 0x7f179c15c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43738/0 (socket says 172.21.15.67:43738) 2023-10-14T10:36:01.429 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.423+0000 7f179a59c700 1 -- 172.21.15.67:0/2434109669 learned_addr learned my addr 172.21.15.67:0/2434109669 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:01.429 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.423+0000 7f1799d9b700 1 --2- 172.21.15.67:0/2434109669 >> v2:172.21.15.72:3300/0 conn(0x7f179c06d700 0x7f179c071210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:01.429 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.423+0000 7f179ad9d700 1 --2- 172.21.15.67:0/2434109669 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f179c071750 0x7f179c15f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:01.429 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.423+0000 7f179a59c700 1 -- 172.21.15.67:0/2434109669 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f179c071750 msgr2=0x7f179c15f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:01.430 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.423+0000 7f179a59c700 1 --2- 172.21.15.67:0/2434109669 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f179c071750 0x7f179c15f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:01.430 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.423+0000 7f179a59c700 1 -- 172.21.15.67:0/2434109669 >> v2:172.21.15.72:3300/0 conn(0x7f179c06d700 msgr2=0x7f179c071210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:01.430 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.423+0000 7f179a59c700 1 --2- 172.21.15.67:0/2434109669 >> v2:172.21.15.72:3300/0 conn(0x7f179c06d700 0x7f179c071210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:01.430 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.423+0000 7f179a59c700 1 -- 172.21.15.67:0/2434109669 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f179c15c290 con 0x7f179c06d320 2023-10-14T10:36:01.431 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.423+0000 7f179a59c700 1 --2- 172.21.15.67:0/2434109669 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f179c06d320 0x7f179c15c7b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f179c0d0d10 tx=0x7f1790009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=693e33194022446c server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:01.431 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.424+0000 7f179959a700 1 -- 172.21.15.67:0/2434109669 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f179000b870 con 0x7f179c06d320 2023-10-14T10:36:01.431 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.424+0000 7f179959a700 1 -- 172.21.15.67:0/2434109669 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f179000b9d0 con 0x7f179c06d320 2023-10-14T10:36:01.431 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.424+0000 7f179959a700 1 -- 172.21.15.67:0/2434109669 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f179000bd60 con 0x7f179c06d320 2023-10-14T10:36:01.431 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.424+0000 7f17a0902700 1 -- 172.21.15.67:0/2434109669 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f179c06d320 msgr2=0x7f179c15c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:01.432 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.424+0000 7f17a0902700 1 --2- 172.21.15.67:0/2434109669 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f179c06d320 0x7f179c15c7b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f179c0d0d10 tx=0x7f1790009bd0 comp rx=0 tx=0).stop 2023-10-14T10:36:01.432 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.424+0000 7f17a0902700 1 -- 172.21.15.67:0/2434109669 shutdown_connections 2023-10-14T10:36:01.432 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.424+0000 7f17a0902700 1 --2- 172.21.15.67:0/2434109669 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f179c071750 0x7f179c15f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:01.432 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.424+0000 7f17a0902700 1 --2- 172.21.15.67:0/2434109669 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f179c06d320 0x7f179c15c7b0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:01.432 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.424+0000 7f17a0902700 1 --2- 172.21.15.67:0/2434109669 >> v2:172.21.15.72:3300/0 conn(0x7f179c06d700 0x7f179c071210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:01.433 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.424+0000 7f17a0902700 1 -- 172.21.15.67:0/2434109669 >> 172.21.15.67:0/2434109669 conn(0x7f179c0cfc10 msgr2=0x7f179c0d0020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:01.433 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.424+0000 7f17a0902700 1 -- 172.21.15.67:0/2434109669 shutdown_connections 2023-10-14T10:36:01.433 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.424+0000 7f17a0902700 1 -- 172.21.15.67:0/2434109669 wait complete. 2023-10-14T10:36:01.436 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.426+0000 7f17a0902700 1 Processor -- start 2023-10-14T10:36:01.436 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.426+0000 7f17a0902700 1 -- start start 2023-10-14T10:36:01.436 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.426+0000 7f17a0902700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f179c06d320 0x7f179c149f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:01.436 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.426+0000 7f17a0902700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f179c06d700 0x7f179c154720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:01.436 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.426+0000 7f17a0902700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f179c071750 0x7f179c159e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:01.437 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.426+0000 7f17a0902700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f179c14a460 con 0x7f179c06d700 2023-10-14T10:36:01.437 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.426+0000 7f17a0902700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f179c153680 con 0x7f179c06d320 2023-10-14T10:36:01.437 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.426+0000 7f17a0902700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f179c153960 con 0x7f179c071750 2023-10-14T10:36:01.437 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.426+0000 7f1799d9b700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f179c06d700 0x7f179c154720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:01.437 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.427+0000 7f1799d9b700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f179c06d700 0x7f179c154720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43746/0 (socket says 172.21.15.67:43746) 2023-10-14T10:36:01.438 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.427+0000 7f1799d9b700 1 -- 172.21.15.67:0/3083732235 learned_addr learned my addr 172.21.15.67:0/3083732235 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:01.438 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.427+0000 7f179a59c700 1 --2- 172.21.15.67:0/3083732235 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f179c06d320 0x7f179c149f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:01.438 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.427+0000 7f179ad9d700 1 --2- 172.21.15.67:0/3083732235 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f179c071750 0x7f179c159e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:01.438 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.427+0000 7f1799d9b700 1 -- 172.21.15.67:0/3083732235 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f179c071750 msgr2=0x7f179c159e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:01.438 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.427+0000 7f1799d9b700 1 --2- 172.21.15.67:0/3083732235 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f179c071750 0x7f179c159e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:01.439 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.427+0000 7f1799d9b700 1 -- 172.21.15.67:0/3083732235 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f179c06d320 msgr2=0x7f179c149f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:01.439 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.427+0000 7f1799d9b700 1 --2- 172.21.15.67:0/3083732235 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f179c06d320 0x7f179c149f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:01.439 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.427+0000 7f1799d9b700 1 -- 172.21.15.67:0/3083732235 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1790009550 con 0x7f179c06d700 2023-10-14T10:36:01.439 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.427+0000 7f179ad9d700 1 --2- 172.21.15.67:0/3083732235 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f179c071750 0x7f179c159e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-14T10:36:01.439 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.427+0000 7f1799d9b700 1 --2- 172.21.15.67:0/3083732235 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f179c06d700 0x7f179c154720 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f178c00e810 tx=0x7f178c00eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:01.440 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.427+0000 7f178affd700 1 -- 172.21.15.67:0/3083732235 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f178c013610 con 0x7f179c06d700 2023-10-14T10:36:01.440 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.427+0000 7f17a0902700 1 -- 172.21.15.67:0/3083732235 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f179c153c40 con 0x7f179c06d700 2023-10-14T10:36:01.440 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.427+0000 7f178affd700 1 -- 172.21.15.67:0/3083732235 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f178c013770 con 0x7f179c06d700 2023-10-14T10:36:01.440 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.427+0000 7f178affd700 1 -- 172.21.15.67:0/3083732235 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f178c013b20 con 0x7f179c06d700 2023-10-14T10:36:01.440 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.427+0000 7f17a0902700 1 -- 172.21.15.67:0/3083732235 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f179c154200 con 0x7f179c06d700 2023-10-14T10:36:01.441 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.428+0000 7f17a0902700 1 -- 172.21.15.67:0/3083732235 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f179c067d20 con 0x7f179c06d700 2023-10-14T10:36:01.441 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.429+0000 7f178affd700 1 -- 172.21.15.67:0/3083732235 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 4) v1 ==== 89173+0+0 (secure 0 0 0) 0x7f178c013c80 con 0x7f179c06d700 2023-10-14T10:36:01.441 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.429+0000 7f178affd700 1 --2- 172.21.15.67:0/3083732235 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f177c06ae70 0x7f177c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:01.441 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.429+0000 7f178affd700 1 -- 172.21.15.67:0/3083732235 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f178c088e40 con 0x7f179c06d700 2023-10-14T10:36:01.441 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.429+0000 7f179a59c700 1 --2- 172.21.15.67:0/3083732235 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f177c06ae70 0x7f177c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:01.442 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.430+0000 7f179a59c700 1 --2- 172.21.15.67:0/3083732235 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f177c06ae70 0x7f177c06d330 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1790014040 tx=0x7f1790008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:01.442 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.433+0000 7f178affd700 1 -- 172.21.15.67:0/3083732235 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f178c054330 con 0x7f179c06d700 2023-10-14T10:36:01.638 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.636+0000 7f17a0902700 1 -- 172.21.15.67:0/3083732235 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f179c158600 con 0x7f179c06d700 2023-10-14T10:36:01.825 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.823+0000 7f178affd700 1 -- 172.21.15.67:0/3083732235 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v19) v1 ==== 138+0+0 (secure 0 0 0) 0x7f178c057950 con 0x7f179c06d700 2023-10-14T10:36:01.827 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.825+0000 7f17a0902700 1 -- 172.21.15.67:0/3083732235 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f177c06ae70 msgr2=0x7f177c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:01.827 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.825+0000 7f17a0902700 1 --2- 172.21.15.67:0/3083732235 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f177c06ae70 0x7f177c06d330 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1790014040 tx=0x7f1790008030 comp rx=0 tx=0).stop 2023-10-14T10:36:01.828 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.825+0000 7f17a0902700 1 -- 172.21.15.67:0/3083732235 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f179c06d700 msgr2=0x7f179c154720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:01.828 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.825+0000 7f17a0902700 1 --2- 172.21.15.67:0/3083732235 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f179c06d700 0x7f179c154720 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f178c00e810 tx=0x7f178c00eb20 comp rx=0 tx=0).stop 2023-10-14T10:36:01.828 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.825+0000 7f17a0902700 1 -- 172.21.15.67:0/3083732235 shutdown_connections 2023-10-14T10:36:01.828 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.825+0000 7f17a0902700 1 --2- 172.21.15.67:0/3083732235 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f177c06ae70 0x7f177c06d330 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:01.828 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.825+0000 7f17a0902700 1 --2- 172.21.15.67:0/3083732235 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f179c071750 0x7f179c159e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:01.829 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.825+0000 7f17a0902700 1 --2- 172.21.15.67:0/3083732235 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f179c06d320 0x7f179c149f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:01.829 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.825+0000 7f17a0902700 1 --2- 172.21.15.67:0/3083732235 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f179c06d700 0x7f179c154720 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:01.829 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.825+0000 7f17a0902700 1 -- 172.21.15.67:0/3083732235 >> 172.21.15.67:0/3083732235 conn(0x7f179c0cfc10 msgr2=0x7f179c156090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:01.829 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.825+0000 7f17a0902700 1 -- 172.21.15.67:0/3083732235 shutdown_connections 2023-10-14T10:36:01.829 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.825+0000 7f17a0902700 1 -- 172.21.15.67:0/3083732235 wait complete. 2023-10-14T10:36:01.830 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs_metadata' created 2023-10-14T10:36:01.845 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-14T10:36:01.977 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.975+0000 7fd648166700 1 Processor -- start 2023-10-14T10:36:01.978 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.976+0000 7fd648166700 1 -- start start 2023-10-14T10:36:01.980 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.976+0000 7fd648166700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fd64006d390 0x7fd64006d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:01.981 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.976+0000 7fd648166700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd64006ba60 0x7fd64006a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:01.981 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.976+0000 7fd648166700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6400721e0 con 0x7fd6400ce550 2023-10-14T10:36:01.981 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.976+0000 7fd648166700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd64006dcb0 con 0x7fd64006ba60 2023-10-14T10:36:01.981 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.976+0000 7fd648166700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6400715d0 con 0x7fd64006d390 2023-10-14T10:36:01.981 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.976+0000 7fd645701700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd64006ba60 0x7fd64006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:01.982 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.976+0000 7fd645701700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd64006ba60 0x7fd64006a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43752/0 (socket says 172.21.15.67:43752) 2023-10-14T10:36:01.982 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.976+0000 7fd645701700 1 -- 172.21.15.67:0/3513085409 learned_addr learned my addr 172.21.15.67:0/3513085409 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:01.982 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.977+0000 7fd645f02700 1 --2- 172.21.15.67:0/3513085409 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fd64006d390 0x7fd64006d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:01.982 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.977+0000 7fd645701700 1 -- 172.21.15.67:0/3513085409 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fd64006d390 msgr2=0x7fd64006d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:01.982 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.977+0000 7fd645701700 1 --2- 172.21.15.67:0/3513085409 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fd64006d390 0x7fd64006d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:01.983 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.977+0000 7fd645701700 1 -- 172.21.15.67:0/3513085409 >> v1:172.21.15.72:6789/0 conn(0x7fd6400ce550 legacy=0x7fd64006ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:01.983 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.977+0000 7fd645701700 1 -- 172.21.15.67:0/3513085409 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6400718b0 con 0x7fd64006ba60 2023-10-14T10:36:01.983 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.977+0000 7fd645f02700 1 --2- 172.21.15.67:0/3513085409 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fd64006d390 0x7fd64006d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-14T10:36:01.983 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.977+0000 7fd645701700 1 --2- 172.21.15.67:0/3513085409 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd64006ba60 0x7fd64006a840 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd63c0098f0 tx=0x7fd63c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75d6fde81414de2e server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:01.983 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.977+0000 7fd644f00700 1 -- 172.21.15.67:0/3513085409 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd63c00b8a0 con 0x7fd64006ba60 2023-10-14T10:36:01.984 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.977+0000 7fd644f00700 1 -- 172.21.15.67:0/3513085409 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd63c00ba00 con 0x7fd64006ba60 2023-10-14T10:36:01.984 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.977+0000 7fd644f00700 1 -- 172.21.15.67:0/3513085409 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd63c00bd90 con 0x7fd64006ba60 2023-10-14T10:36:01.984 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.978+0000 7fd648166700 1 -- 172.21.15.67:0/3513085409 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd64006ba60 msgr2=0x7fd64006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:01.984 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.978+0000 7fd648166700 1 --2- 172.21.15.67:0/3513085409 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd64006ba60 0x7fd64006a840 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd63c0098f0 tx=0x7fd63c009c00 comp rx=0 tx=0).stop 2023-10-14T10:36:01.984 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.978+0000 7fd648166700 1 -- 172.21.15.67:0/3513085409 shutdown_connections 2023-10-14T10:36:01.985 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.978+0000 7fd648166700 1 --2- 172.21.15.67:0/3513085409 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fd64006d390 0x7fd64006d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:01.985 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.978+0000 7fd648166700 1 --2- 172.21.15.67:0/3513085409 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd64006ba60 0x7fd64006a840 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:01.985 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.978+0000 7fd648166700 1 -- 172.21.15.67:0/3513085409 >> 172.21.15.67:0/3513085409 conn(0x7fd6400cfc40 msgr2=0x7fd6400d0020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:01.985 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.978+0000 7fd648166700 1 -- 172.21.15.67:0/3513085409 shutdown_connections 2023-10-14T10:36:01.985 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.978+0000 7fd648166700 1 -- 172.21.15.67:0/3513085409 wait complete. 2023-10-14T10:36:01.988 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.979+0000 7fd648166700 1 Processor -- start 2023-10-14T10:36:01.988 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.979+0000 7fd648166700 1 -- start start 2023-10-14T10:36:01.989 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.980+0000 7fd648166700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd64006ba60 0x7fd64014ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:01.989 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.980+0000 7fd648166700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd64006d390 0x7fd6400a84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:01.989 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.980+0000 7fd648166700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fd6400ce550 0x7fd6400c9190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:01.989 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.980+0000 7fd648166700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd640150470 con 0x7fd64006ba60 2023-10-14T10:36:01.989 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.980+0000 7fd648166700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6400c8d00 con 0x7fd64006d390 2023-10-14T10:36:01.990 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.980+0000 7fd648166700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6400c80f0 con 0x7fd6400ce550 2023-10-14T10:36:01.990 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.980+0000 7fd645f02700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd64006ba60 0x7fd64014ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:01.990 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.980+0000 7fd645f02700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd64006ba60 0x7fd64014ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43760/0 (socket says 172.21.15.67:43760) 2023-10-14T10:36:01.990 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.980+0000 7fd645f02700 1 -- 172.21.15.67:0/165093878 learned_addr learned my addr 172.21.15.67:0/165093878 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:01.990 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.980+0000 7fd645701700 1 --2- 172.21.15.67:0/165093878 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd64006d390 0x7fd6400a84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:01.991 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.980+0000 7fd646703700 1 --2- 172.21.15.67:0/165093878 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fd6400ce550 0x7fd6400c9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:01.991 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.980+0000 7fd645f02700 1 -- 172.21.15.67:0/165093878 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fd6400ce550 msgr2=0x7fd6400c9190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:01.991 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.980+0000 7fd645f02700 1 --2- 172.21.15.67:0/165093878 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fd6400ce550 0x7fd6400c9190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:01.991 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.980+0000 7fd645f02700 1 -- 172.21.15.67:0/165093878 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd64006d390 msgr2=0x7fd6400a84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:01.991 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.980+0000 7fd645f02700 1 --2- 172.21.15.67:0/165093878 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd64006d390 0x7fd6400a84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:01.992 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.980+0000 7fd645f02700 1 -- 172.21.15.67:0/165093878 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd63c009550 con 0x7fd64006ba60 2023-10-14T10:36:01.992 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.980+0000 7fd645f02700 1 --2- 172.21.15.67:0/165093878 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd64006ba60 0x7fd64014ff30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd63000e810 tx=0x7fd63000eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:01.992 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.980+0000 7fd6367fc700 1 -- 172.21.15.67:0/165093878 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd63000c070 con 0x7fd64006ba60 2023-10-14T10:36:01.992 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.981+0000 7fd6367fc700 1 -- 172.21.15.67:0/165093878 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd630004230 con 0x7fd64006ba60 2023-10-14T10:36:01.992 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.981+0000 7fd6367fc700 1 -- 172.21.15.67:0/165093878 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6300036a0 con 0x7fd64006ba60 2023-10-14T10:36:01.993 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.981+0000 7fd648166700 1 -- 172.21.15.67:0/165093878 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6400c83d0 con 0x7fd64006ba60 2023-10-14T10:36:01.993 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.981+0000 7fd648166700 1 -- 172.21.15.67:0/165093878 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6400c8990 con 0x7fd64006ba60 2023-10-14T10:36:01.993 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.982+0000 7fd648166700 1 -- 172.21.15.67:0/165093878 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd640067d20 con 0x7fd64006ba60 2023-10-14T10:36:01.993 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.982+0000 7fd6367fc700 1 -- 172.21.15.67:0/165093878 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd630003800 con 0x7fd64006ba60 2023-10-14T10:36:01.993 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.982+0000 7fd6367fc700 1 --2- 172.21.15.67:0/165093878 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fd62406af00 0x7fd62406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:01.994 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.982+0000 7fd6367fc700 1 -- 172.21.15.67:0/165093878 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fd630089070 con 0x7fd64006ba60 2023-10-14T10:36:01.994 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.982+0000 7fd645701700 1 --2- 172.21.15.67:0/165093878 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fd62406af00 0x7fd62406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:01.994 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.983+0000 7fd645701700 1 --2- 172.21.15.67:0/165093878 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fd62406af00 0x7fd62406d3c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd63c009c40 tx=0x7fd63c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:01.994 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:01.987+0000 7fd6367fc700 1 -- 172.21.15.67:0/165093878 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd630054300 con 0x7fd64006ba60 2023-10-14T10:36:02.190 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:02.188+0000 7fd648166700 1 -- 172.21.15.67:0/165093878 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fd6400b44c0 con 0x7fd64006ba60 2023-10-14T10:36:02.827 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:02.825+0000 7fd6367fc700 1 -- 172.21.15.67:0/165093878 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v20) v1 ==== 170+0+0 (secure 0 0 0) 0x7fd6300094c0 con 0x7fd64006ba60 2023-10-14T10:36:02.830 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:02.828+0000 7fd648166700 1 -- 172.21.15.67:0/165093878 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fd62406af00 msgr2=0x7fd62406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:02.831 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:02.828+0000 7fd648166700 1 --2- 172.21.15.67:0/165093878 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fd62406af00 0x7fd62406d3c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd63c009c40 tx=0x7fd63c008030 comp rx=0 tx=0).stop 2023-10-14T10:36:02.831 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:02.828+0000 7fd648166700 1 -- 172.21.15.67:0/165093878 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd64006ba60 msgr2=0x7fd64014ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:02.831 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:02.828+0000 7fd648166700 1 --2- 172.21.15.67:0/165093878 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd64006ba60 0x7fd64014ff30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd63000e810 tx=0x7fd63000eb20 comp rx=0 tx=0).stop 2023-10-14T10:36:02.831 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:02.828+0000 7fd648166700 1 -- 172.21.15.67:0/165093878 shutdown_connections 2023-10-14T10:36:02.831 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:02.828+0000 7fd648166700 1 --2- 172.21.15.67:0/165093878 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fd62406af00 0x7fd62406d3c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:02.832 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:02.828+0000 7fd648166700 1 --2- 172.21.15.67:0/165093878 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fd6400ce550 0x7fd6400c9190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:02.832 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:02.828+0000 7fd648166700 1 --2- 172.21.15.67:0/165093878 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd64006ba60 0x7fd64014ff30 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:02.832 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:02.828+0000 7fd648166700 1 --2- 172.21.15.67:0/165093878 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd64006d390 0x7fd6400a84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:02.832 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:02.828+0000 7fd648166700 1 -- 172.21.15.67:0/165093878 >> 172.21.15.67:0/165093878 conn(0x7fd6400cfc40 msgr2=0x7fd6400aff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:02.832 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:02.828+0000 7fd648166700 1 -- 172.21.15.67:0/165093878 shutdown_connections 2023-10-14T10:36:02.832 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:02.828+0000 7fd648166700 1 -- 172.21.15.67:0/165093878 wait complete. 2023-10-14T10:36:02.833 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs_data' created 2023-10-14T10:36:02.855 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-14T10:36:02.988 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:02.986+0000 7f36e7467700 1 Processor -- start 2023-10-14T10:36:02.988 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:02.986+0000 7f36e7467700 1 -- start start 2023-10-14T10:36:02.991 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:02.987+0000 7f36e7467700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f36e015bb50 0x7f36e006e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:02.991 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:02.987+0000 7f36e7467700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36e015bf30 con 0x7f36e006ed50 2023-10-14T10:36:02.991 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:02.987+0000 7f36e7467700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36e006d500 con 0x7f36e00ce280 2023-10-14T10:36:02.992 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:02.987+0000 7f36e7467700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36e006f1d0 con 0x7f36e015bb50 2023-10-14T10:36:02.992 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:02.987+0000 7f36e4a02700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f36e006ed50 0x7f36e00cdb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:51534/0 (socket says 172.21.15.67:51534) 2023-10-14T10:36:02.992 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:02.987+0000 7f36e4a02700 1 -- 172.21.15.67:0/4258612649 learned_addr learned my addr 172.21.15.67:0/4258612649 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:36:02.992 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:02.987+0000 7f36e5203700 1 --2- 172.21.15.67:0/4258612649 >> v2:172.21.15.72:3300/0 conn(0x7f36e015bb50 0x7f36e006e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:02.992 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:02.988+0000 7f36d7fff700 1 -- 172.21.15.67:0/4258612649 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2220849320 0 0) 0x7f36e015bf30 con 0x7f36e006ed50 2023-10-14T10:36:02.993 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:02.988+0000 7f36d7fff700 1 -- 172.21.15.67:0/4258612649 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36cc0035b0 con 0x7f36e006ed50 2023-10-14T10:36:02.993 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:02.988+0000 7f36d7fff700 1 -- 172.21.15.67:0/4258612649 <== mon.1 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2461983858 0 0) 0x7f36e006d500 con 0x7f36e00ce280 2023-10-14T10:36:02.993 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:02.988+0000 7f36d7fff700 1 -- 172.21.15.67:0/4258612649 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36e015bf30 con 0x7f36e00ce280 2023-10-14T10:36:02.993 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:02.988+0000 7f36e5203700 1 -- 172.21.15.67:0/4258612649 >> v1:172.21.15.72:6789/0 conn(0x7f36e00ce280 legacy=0x7f36e006a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:02.994 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:02.988+0000 7f36e5203700 1 -- 172.21.15.67:0/4258612649 >> v1:172.21.15.67:6789/0 conn(0x7f36e006ed50 legacy=0x7f36e00cdb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:02.994 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:02.988+0000 7f36e5203700 1 -- 172.21.15.67:0/4258612649 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36e006d7e0 con 0x7f36e015bb50 2023-10-14T10:36:02.994 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:02.988+0000 7f36e5203700 1 --2- 172.21.15.67:0/4258612649 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f36e015bb50 0x7f36e006e810 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f36d0009840 tx=0x7f36d0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d2c2798c70885de server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:02.994 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:02.988+0000 7f36d7fff700 1 -- 172.21.15.67:0/4258612649 <== mon.1 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d0011830 con 0x7f36e015bb50 2023-10-14T10:36:02.995 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:02.988+0000 7f36d7fff700 1 -- 172.21.15.67:0/4258612649 <== mon.1 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36d0011990 con 0x7f36e015bb50 2023-10-14T10:36:02.995 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:02.988+0000 7f36d7fff700 1 -- 172.21.15.67:0/4258612649 <== mon.1 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d0011d00 con 0x7f36e015bb50 2023-10-14T10:36:02.995 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:02.989+0000 7f36e7467700 1 -- 172.21.15.67:0/4258612649 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f36e015bb50 msgr2=0x7f36e006e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:02.995 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:02.989+0000 7f36e7467700 1 --2- 172.21.15.67:0/4258612649 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f36e015bb50 0x7f36e006e810 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f36d0009840 tx=0x7f36d0009b50 comp rx=0 tx=0).stop 2023-10-14T10:36:02.995 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:02.989+0000 7f36e7467700 1 -- 172.21.15.67:0/4258612649 shutdown_connections 2023-10-14T10:36:02.996 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:02.989+0000 7f36e7467700 1 --2- 172.21.15.67:0/4258612649 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f36e015bb50 0x7f36e006e810 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:02.996 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:02.989+0000 7f36e7467700 1 -- 172.21.15.67:0/4258612649 >> 172.21.15.67:0/4258612649 conn(0x7f36e00cc6f0 msgr2=0x7f36e00ccb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:02.996 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:02.989+0000 7f36e7467700 1 -- 172.21.15.67:0/4258612649 shutdown_connections 2023-10-14T10:36:02.996 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:02.989+0000 7f36e7467700 1 -- 172.21.15.67:0/4258612649 wait complete. 2023-10-14T10:36:02.999 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:02.990+0000 7f36e7467700 1 Processor -- start 2023-10-14T10:36:03.000 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:02.990+0000 7f36e7467700 1 -- start start 2023-10-14T10:36:03.000 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:02.991+0000 7f36e7467700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f36e006ed50 0x7f36e01511b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:03.000 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:02.991+0000 7f36e7467700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f36e00ce280 0x7f36e0157cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:03.000 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:02.991+0000 7f36e7467700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f36e015bb50 0x7f36e015e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:03.000 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:02.991+0000 7f36e7467700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36e0152a30 con 0x7f36e015bb50 2023-10-14T10:36:03.001 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:02.991+0000 7f36e7467700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36e0152e50 con 0x7f36e00ce280 2023-10-14T10:36:03.001 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:02.991+0000 7f36e7467700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36e0151b60 con 0x7f36e006ed50 2023-10-14T10:36:03.001 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:02.991+0000 7f36e5a04700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f36e015bb50 0x7f36e015e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:03.001 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:02.991+0000 7f36e5a04700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f36e015bb50 0x7f36e015e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43774/0 (socket says 172.21.15.67:43774) 2023-10-14T10:36:03.001 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:02.991+0000 7f36e5a04700 1 -- 172.21.15.67:0/3713144526 learned_addr learned my addr 172.21.15.67:0/3713144526 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:03.002 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:02.991+0000 7f36e4a02700 1 --2- 172.21.15.67:0/3713144526 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f36e00ce280 0x7f36e0157cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:03.002 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:02.991+0000 7f36e5203700 1 --2- 172.21.15.67:0/3713144526 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f36e006ed50 0x7f36e01511b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:03.002 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:02.991+0000 7f36e5a04700 1 -- 172.21.15.67:0/3713144526 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f36e006ed50 msgr2=0x7f36e01511b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:03.002 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:02.991+0000 7f36e5a04700 1 --2- 172.21.15.67:0/3713144526 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f36e006ed50 0x7f36e01511b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:03.002 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:02.991+0000 7f36e5a04700 1 -- 172.21.15.67:0/3713144526 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f36e00ce280 msgr2=0x7f36e0157cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:03.003 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:02.991+0000 7f36e5a04700 1 --2- 172.21.15.67:0/3713144526 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f36e00ce280 0x7f36e0157cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:03.003 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:02.991+0000 7f36e5a04700 1 -- 172.21.15.67:0/3713144526 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36d00094a0 con 0x7f36e015bb50 2023-10-14T10:36:03.003 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:02.991+0000 7f36e4a02700 1 --2- 172.21.15.67:0/3713144526 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f36e00ce280 0x7f36e0157cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-14T10:36:03.003 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:02.991+0000 7f36e5a04700 1 --2- 172.21.15.67:0/3713144526 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f36e015bb50 0x7f36e015e3b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f36d800cc50 tx=0x7f36d800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:03.003 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:02.992+0000 7f36d5ffb700 1 -- 172.21.15.67:0/3713144526 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d800ada0 con 0x7f36e015bb50 2023-10-14T10:36:03.004 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:02.992+0000 7f36d5ffb700 1 -- 172.21.15.67:0/3713144526 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36d800ce50 con 0x7f36e015bb50 2023-10-14T10:36:03.004 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:02.992+0000 7f36d5ffb700 1 -- 172.21.15.67:0/3713144526 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d8003c40 con 0x7f36e015bb50 2023-10-14T10:36:03.004 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:02.992+0000 7f36e7467700 1 -- 172.21.15.67:0/3713144526 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36e01516f0 con 0x7f36e015bb50 2023-10-14T10:36:03.004 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:02.992+0000 7f36e7467700 1 -- 172.21.15.67:0/3713144526 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36e0152090 con 0x7f36e015bb50 2023-10-14T10:36:03.004 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:02.993+0000 7f36e7467700 1 -- 172.21.15.67:0/3713144526 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36e0152b70 con 0x7f36e015bb50 2023-10-14T10:36:03.005 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:02.993+0000 7f36d5ffb700 1 -- 172.21.15.67:0/3713144526 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f36d80050d0 con 0x7f36e015bb50 2023-10-14T10:36:03.005 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:02.993+0000 7f36d5ffb700 1 --2- 172.21.15.67:0/3713144526 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f36c406af00 0x7f36c406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:03.005 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:02.994+0000 7f36d5ffb700 1 -- 172.21.15.67:0/3713144526 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f36d8089770 con 0x7f36e015bb50 2023-10-14T10:36:03.005 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:02.994+0000 7f36e5203700 1 --2- 172.21.15.67:0/3713144526 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f36c406af00 0x7f36c406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:03.005 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:02.994+0000 7f36e5203700 1 --2- 172.21.15.67:0/3713144526 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f36c406af00 0x7f36c406d3c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f36d0009fd0 tx=0x7f36d0008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:03.006 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:02.998+0000 7f36d5ffb700 1 -- 172.21.15.67:0/3713144526 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f36e0152b70 con 0x7f36e015bb50 2023-10-14T10:36:03.248 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:03.246+0000 7f36e7467700 1 -- 172.21.15.67:0/3713144526 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f36e0152b70 con 0x7f36e015bb50 2023-10-14T10:36:03.838 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:03.836+0000 7f36d5ffb700 1 -- 172.21.15.67:0/3713144526 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f36e0152b70 con 0x7f36e015bb50 2023-10-14T10:36:03.841 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:03.839+0000 7f36e7467700 1 -- 172.21.15.67:0/3713144526 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f36c406af00 msgr2=0x7f36c406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:03.841 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:03.839+0000 7f36e7467700 1 --2- 172.21.15.67:0/3713144526 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f36c406af00 0x7f36c406d3c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f36d0009fd0 tx=0x7f36d0008030 comp rx=0 tx=0).stop 2023-10-14T10:36:03.841 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:03.839+0000 7f36e7467700 1 -- 172.21.15.67:0/3713144526 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f36e015bb50 msgr2=0x7f36e015e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:03.842 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:03.839+0000 7f36e7467700 1 --2- 172.21.15.67:0/3713144526 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f36e015bb50 0x7f36e015e3b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f36d800cc50 tx=0x7f36d800a340 comp rx=0 tx=0).stop 2023-10-14T10:36:03.842 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:03.840+0000 7f36e7467700 1 -- 172.21.15.67:0/3713144526 shutdown_connections 2023-10-14T10:36:03.843 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:03.840+0000 7f36e7467700 1 --2- 172.21.15.67:0/3713144526 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f36c406af00 0x7f36c406d3c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:03.843 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:03.840+0000 7f36e7467700 1 --2- 172.21.15.67:0/3713144526 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f36e015bb50 0x7f36e015e3b0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:03.843 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:03.840+0000 7f36e7467700 1 --2- 172.21.15.67:0/3713144526 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f36e006ed50 0x7f36e01511b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:03.843 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:03.840+0000 7f36e7467700 1 --2- 172.21.15.67:0/3713144526 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f36e00ce280 0x7f36e0157cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:03.843 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:03.840+0000 7f36e7467700 1 -- 172.21.15.67:0/3713144526 >> 172.21.15.67:0/3713144526 conn(0x7f36e00cc6f0 msgr2=0x7f36e006a840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:03.844 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:03.840+0000 7f36e7467700 1 -- 172.21.15.67:0/3713144526 shutdown_connections 2023-10-14T10:36:03.844 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:03.841+0000 7f36e7467700 1 -- 172.21.15.67:0/3713144526 wait complete. 2023-10-14T10:36:03.847 INFO:teuthology.orchestra.run.smithi067.stderr:new fs with metadata pool 2 and data pool 3 2023-10-14T10:36:03.872 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-14T10:36:04.002 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.000+0000 7f404135b700 1 Processor -- start 2023-10-14T10:36:04.002 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.000+0000 7f404135b700 1 -- start start 2023-10-14T10:36:04.005 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.001+0000 7f404135b700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f403c0d3fc0 0x7f403c15dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:04.005 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.001+0000 7f404135b700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f403c0b7050 0x7f403c0d21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:04.005 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.001+0000 7f404135b700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f403c15e4e0 con 0x7f403c15dbc0 2023-10-14T10:36:04.005 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.001+0000 7f404135b700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f403c0b7560 con 0x7f403c0b7050 2023-10-14T10:36:04.005 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.001+0000 7f404135b700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f403c0b7840 con 0x7f403c0d3fc0 2023-10-14T10:36:04.006 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.001+0000 7f403b7fe700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f403c0b7050 0x7f403c0d21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:04.006 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.001+0000 7f403b7fe700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f403c0b7050 0x7f403c0d21d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43782/0 (socket says 172.21.15.67:43782) 2023-10-14T10:36:04.006 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.001+0000 7f403b7fe700 1 -- 172.21.15.67:0/49085852 learned_addr learned my addr 172.21.15.67:0/49085852 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:04.006 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.001+0000 7f403a7fc700 1 --2- 172.21.15.67:0/49085852 >> v2:172.21.15.72:3300/0 conn(0x7f403c0d3fc0 0x7f403c15dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:04.007 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.001+0000 7f403b7fe700 1 -- 172.21.15.67:0/49085852 >> v2:172.21.15.72:3300/0 conn(0x7f403c0d3fc0 msgr2=0x7f403c15dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:04.007 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.001+0000 7f403b7fe700 1 --2- 172.21.15.67:0/49085852 >> v2:172.21.15.72:3300/0 conn(0x7f403c0d3fc0 0x7f403c15dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:04.007 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.001+0000 7f403b7fe700 1 -- 172.21.15.67:0/49085852 >> v1:172.21.15.72:6789/0 conn(0x7f403c15dbc0 legacy=0x7f403c0d1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:04.007 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.001+0000 7f403b7fe700 1 -- 172.21.15.67:0/49085852 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f403c0b7b20 con 0x7f403c0b7050 2023-10-14T10:36:04.007 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.001+0000 7f403b7fe700 1 --2- 172.21.15.67:0/49085852 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f403c0b7050 0x7f403c0d21d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f403c0d14e0 tx=0x7f403400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48ceb22cd0ed97a4 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:04.008 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.001+0000 7f4039ffb700 1 -- 172.21.15.67:0/49085852 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f403400e070 con 0x7f403c0b7050 2023-10-14T10:36:04.008 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.001+0000 7f4039ffb700 1 -- 172.21.15.67:0/49085852 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f403400b9c0 con 0x7f403c0b7050 2023-10-14T10:36:04.008 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.001+0000 7f4039ffb700 1 -- 172.21.15.67:0/49085852 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f403400bd70 con 0x7f403c0b7050 2023-10-14T10:36:04.008 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.002+0000 7f404135b700 1 -- 172.21.15.67:0/49085852 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f403c0b7050 msgr2=0x7f403c0d21d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:04.008 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.002+0000 7f404135b700 1 --2- 172.21.15.67:0/49085852 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f403c0b7050 0x7f403c0d21d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f403c0d14e0 tx=0x7f403400b330 comp rx=0 tx=0).stop 2023-10-14T10:36:04.008 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.002+0000 7f404135b700 1 -- 172.21.15.67:0/49085852 shutdown_connections 2023-10-14T10:36:04.009 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.002+0000 7f404135b700 1 --2- 172.21.15.67:0/49085852 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f403c0b7050 0x7f403c0d21d0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:04.009 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.002+0000 7f404135b700 1 --2- 172.21.15.67:0/49085852 >> v2:172.21.15.72:3300/0 conn(0x7f403c0d3fc0 0x7f403c15dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:04.009 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.002+0000 7f404135b700 1 -- 172.21.15.67:0/49085852 >> 172.21.15.67:0/49085852 conn(0x7f403c09cb40 msgr2=0x7f403c0d3820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:04.009 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.002+0000 7f404135b700 1 -- 172.21.15.67:0/49085852 shutdown_connections 2023-10-14T10:36:04.009 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.002+0000 7f404135b700 1 -- 172.21.15.67:0/49085852 wait complete. 2023-10-14T10:36:04.012 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.003+0000 7f404135b700 1 Processor -- start 2023-10-14T10:36:04.012 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.003+0000 7f404135b700 1 -- start start 2023-10-14T10:36:04.012 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.004+0000 7f404135b700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f403c0b7050 0x7f403c09fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:04.013 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.004+0000 7f404135b700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f403c0d3fc0 0x7f403c14ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:04.013 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.004+0000 7f404135b700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f403c15dbc0 0x7f403c0ce5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:04.013 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.004+0000 7f404135b700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f403c15b0a0 con 0x7f403c0d3fc0 2023-10-14T10:36:04.013 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.004+0000 7f404135b700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f403c0a0260 con 0x7f403c15dbc0 2023-10-14T10:36:04.013 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.004+0000 7f404135b700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f403c0a0540 con 0x7f403c0b7050 2023-10-14T10:36:04.014 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.004+0000 7f403a7fc700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f403c0d3fc0 0x7f403c14ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:04.014 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.004+0000 7f403a7fc700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f403c0d3fc0 0x7f403c14ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43786/0 (socket says 172.21.15.67:43786) 2023-10-14T10:36:04.014 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.004+0000 7f403a7fc700 1 -- 172.21.15.67:0/510351867 learned_addr learned my addr 172.21.15.67:0/510351867 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:04.014 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.004+0000 7f403affd700 1 --2- 172.21.15.67:0/510351867 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f403c0b7050 0x7f403c09fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:04.014 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.004+0000 7f403b7fe700 1 --2- 172.21.15.67:0/510351867 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f403c15dbc0 0x7f403c0ce5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:04.015 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.005+0000 7f403a7fc700 1 -- 172.21.15.67:0/510351867 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f403c0b7050 msgr2=0x7f403c09fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:04.015 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.005+0000 7f403a7fc700 1 --2- 172.21.15.67:0/510351867 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f403c0b7050 0x7f403c09fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:04.015 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.005+0000 7f403a7fc700 1 -- 172.21.15.67:0/510351867 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f403c15dbc0 msgr2=0x7f403c0ce5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:04.015 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.005+0000 7f403a7fc700 1 --2- 172.21.15.67:0/510351867 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f403c15dbc0 0x7f403c0ce5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:04.015 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.005+0000 7f403a7fc700 1 -- 172.21.15.67:0/510351867 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4034009b30 con 0x7f403c0d3fc0 2023-10-14T10:36:04.015 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.005+0000 7f403a7fc700 1 --2- 172.21.15.67:0/510351867 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f403c0d3fc0 0x7f403c14ec00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f402c00e810 tx=0x7f402c00eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:04.016 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.005+0000 7f40237fe700 1 -- 172.21.15.67:0/510351867 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f402c00c070 con 0x7f403c0d3fc0 2023-10-14T10:36:04.016 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.005+0000 7f40237fe700 1 -- 172.21.15.67:0/510351867 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f402c013610 con 0x7f403c0d3fc0 2023-10-14T10:36:04.016 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.005+0000 7f404135b700 1 -- 172.21.15.67:0/510351867 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f403c0a0820 con 0x7f403c0d3fc0 2023-10-14T10:36:04.016 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.005+0000 7f40237fe700 1 -- 172.21.15.67:0/510351867 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f402c0139c0 con 0x7f403c0d3fc0 2023-10-14T10:36:04.016 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.005+0000 7f404135b700 1 -- 172.21.15.67:0/510351867 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f403c0a0de0 con 0x7f403c0d3fc0 2023-10-14T10:36:04.017 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.006+0000 7f404135b700 1 -- 172.21.15.67:0/510351867 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f403c0a1150 con 0x7f403c0d3fc0 2023-10-14T10:36:04.017 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.006+0000 7f40237fe700 1 -- 172.21.15.67:0/510351867 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f402c013b20 con 0x7f403c0d3fc0 2023-10-14T10:36:04.017 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.007+0000 7f40237fe700 1 --2- 172.21.15.67:0/510351867 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f402806af00 0x7f402806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:04.017 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.007+0000 7f40237fe700 1 -- 172.21.15.67:0/510351867 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f402c0892e0 con 0x7f403c0d3fc0 2023-10-14T10:36:04.018 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.007+0000 7f403affd700 1 --2- 172.21.15.67:0/510351867 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f402806af00 0x7f402806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:04.018 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.007+0000 7f403affd700 1 --2- 172.21.15.67:0/510351867 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f402806af00 0x7f402806d3c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4030000fb0 tx=0x7f4030009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:04.018 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.011+0000 7f40237fe700 1 -- 172.21.15.67:0/510351867 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f402c0543a0 con 0x7f403c0d3fc0 2023-10-14T10:36:04.214 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.212+0000 7f404135b700 1 -- 172.21.15.67:0/510351867 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f403c04ebb0 con 0x7f403c0d3fc0 2023-10-14T10:36:04.215 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.213+0000 7f40237fe700 1 -- 172.21.15.67:0/510351867 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16470 (secure 0 0 0) 0x7f402c0579c0 con 0x7f403c0d3fc0 2023-10-14T10:36:04.216 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-14T10:36:04.216 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":21,"fsid":"37f9ab46-3082-4b71-8b24-08d39897b008","created":"2023-10-14T10:35:46.067554+0000","modified":"2023-10-14T10:36:03.247589+0000","last_up_change":"2023-10-14T10:35:56.668142+0000","last_in_change":"2023-10-14T10:35:48.551490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-14T10:35:57.809566+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-14T10:36:01.667146+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-14T10:36:02.219700+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2b59a07e-11ef-440c-b766-9f64ac885838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6801","nonce":1287255499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6803","nonce":1287255499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6807","nonce":1287255499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6805","nonce":1287255499}]},"public_addr":"172.21.15.67:6801/1287255499","cluster_addr":"172.21.15.67:6803/1287255499","heartbeat_back_addr":"172.21.15.67:6807/1287255499","heartbeat_front_addr":"172.21.15.67:6805/1287255499","state":["exists","up"]},{"osd":1,"uuid":"6f813bd6-fa0b-487d-a204-431eb1276a60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6818","nonce":2435538787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6822","nonce":2435538787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6830","nonce":2435538787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6826","nonce":2435538787}]},"public_addr":"172.21.15.67:6818/2435538787","cluster_addr":"172.21.15.67:6822/2435538787","heartbeat_back_addr":"172.21.15.67:6830/2435538787","heartbeat_front_addr":"172.21.15.67:6826/2435538787","state":["exists","up"]},{"osd":2,"uuid":"6af42c6a-1b59-4669-9c04-b6876788446d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6809","nonce":2827491791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6811","nonce":2827491791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6815","nonce":2827491791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6813","nonce":2827491791}]},"public_addr":"172.21.15.67:6809/2827491791","cluster_addr":"172.21.15.67:6811/2827491791","heartbeat_back_addr":"172.21.15.67:6815/2827491791","heartbeat_front_addr":"172.21.15.67:6813/2827491791","state":["exists","up"]},{"osd":3,"uuid":"1d889d76-a646-499d-bfff-ed5bae5434ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6819","nonce":1848001176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6821","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6823","nonce":1848001176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6831","nonce":1848001176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6825","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6827","nonce":1848001176}]},"public_addr":"172.21.15.67:6819/1848001176","cluster_addr":"172.21.15.67:6823/1848001176","heartbeat_back_addr":"172.21.15.67:6831/1848001176","heartbeat_front_addr":"172.21.15.67:6827/1848001176","state":["exists","up"]},{"osd":4,"uuid":"c3271dff-b437-4fdd-b13b-6e49a35473af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6804","nonce":1891230785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6808","nonce":1891230785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6815","nonce":1891230785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6811","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6813","nonce":1891230785}]},"public_addr":"172.21.15.72:6804/1891230785","cluster_addr":"172.21.15.72:6808/1891230785","heartbeat_back_addr":"172.21.15.72:6815/1891230785","heartbeat_front_addr":"172.21.15.72:6813/1891230785","state":["exists","up"]},{"osd":5,"uuid":"0337bb18-4b68-4eff-b528-a667275e6926","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6801","nonce":569686991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6803","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6805","nonce":569686991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6812","nonce":569686991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6807","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6809","nonce":569686991}]},"public_addr":"172.21.15.72:6801/569686991","cluster_addr":"172.21.15.72:6805/569686991","heartbeat_back_addr":"172.21.15.72:6812/569686991","heartbeat_front_addr":"172.21.15.72:6809/569686991","state":["exists","up"]},{"osd":6,"uuid":"8e92d9cf-0c8c-46eb-89a7-db2023ef1509","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6817","nonce":3767685934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6819","nonce":3767685934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6823","nonce":3767685934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6821","nonce":3767685934}]},"public_addr":"172.21.15.72:6817/3767685934","cluster_addr":"172.21.15.72:6819/3767685934","heartbeat_back_addr":"172.21.15.72:6823/3767685934","heartbeat_front_addr":"172.21.15.72:6821/3767685934","state":["exists","up"]},{"osd":7,"uuid":"3fd6aa7d-50a9-45c9-9561-5902a044a119","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6825","nonce":3567465926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6827","nonce":3567465926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6831","nonce":3567465926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6829","nonce":3567465926}]},"public_addr":"172.21.15.72:6825/3567465926","cluster_addr":"172.21.15.72:6827/3567465926","heartbeat_back_addr":"172.21.15.72:6831/3567465926","heartbeat_front_addr":"172.21.15.72:6829/3567465926","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.511466+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.795107+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.858674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.728895+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.794842+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.750064+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.732122+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.765185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-14T10:36:04.217 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.215+0000 7f404135b700 1 -- 172.21.15.67:0/510351867 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f402806af00 msgr2=0x7f402806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:04.218 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.215+0000 7f404135b700 1 --2- 172.21.15.67:0/510351867 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f402806af00 0x7f402806d3c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4030000fb0 tx=0x7f4030009040 comp rx=0 tx=0).stop 2023-10-14T10:36:04.218 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.215+0000 7f404135b700 1 -- 172.21.15.67:0/510351867 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f403c0d3fc0 msgr2=0x7f403c14ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:04.218 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.215+0000 7f404135b700 1 --2- 172.21.15.67:0/510351867 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f403c0d3fc0 0x7f403c14ec00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f402c00e810 tx=0x7f402c00eb20 comp rx=0 tx=0).stop 2023-10-14T10:36:04.218 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.215+0000 7f404135b700 1 -- 172.21.15.67:0/510351867 shutdown_connections 2023-10-14T10:36:04.219 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.215+0000 7f404135b700 1 --2- 172.21.15.67:0/510351867 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f402806af00 0x7f402806d3c0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:04.219 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.215+0000 7f404135b700 1 --2- 172.21.15.67:0/510351867 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f403c15dbc0 0x7f403c0ce5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:04.219 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.215+0000 7f404135b700 1 --2- 172.21.15.67:0/510351867 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f403c0b7050 0x7f403c09fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:04.219 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.215+0000 7f404135b700 1 --2- 172.21.15.67:0/510351867 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f403c0d3fc0 0x7f403c14ec00 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:04.219 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.215+0000 7f404135b700 1 -- 172.21.15.67:0/510351867 >> 172.21.15.67:0/510351867 conn(0x7f403c09cb40 msgr2=0x7f403c0b6190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:04.220 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.215+0000 7f404135b700 1 -- 172.21.15.67:0/510351867 shutdown_connections 2023-10-14T10:36:04.220 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.215+0000 7f404135b700 1 -- 172.21.15.67:0/510351867 wait complete. 2023-10-14T10:36:04.230 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-14T10:36:04.354 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.352+0000 7fce954b7700 1 Processor -- start 2023-10-14T10:36:04.355 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.352+0000 7fce954b7700 1 -- start start 2023-10-14T10:36:04.357 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.353+0000 7fce954b7700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fce9015eab0 0x7fce900cf190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:04.357 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.353+0000 7fce954b7700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fce900d1d90 0x7fce900cbc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:04.357 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.353+0000 7fce954b7700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce900d2170 con 0x7fce900cf6d0 2023-10-14T10:36:04.358 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.353+0000 7fce954b7700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce900cfb60 con 0x7fce9015eab0 2023-10-14T10:36:04.358 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.353+0000 7fce954b7700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce9015ee90 con 0x7fce900d1d90 2023-10-14T10:36:04.358 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.353+0000 7fce8effd700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fce9015eab0 0x7fce900cf190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:04.358 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.353+0000 7fce8e7fc700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fce900cf6d0 0x7fce900d1680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:51562/0 (socket says 172.21.15.67:51562) 2023-10-14T10:36:04.358 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.353+0000 7fce8effd700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fce9015eab0 0x7fce900cf190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43792/0 (socket says 172.21.15.67:43792) 2023-10-14T10:36:04.358 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.353+0000 7fce8e7fc700 1 -- 172.21.15.67:0/1441130850 learned_addr learned my addr 172.21.15.67:0/1441130850 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:36:04.359 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.353+0000 7fce8f7fe700 1 --2- 172.21.15.67:0/1441130850 >> v2:172.21.15.72:3300/0 conn(0x7fce900d1d90 0x7fce900cbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:04.359 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.353+0000 7fce8effd700 1 -- 172.21.15.67:0/1441130850 >> v2:172.21.15.72:3300/0 conn(0x7fce900d1d90 msgr2=0x7fce900cbc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:04.359 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.353+0000 7fce8effd700 1 --2- 172.21.15.67:0/1441130850 >> v2:172.21.15.72:3300/0 conn(0x7fce900d1d90 0x7fce900cbc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:04.359 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.353+0000 7fce8effd700 1 -- 172.21.15.67:0/1441130850 >> v1:172.21.15.67:6789/0 conn(0x7fce900cf6d0 legacy=0x7fce900d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:04.359 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.353+0000 7fce8dffb700 1 -- 172.21.15.67:0/1441130850 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2351536876 0 0) 0x7fce80001110 con 0x7fce900cf6d0 2023-10-14T10:36:04.360 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.353+0000 7fce8effd700 1 -- 172.21.15.67:0/1441130850 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce90158370 con 0x7fce9015eab0 2023-10-14T10:36:04.360 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.353+0000 7fce8effd700 1 --2- 172.21.15.67:0/1441130850 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fce9015eab0 0x7fce900cf190 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fce900ccfa0 tx=0x7fce84009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8e5992e4fc69f9 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:04.360 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.354+0000 7fce8dffb700 1 -- 172.21.15.67:0/1441130850 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce84011830 con 0x7fce9015eab0 2023-10-14T10:36:04.360 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.354+0000 7fce8dffb700 1 -- 172.21.15.67:0/1441130850 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce84011990 con 0x7fce9015eab0 2023-10-14T10:36:04.360 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.354+0000 7fce8dffb700 1 -- 172.21.15.67:0/1441130850 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce84011d20 con 0x7fce9015eab0 2023-10-14T10:36:04.361 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.354+0000 7fce954b7700 1 -- 172.21.15.67:0/1441130850 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fce9015eab0 msgr2=0x7fce900cf190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:04.361 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.354+0000 7fce954b7700 1 --2- 172.21.15.67:0/1441130850 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fce9015eab0 0x7fce900cf190 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fce900ccfa0 tx=0x7fce84009b20 comp rx=0 tx=0).stop 2023-10-14T10:36:04.361 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.354+0000 7fce954b7700 1 -- 172.21.15.67:0/1441130850 shutdown_connections 2023-10-14T10:36:04.361 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.354+0000 7fce954b7700 1 --2- 172.21.15.67:0/1441130850 >> v2:172.21.15.72:3300/0 conn(0x7fce900d1d90 0x7fce900cbc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:04.361 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.354+0000 7fce954b7700 1 --2- 172.21.15.67:0/1441130850 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fce9015eab0 0x7fce900cf190 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:04.361 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.354+0000 7fce954b7700 1 -- 172.21.15.67:0/1441130850 >> 172.21.15.67:0/1441130850 conn(0x7fce900665a0 msgr2=0x7fce90066980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:04.362 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.354+0000 7fce954b7700 1 -- 172.21.15.67:0/1441130850 shutdown_connections 2023-10-14T10:36:04.362 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.354+0000 7fce954b7700 1 -- 172.21.15.67:0/1441130850 wait complete. 2023-10-14T10:36:04.365 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.356+0000 7fce954b7700 1 Processor -- start 2023-10-14T10:36:04.365 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.356+0000 7fce954b7700 1 -- start start 2023-10-14T10:36:04.365 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.356+0000 7fce954b7700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fce900cf6d0 0x7fce900c9070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:04.365 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.356+0000 7fce954b7700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fce900d1d90 0x7fce900c95b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:04.366 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.356+0000 7fce954b7700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fce9015eab0 0x7fce901f3090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:04.366 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.356+0000 7fce954b7700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce90157fa0 con 0x7fce900d1d90 2023-10-14T10:36:04.366 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.356+0000 7fce954b7700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce900ac8b0 con 0x7fce9015eab0 2023-10-14T10:36:04.366 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.356+0000 7fce954b7700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce900acb90 con 0x7fce900cf6d0 2023-10-14T10:36:04.366 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.357+0000 7fce8e7fc700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fce900d1d90 0x7fce900c95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:04.367 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.357+0000 7fce8e7fc700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fce900d1d90 0x7fce900c95b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43800/0 (socket says 172.21.15.67:43800) 2023-10-14T10:36:04.367 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.357+0000 7fce8e7fc700 1 -- 172.21.15.67:0/3566922223 learned_addr learned my addr 172.21.15.67:0/3566922223 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:04.367 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.357+0000 7fce8effd700 1 --2- 172.21.15.67:0/3566922223 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fce900cf6d0 0x7fce900c9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:04.367 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.357+0000 7fce8f7fe700 1 --2- 172.21.15.67:0/3566922223 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fce9015eab0 0x7fce901f3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:04.367 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.357+0000 7fce8e7fc700 1 -- 172.21.15.67:0/3566922223 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fce900cf6d0 msgr2=0x7fce900c9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:04.368 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.357+0000 7fce8e7fc700 1 --2- 172.21.15.67:0/3566922223 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fce900cf6d0 0x7fce900c9070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:04.368 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.357+0000 7fce8e7fc700 1 -- 172.21.15.67:0/3566922223 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fce9015eab0 msgr2=0x7fce901f3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:04.368 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.357+0000 7fce8e7fc700 1 --2- 172.21.15.67:0/3566922223 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fce9015eab0 0x7fce901f3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:04.368 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.357+0000 7fce8e7fc700 1 -- 172.21.15.67:0/3566922223 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce840094a0 con 0x7fce900d1d90 2023-10-14T10:36:04.368 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.357+0000 7fce8effd700 1 --2- 172.21.15.67:0/3566922223 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fce900cf6d0 0x7fce900c9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-14T10:36:04.369 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.357+0000 7fce8e7fc700 1 --2- 172.21.15.67:0/3566922223 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fce900d1d90 0x7fce900c95b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fce8000c550 tx=0x7fce8000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:04.369 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.357+0000 7fce777fe700 1 -- 172.21.15.67:0/3566922223 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce80004c80 con 0x7fce900d1d90 2023-10-14T10:36:04.369 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.357+0000 7fce777fe700 1 -- 172.21.15.67:0/3566922223 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce80004e00 con 0x7fce900d1d90 2023-10-14T10:36:04.369 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.357+0000 7fce954b7700 1 -- 172.21.15.67:0/3566922223 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce900ccaa0 con 0x7fce900d1d90 2023-10-14T10:36:04.369 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.357+0000 7fce777fe700 1 -- 172.21.15.67:0/3566922223 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce80007310 con 0x7fce900d1d90 2023-10-14T10:36:04.370 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.357+0000 7fce954b7700 1 -- 172.21.15.67:0/3566922223 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce900cbcf0 con 0x7fce900d1d90 2023-10-14T10:36:04.370 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.358+0000 7fce954b7700 1 -- 172.21.15.67:0/3566922223 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce90067d20 con 0x7fce900d1d90 2023-10-14T10:36:04.370 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.359+0000 7fce777fe700 1 -- 172.21.15.67:0/3566922223 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fce80007470 con 0x7fce900d1d90 2023-10-14T10:36:04.370 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.359+0000 7fce777fe700 1 --2- 172.21.15.67:0/3566922223 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fce7c06af00 0x7fce7c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:04.370 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.359+0000 7fce777fe700 1 -- 172.21.15.67:0/3566922223 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fce80089800 con 0x7fce900d1d90 2023-10-14T10:36:04.370 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.360+0000 7fce8effd700 1 --2- 172.21.15.67:0/3566922223 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fce7c06af00 0x7fce7c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:04.371 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.360+0000 7fce8effd700 1 --2- 172.21.15.67:0/3566922223 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fce7c06af00 0x7fce7c06d3c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fce840090b0 tx=0x7fce84008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:04.371 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.363+0000 7fce777fe700 1 -- 172.21.15.67:0/3566922223 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fce80054920 con 0x7fce900d1d90 2023-10-14T10:36:04.565 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.563+0000 7fce954b7700 1 -- 172.21.15.67:0/3566922223 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fce900d2170 con 0x7fce900d1d90 2023-10-14T10:36:04.566 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.565+0000 7fce777fe700 1 -- 172.21.15.67:0/3566922223 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16470 (secure 0 0 0) 0x7fce900d2170 con 0x7fce900d1d90 2023-10-14T10:36:04.567 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-14T10:36:04.567 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":21,"fsid":"37f9ab46-3082-4b71-8b24-08d39897b008","created":"2023-10-14T10:35:46.067554+0000","modified":"2023-10-14T10:36:03.247589+0000","last_up_change":"2023-10-14T10:35:56.668142+0000","last_in_change":"2023-10-14T10:35:48.551490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-14T10:35:57.809566+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-14T10:36:01.667146+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-14T10:36:02.219700+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2b59a07e-11ef-440c-b766-9f64ac885838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6801","nonce":1287255499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6803","nonce":1287255499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6807","nonce":1287255499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6805","nonce":1287255499}]},"public_addr":"172.21.15.67:6801/1287255499","cluster_addr":"172.21.15.67:6803/1287255499","heartbeat_back_addr":"172.21.15.67:6807/1287255499","heartbeat_front_addr":"172.21.15.67:6805/1287255499","state":["exists","up"]},{"osd":1,"uuid":"6f813bd6-fa0b-487d-a204-431eb1276a60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6818","nonce":2435538787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6822","nonce":2435538787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6830","nonce":2435538787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6826","nonce":2435538787}]},"public_addr":"172.21.15.67:6818/2435538787","cluster_addr":"172.21.15.67:6822/2435538787","heartbeat_back_addr":"172.21.15.67:6830/2435538787","heartbeat_front_addr":"172.21.15.67:6826/2435538787","state":["exists","up"]},{"osd":2,"uuid":"6af42c6a-1b59-4669-9c04-b6876788446d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6809","nonce":2827491791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6811","nonce":2827491791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6815","nonce":2827491791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6813","nonce":2827491791}]},"public_addr":"172.21.15.67:6809/2827491791","cluster_addr":"172.21.15.67:6811/2827491791","heartbeat_back_addr":"172.21.15.67:6815/2827491791","heartbeat_front_addr":"172.21.15.67:6813/2827491791","state":["exists","up"]},{"osd":3,"uuid":"1d889d76-a646-499d-bfff-ed5bae5434ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6819","nonce":1848001176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6821","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6823","nonce":1848001176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6831","nonce":1848001176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6825","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6827","nonce":1848001176}]},"public_addr":"172.21.15.67:6819/1848001176","cluster_addr":"172.21.15.67:6823/1848001176","heartbeat_back_addr":"172.21.15.67:6831/1848001176","heartbeat_front_addr":"172.21.15.67:6827/1848001176","state":["exists","up"]},{"osd":4,"uuid":"c3271dff-b437-4fdd-b13b-6e49a35473af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6804","nonce":1891230785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6808","nonce":1891230785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6815","nonce":1891230785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6811","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6813","nonce":1891230785}]},"public_addr":"172.21.15.72:6804/1891230785","cluster_addr":"172.21.15.72:6808/1891230785","heartbeat_back_addr":"172.21.15.72:6815/1891230785","heartbeat_front_addr":"172.21.15.72:6813/1891230785","state":["exists","up"]},{"osd":5,"uuid":"0337bb18-4b68-4eff-b528-a667275e6926","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6801","nonce":569686991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6803","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6805","nonce":569686991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6812","nonce":569686991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6807","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6809","nonce":569686991}]},"public_addr":"172.21.15.72:6801/569686991","cluster_addr":"172.21.15.72:6805/569686991","heartbeat_back_addr":"172.21.15.72:6812/569686991","heartbeat_front_addr":"172.21.15.72:6809/569686991","state":["exists","up"]},{"osd":6,"uuid":"8e92d9cf-0c8c-46eb-89a7-db2023ef1509","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6817","nonce":3767685934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6819","nonce":3767685934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6823","nonce":3767685934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6821","nonce":3767685934}]},"public_addr":"172.21.15.72:6817/3767685934","cluster_addr":"172.21.15.72:6819/3767685934","heartbeat_back_addr":"172.21.15.72:6823/3767685934","heartbeat_front_addr":"172.21.15.72:6821/3767685934","state":["exists","up"]},{"osd":7,"uuid":"3fd6aa7d-50a9-45c9-9561-5902a044a119","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6825","nonce":3567465926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6827","nonce":3567465926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6831","nonce":3567465926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6829","nonce":3567465926}]},"public_addr":"172.21.15.72:6825/3567465926","cluster_addr":"172.21.15.72:6827/3567465926","heartbeat_back_addr":"172.21.15.72:6831/3567465926","heartbeat_front_addr":"172.21.15.72:6829/3567465926","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.511466+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.795107+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.858674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.728895+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.794842+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.750064+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.732122+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.765185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-14T10:36:04.569 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.567+0000 7fce954b7700 1 -- 172.21.15.67:0/3566922223 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fce7c06af00 msgr2=0x7fce7c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:04.569 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.567+0000 7fce954b7700 1 --2- 172.21.15.67:0/3566922223 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fce7c06af00 0x7fce7c06d3c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fce840090b0 tx=0x7fce84008030 comp rx=0 tx=0).stop 2023-10-14T10:36:04.570 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.567+0000 7fce954b7700 1 -- 172.21.15.67:0/3566922223 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fce900d1d90 msgr2=0x7fce900c95b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:04.570 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.567+0000 7fce954b7700 1 --2- 172.21.15.67:0/3566922223 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fce900d1d90 0x7fce900c95b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fce8000c550 tx=0x7fce8000c910 comp rx=0 tx=0).stop 2023-10-14T10:36:04.570 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.567+0000 7fce954b7700 1 -- 172.21.15.67:0/3566922223 shutdown_connections 2023-10-14T10:36:04.570 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.567+0000 7fce954b7700 1 --2- 172.21.15.67:0/3566922223 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fce7c06af00 0x7fce7c06d3c0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:04.570 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.567+0000 7fce954b7700 1 --2- 172.21.15.67:0/3566922223 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fce9015eab0 0x7fce901f3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:04.571 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.567+0000 7fce954b7700 1 --2- 172.21.15.67:0/3566922223 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fce900cf6d0 0x7fce900c9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:04.571 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.567+0000 7fce954b7700 1 --2- 172.21.15.67:0/3566922223 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fce900d1d90 0x7fce900c95b0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:04.571 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.567+0000 7fce954b7700 1 -- 172.21.15.67:0/3566922223 >> 172.21.15.67:0/3566922223 conn(0x7fce900665a0 msgr2=0x7fce900b27c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:04.571 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.567+0000 7fce954b7700 1 -- 172.21.15.67:0/3566922223 shutdown_connections 2023-10-14T10:36:04.571 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.567+0000 7fce954b7700 1 -- 172.21.15.67:0/3566922223 wait complete. 2023-10-14T10:36:04.581 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-14T10:36:04.708 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.706+0000 7f3295cd1700 1 Processor -- start 2023-10-14T10:36:04.709 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.706+0000 7f3295cd1700 1 -- start start 2023-10-14T10:36:04.712 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.707+0000 7f3295cd1700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f32900d0c70 0x7f32900cdfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:04.713 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.707+0000 7f3295cd1700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32900d23e0 con 0x7f32900c5110 2023-10-14T10:36:04.713 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.707+0000 7f3295cd1700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32900cf190 con 0x7f329015b740 2023-10-14T10:36:04.713 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.707+0000 7f3295cd1700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f329015bb20 con 0x7f32900d0c70 2023-10-14T10:36:04.713 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.707+0000 7f328ffff700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f32900c5110 0x7f32900cf690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:51578/0 (socket says 172.21.15.67:51578) 2023-10-14T10:36:04.713 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.707+0000 7f328ffff700 1 -- 172.21.15.67:0/4084227666 learned_addr learned my addr 172.21.15.67:0/4084227666 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:36:04.714 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.707+0000 7f328effd700 1 --2- 172.21.15.67:0/4084227666 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f32900d0c70 0x7f32900cdfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:04.714 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.707+0000 7f328e7fc700 1 -- 172.21.15.67:0/4084227666 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3705667455 0 0) 0x7f3288001490 con 0x7f32900c5110 2023-10-14T10:36:04.714 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.707+0000 7f328e7fc700 1 -- 172.21.15.67:0/4084227666 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f327c0035b0 con 0x7f32900c5110 2023-10-14T10:36:04.714 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.708+0000 7f328e7fc700 1 -- 172.21.15.67:0/4084227666 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2771131789 0 0) 0x7f327c0035b0 con 0x7f32900c5110 2023-10-14T10:36:04.714 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.708+0000 7f328e7fc700 1 -- 172.21.15.67:0/4084227666 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f32900d0c70 msgr2=0x7f32900cdfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:04.715 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.708+0000 7f328e7fc700 1 --2- 172.21.15.67:0/4084227666 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f32900d0c70 0x7f32900cdfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:04.715 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.708+0000 7f328e7fc700 1 -- 172.21.15.67:0/4084227666 >> v1:172.21.15.72:6789/0 conn(0x7f329015b740 legacy=0x7f32900d1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:04.715 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.708+0000 7f328e7fc700 1 -- 172.21.15.67:0/4084227666 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32900cbd40 con 0x7f32900c5110 2023-10-14T10:36:04.715 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.708+0000 7f328e7fc700 1 -- 172.21.15.67:0/4084227666 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3180043755 0 0) 0x7f3288001b80 con 0x7f32900c5110 2023-10-14T10:36:04.715 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.708+0000 7f328e7fc700 1 -- 172.21.15.67:0/4084227666 >> v1:172.21.15.67:6789/0 conn(0x7f32900c5110 legacy=0x7f32900cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:04.716 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.708+0000 7f328e7fc700 1 --2- 172.21.15.67:0/4084227666 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f327c003a30 0x7f327c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:04.716 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.708+0000 7f328e7fc700 1 --2- 172.21.15.67:0/4084227666 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f32900d0c70 0x7f327c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:04.716 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.708+0000 7f328e7fc700 1 --2- 172.21.15.67:0/4084227666 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f327c004d90 0x7f327c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:04.716 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.708+0000 7f328e7fc700 1 -- 172.21.15.67:0/4084227666 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f329015bb20 con 0x7f327c003a30 2023-10-14T10:36:04.717 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.708+0000 7f328e7fc700 1 -- 172.21.15.67:0/4084227666 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f327c00bac0 con 0x7f327c004d90 2023-10-14T10:36:04.717 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.708+0000 7f328e7fc700 1 -- 172.21.15.67:0/4084227666 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f327c00bc80 con 0x7f32900d0c70 2023-10-14T10:36:04.717 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.708+0000 7f328f7fe700 1 --2- 172.21.15.67:0/4084227666 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f327c003a30 0x7f327c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:04.717 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.708+0000 7f328effd700 1 --2- 172.21.15.67:0/4084227666 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f32900d0c70 0x7f327c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:04.718 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.708+0000 7f328ffff700 1 --2- 172.21.15.67:0/4084227666 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f327c004d90 0x7f327c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:04.718 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.708+0000 7f328f7fe700 1 -- 172.21.15.67:0/4084227666 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f32900d0c70 msgr2=0x7f327c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:04.718 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.708+0000 7f328f7fe700 1 --2- 172.21.15.67:0/4084227666 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f32900d0c70 0x7f327c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:04.718 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.708+0000 7f328f7fe700 1 -- 172.21.15.67:0/4084227666 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f327c004d90 msgr2=0x7f327c005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:04.718 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.708+0000 7f328f7fe700 1 --2- 172.21.15.67:0/4084227666 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f327c004d90 0x7f327c005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:04.719 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.708+0000 7f328f7fe700 1 -- 172.21.15.67:0/4084227666 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f327c0035b0 con 0x7f327c003a30 2023-10-14T10:36:04.719 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.708+0000 7f328ffff700 1 --2- 172.21.15.67:0/4084227666 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f327c004d90 0x7f327c005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-14T10:36:04.719 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.708+0000 7f328f7fe700 1 --2- 172.21.15.67:0/4084227666 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f327c003a30 0x7f327c003e10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f32780018d0 tx=0x7f327c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfa915af0cfc3715 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:04.719 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.709+0000 7f328e7fc700 1 -- 172.21.15.67:0/4084227666 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f327800c690 con 0x7f327c003a30 2023-10-14T10:36:04.719 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.709+0000 7f328e7fc700 1 -- 172.21.15.67:0/4084227666 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f327800c810 con 0x7f327c003a30 2023-10-14T10:36:04.719 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.709+0000 7f328e7fc700 1 -- 172.21.15.67:0/4084227666 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f327800cbc0 con 0x7f327c003a30 2023-10-14T10:36:04.720 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.709+0000 7f3295cd1700 1 -- 172.21.15.67:0/4084227666 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f327c003a30 msgr2=0x7f327c003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:04.720 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.709+0000 7f3295cd1700 1 --2- 172.21.15.67:0/4084227666 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f327c003a30 0x7f327c003e10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f32780018d0 tx=0x7f327c005880 comp rx=0 tx=0).stop 2023-10-14T10:36:04.720 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.709+0000 7f328ffff700 1 -- 172.21.15.67:0/4084227666 reap_dead start 2023-10-14T10:36:04.720 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.709+0000 7f3295cd1700 1 -- 172.21.15.67:0/4084227666 shutdown_connections 2023-10-14T10:36:04.720 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.709+0000 7f3295cd1700 1 -- 172.21.15.67:0/4084227666 >> 172.21.15.67:0/4084227666 conn(0x7f32900665a0 msgr2=0x7f3290066980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:04.721 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.709+0000 7f3295cd1700 1 -- 172.21.15.67:0/4084227666 shutdown_connections 2023-10-14T10:36:04.721 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.709+0000 7f3295cd1700 1 -- 172.21.15.67:0/4084227666 wait complete. 2023-10-14T10:36:04.722 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.710+0000 7f3295cd1700 1 Processor -- start 2023-10-14T10:36:04.722 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.711+0000 7f3295cd1700 1 -- start start 2023-10-14T10:36:04.725 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.711+0000 7f3295cd1700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f327c003a30 0x7f3290153ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:04.725 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.711+0000 7f3295cd1700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3290156c40 0x7f329006d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:04.725 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.711+0000 7f3295cd1700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f32901e5530 0x7f32901e59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:04.726 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.711+0000 7f3295cd1700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32901e5f00 con 0x7f3290156c40 2023-10-14T10:36:04.726 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.711+0000 7f3295cd1700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32901f3030 con 0x7f327c003a30 2023-10-14T10:36:04.726 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.711+0000 7f3295cd1700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3290157140 con 0x7f32901e5530 2023-10-14T10:36:04.726 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.711+0000 7f328effd700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3290156c40 0x7f329006d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:04.727 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.711+0000 7f328effd700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3290156c40 0x7f329006d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43820/0 (socket says 172.21.15.67:43820) 2023-10-14T10:36:04.727 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.711+0000 7f328effd700 1 -- 172.21.15.67:0/373962825 learned_addr learned my addr 172.21.15.67:0/373962825 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:04.727 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.711+0000 7f328f7fe700 1 --2- 172.21.15.67:0/373962825 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f327c003a30 0x7f3290153ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:04.727 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.711+0000 7f328ffff700 1 --2- 172.21.15.67:0/373962825 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f32901e5530 0x7f32901e59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:04.727 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.712+0000 7f328effd700 1 -- 172.21.15.67:0/373962825 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f32901e5530 msgr2=0x7f32901e59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:04.727 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.712+0000 7f328effd700 1 --2- 172.21.15.67:0/373962825 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f32901e5530 0x7f32901e59c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:04.728 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.712+0000 7f328effd700 1 -- 172.21.15.67:0/373962825 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f327c003a30 msgr2=0x7f3290153ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:04.728 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.712+0000 7f328effd700 1 --2- 172.21.15.67:0/373962825 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f327c003a30 0x7f3290153ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:04.728 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.712+0000 7f328effd700 1 -- 172.21.15.67:0/373962825 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3290157420 con 0x7f3290156c40 2023-10-14T10:36:04.728 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.712+0000 7f328f7fe700 1 --2- 172.21.15.67:0/373962825 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f327c003a30 0x7f3290153ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-14T10:36:04.728 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.712+0000 7f328ffff700 1 --2- 172.21.15.67:0/373962825 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f32901e5530 0x7f32901e59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-14T10:36:04.729 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.712+0000 7f328effd700 1 --2- 172.21.15.67:0/373962825 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3290156c40 0x7f329006d280 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3280011aa0 tx=0x7f3280011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:04.729 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.712+0000 7f3287fff700 1 -- 172.21.15.67:0/373962825 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f328000f980 con 0x7f3290156c40 2023-10-14T10:36:04.729 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.712+0000 7f3287fff700 1 -- 172.21.15.67:0/373962825 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f328000fae0 con 0x7f3290156c40 2023-10-14T10:36:04.729 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.712+0000 7f3287fff700 1 -- 172.21.15.67:0/373962825 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f328000fe90 con 0x7f3290156c40 2023-10-14T10:36:04.729 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.712+0000 7f3295cd1700 1 -- 172.21.15.67:0/373962825 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3290157700 con 0x7f3290156c40 2023-10-14T10:36:04.730 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.712+0000 7f3295cd1700 1 -- 172.21.15.67:0/373962825 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3290154010 con 0x7f3290156c40 2023-10-14T10:36:04.730 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.713+0000 7f3295cd1700 1 -- 172.21.15.67:0/373962825 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32901f3250 con 0x7f3290156c40 2023-10-14T10:36:04.730 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.713+0000 7f3287fff700 1 -- 172.21.15.67:0/373962825 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f3280014440 con 0x7f3290156c40 2023-10-14T10:36:04.730 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.714+0000 7f3287fff700 1 --2- 172.21.15.67:0/373962825 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f327006af00 0x7f327006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:04.730 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.714+0000 7f3287fff700 1 -- 172.21.15.67:0/373962825 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3280089ce0 con 0x7f3290156c40 2023-10-14T10:36:04.731 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.714+0000 7f328f7fe700 1 --2- 172.21.15.67:0/373962825 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f327006af00 0x7f327006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:04.731 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.714+0000 7f328f7fe700 1 --2- 172.21.15.67:0/373962825 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f327006af00 0x7f327006d3c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3278014040 tx=0x7f3278005040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:04.731 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.718+0000 7f3287fff700 1 -- 172.21.15.67:0/373962825 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f32901f3250 con 0x7f3290156c40 2023-10-14T10:36:04.969 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:04.967+0000 7f3295cd1700 1 -- 172.21.15.67:0/373962825 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f32901f31f0 con 0x7f3290156c40 2023-10-14T10:36:05.873 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:05.871+0000 7f3287fff700 1 -- 172.21.15.67:0/373962825 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7f3280022020 con 0x7f3290156c40 2023-10-14T10:36:05.876 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:05.873+0000 7f3295cd1700 1 -- 172.21.15.67:0/373962825 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f327006af00 msgr2=0x7f327006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:05.876 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:05.873+0000 7f3295cd1700 1 --2- 172.21.15.67:0/373962825 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f327006af00 0x7f327006d3c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3278014040 tx=0x7f3278005040 comp rx=0 tx=0).stop 2023-10-14T10:36:05.876 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:05.873+0000 7f3295cd1700 1 -- 172.21.15.67:0/373962825 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3290156c40 msgr2=0x7f329006d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:05.876 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:05.873+0000 7f3295cd1700 1 --2- 172.21.15.67:0/373962825 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3290156c40 0x7f329006d280 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3280011aa0 tx=0x7f3280011e60 comp rx=0 tx=0).stop 2023-10-14T10:36:05.877 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:05.873+0000 7f3295cd1700 1 -- 172.21.15.67:0/373962825 shutdown_connections 2023-10-14T10:36:05.877 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:05.873+0000 7f3295cd1700 1 --2- 172.21.15.67:0/373962825 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f327006af00 0x7f327006d3c0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:05.877 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:05.874+0000 7f3295cd1700 1 --2- 172.21.15.67:0/373962825 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f32901e5530 0x7f32901e59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:05.877 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:05.874+0000 7f3295cd1700 1 --2- 172.21.15.67:0/373962825 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f327c003a30 0x7f3290153ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:05.877 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:05.874+0000 7f3295cd1700 1 --2- 172.21.15.67:0/373962825 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3290156c40 0x7f329006d280 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:05.878 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:05.874+0000 7f3295cd1700 1 -- 172.21.15.67:0/373962825 >> 172.21.15.67:0/373962825 conn(0x7f32900665a0 msgr2=0x7f32900b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:05.878 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:05.874+0000 7f3295cd1700 1 -- 172.21.15.67:0/373962825 shutdown_connections 2023-10-14T10:36:05.878 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:05.874+0000 7f3295cd1700 1 -- 172.21.15.67:0/373962825 wait complete. 2023-10-14T10:36:05.891 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-14T10:36:05.892 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-14T10:36:06.029 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:06.027+0000 7f417c811700 1 Processor -- start 2023-10-14T10:36:06.029 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:06.027+0000 7f417c811700 1 -- start start 2023-10-14T10:36:06.031 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:06.028+0000 7f417c811700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f4178157d40 0x7f41780cee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:06.032 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:06.028+0000 7f417c811700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f417806e2b0 0x7f41780cb420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:06.032 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:06.028+0000 7f417c811700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4178158120 con 0x7f417806de50 2023-10-14T10:36:06.032 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:06.028+0000 7f417c811700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f417806a840 con 0x7f417806e2b0 2023-10-14T10:36:06.032 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:06.028+0000 7f417c811700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41780caef0 con 0x7f4178157d40 2023-10-14T10:36:06.033 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:06.028+0000 7f4176d9d700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f417806e2b0 0x7f41780cb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:06.033 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:06.028+0000 7f4175d9b700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f417806de50 0x7f417815dd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:51596/0 (socket says 172.21.15.67:51596) 2023-10-14T10:36:06.033 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:06.028+0000 7f4175d9b700 1 -- 172.21.15.67:0/568007264 learned_addr learned my addr 172.21.15.67:0/568007264 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:36:06.033 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:06.028+0000 7f417659c700 1 --2- 172.21.15.67:0/568007264 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f4178157d40 0x7f41780cee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:06.034 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:06.028+0000 7f4176d9d700 1 -- 172.21.15.67:0/568007264 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f4178157d40 msgr2=0x7f41780cee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:06.034 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:06.028+0000 7f4176d9d700 1 --2- 172.21.15.67:0/568007264 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f4178157d40 0x7f41780cee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:06.034 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:06.028+0000 7f4176d9d700 1 -- 172.21.15.67:0/568007264 >> v1:172.21.15.67:6789/0 conn(0x7f417806de50 legacy=0x7f417815dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:06.034 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:06.028+0000 7f4176d9d700 1 -- 172.21.15.67:0/568007264 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f417806e750 con 0x7f417806e2b0 2023-10-14T10:36:06.034 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:06.028+0000 7f417559a700 1 -- 172.21.15.67:0/568007264 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3395388678 0 0) 0x7f4178158120 con 0x7f417806de50 2023-10-14T10:36:06.035 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:06.028+0000 7f4176d9d700 1 --2- 172.21.15.67:0/568007264 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f417806e2b0 0x7f41780cb420 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4170009d90 tx=0x7f417000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1d066104a7b59fa server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:06.035 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:06.028+0000 7f417559a700 1 -- 172.21.15.67:0/568007264 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f417000e070 con 0x7f417806e2b0 2023-10-14T10:36:06.035 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:06.028+0000 7f417559a700 1 -- 172.21.15.67:0/568007264 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4170009e20 con 0x7f417806e2b0 2023-10-14T10:36:06.035 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:06.029+0000 7f417559a700 1 -- 172.21.15.67:0/568007264 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f417000be10 con 0x7f417806e2b0 2023-10-14T10:36:06.035 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:06.029+0000 7f417c811700 1 -- 172.21.15.67:0/568007264 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f417806e2b0 msgr2=0x7f41780cb420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:06.036 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:06.029+0000 7f417c811700 1 --2- 172.21.15.67:0/568007264 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f417806e2b0 0x7f41780cb420 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4170009d90 tx=0x7f417000b330 comp rx=0 tx=0).stop 2023-10-14T10:36:06.036 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:06.029+0000 7f417c811700 1 -- 172.21.15.67:0/568007264 shutdown_connections 2023-10-14T10:36:06.036 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:06.029+0000 7f417c811700 1 --2- 172.21.15.67:0/568007264 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f417806e2b0 0x7f41780cb420 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:06.036 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:06.029+0000 7f417c811700 1 --2- 172.21.15.67:0/568007264 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f4178157d40 0x7f41780cee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:06.037 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:06.029+0000 7f417c811700 1 -- 172.21.15.67:0/568007264 >> 172.21.15.67:0/568007264 conn(0x7f41780cc6f0 msgr2=0x7f41780ccb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:06.037 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:06.029+0000 7f417c811700 1 -- 172.21.15.67:0/568007264 shutdown_connections 2023-10-14T10:36:06.037 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:06.029+0000 7f417c811700 1 -- 172.21.15.67:0/568007264 wait complete. 2023-10-14T10:36:06.041 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:06.030+0000 7f417c811700 1 Processor -- start 2023-10-14T10:36:06.041 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:06.031+0000 7f417c811700 1 -- start start 2023-10-14T10:36:06.041 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:06.031+0000 7f417c811700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f417806de50 0x7f4178151e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:06.042 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:06.031+0000 7f417c811700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f417806e2b0 0x7f41780b0f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:06.042 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:06.031+0000 7f417c811700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4178157d40 0x7f41781560f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:06.042 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:06.031+0000 7f417c811700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41781e4840 con 0x7f417806e2b0 2023-10-14T10:36:06.042 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:06.031+0000 7f417c811700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41781e3bc0 con 0x7f4178157d40 2023-10-14T10:36:06.042 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:06.031+0000 7f417c811700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41781e3ea0 con 0x7f417806de50 2023-10-14T10:36:06.043 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:06.031+0000 7f4175d9b700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f417806e2b0 0x7f41780b0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:06.043 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:06.031+0000 7f4175d9b700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f417806e2b0 0x7f41780b0f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43834/0 (socket says 172.21.15.67:43834) 2023-10-14T10:36:06.043 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:06.031+0000 7f4175d9b700 1 -- 172.21.15.67:0/2031117288 learned_addr learned my addr 172.21.15.67:0/2031117288 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:06.043 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:06.032+0000 7f417659c700 1 --2- 172.21.15.67:0/2031117288 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f417806de50 0x7f4178151e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:06.043 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:06.032+0000 7f4175d9b700 1 -- 172.21.15.67:0/2031117288 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f417806de50 msgr2=0x7f4178151e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:06.044 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:06.032+0000 7f4175d9b700 1 --2- 172.21.15.67:0/2031117288 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f417806de50 0x7f4178151e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:06.044 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:06.032+0000 7f4175d9b700 1 -- 172.21.15.67:0/2031117288 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4178157d40 msgr2=0x7f41781560f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:06.044 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:06.032+0000 7f4176d9d700 1 --2- 172.21.15.67:0/2031117288 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4178157d40 0x7f41781560f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:06.044 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:06.032+0000 7f4175d9b700 1 --2- 172.21.15.67:0/2031117288 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4178157d40 0x7f41781560f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:06.044 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:06.032+0000 7f4175d9b700 1 -- 172.21.15.67:0/2031117288 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4170009a70 con 0x7f417806e2b0 2023-10-14T10:36:06.045 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:06.032+0000 7f4175d9b700 1 --2- 172.21.15.67:0/2031117288 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f417806e2b0 0x7f41780b0f70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f416c0011e0 tx=0x7f416c00cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:06.045 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:06.032+0000 7f4166ffd700 1 -- 172.21.15.67:0/2031117288 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f416c004c80 con 0x7f417806e2b0 2023-10-14T10:36:06.045 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:06.032+0000 7f4166ffd700 1 -- 172.21.15.67:0/2031117288 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f416c004e00 con 0x7f417806e2b0 2023-10-14T10:36:06.045 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:06.032+0000 7f417c811700 1 -- 172.21.15.67:0/2031117288 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41781e4180 con 0x7f417806e2b0 2023-10-14T10:36:06.045 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:06.032+0000 7f4166ffd700 1 -- 172.21.15.67:0/2031117288 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f416c007310 con 0x7f417806e2b0 2023-10-14T10:36:06.046 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:06.032+0000 7f417c811700 1 -- 172.21.15.67:0/2031117288 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4178152340 con 0x7f417806e2b0 2023-10-14T10:36:06.046 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:06.033+0000 7f417c811700 1 -- 172.21.15.67:0/2031117288 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41781e4980 con 0x7f417806e2b0 2023-10-14T10:36:06.046 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:06.034+0000 7f4166ffd700 1 -- 172.21.15.67:0/2031117288 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f416c007470 con 0x7f417806e2b0 2023-10-14T10:36:06.046 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:06.034+0000 7f4166ffd700 1 --2- 172.21.15.67:0/2031117288 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f415806af00 0x7f415806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:06.046 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:06.034+0000 7f4166ffd700 1 -- 172.21.15.67:0/2031117288 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f416c089380 con 0x7f417806e2b0 2023-10-14T10:36:06.047 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:06.034+0000 7f417659c700 1 --2- 172.21.15.67:0/2031117288 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f415806af00 0x7f415806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:06.047 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:06.035+0000 7f417659c700 1 --2- 172.21.15.67:0/2031117288 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f415806af00 0x7f415806d3c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f41780c9f20 tx=0x7f4168008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:06.047 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:06.039+0000 7f4166ffd700 1 -- 172.21.15.67:0/2031117288 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f41781e4980 con 0x7f417806e2b0 2023-10-14T10:36:06.283 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:06.281+0000 7f417c811700 1 -- 172.21.15.67:0/2031117288 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f41781e4980 con 0x7f417806e2b0 2023-10-14T10:36:06.874 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:06.872+0000 7f4166ffd700 1 -- 172.21.15.67:0/2031117288 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7f41781e4980 con 0x7f417806e2b0 2023-10-14T10:36:06.877 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:06.874+0000 7f417c811700 1 -- 172.21.15.67:0/2031117288 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f415806af00 msgr2=0x7f415806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:06.877 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:06.874+0000 7f417c811700 1 --2- 172.21.15.67:0/2031117288 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f415806af00 0x7f415806d3c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f41780c9f20 tx=0x7f4168008040 comp rx=0 tx=0).stop 2023-10-14T10:36:06.877 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:06.874+0000 7f417c811700 1 -- 172.21.15.67:0/2031117288 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f417806e2b0 msgr2=0x7f41780b0f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:06.877 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:06.874+0000 7f417c811700 1 --2- 172.21.15.67:0/2031117288 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f417806e2b0 0x7f41780b0f70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f416c0011e0 tx=0x7f416c00cae0 comp rx=0 tx=0).stop 2023-10-14T10:36:06.877 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:06.875+0000 7f417c811700 1 -- 172.21.15.67:0/2031117288 shutdown_connections 2023-10-14T10:36:06.878 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:06.875+0000 7f417c811700 1 --2- 172.21.15.67:0/2031117288 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f415806af00 0x7f415806d3c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:06.878 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:06.875+0000 7f417c811700 1 --2- 172.21.15.67:0/2031117288 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4178157d40 0x7f41781560f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:06.878 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:06.875+0000 7f417c811700 1 --2- 172.21.15.67:0/2031117288 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f417806de50 0x7f4178151e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:06.878 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:06.875+0000 7f417c811700 1 --2- 172.21.15.67:0/2031117288 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f417806e2b0 0x7f41780b0f70 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:06.879 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:06.875+0000 7f417c811700 1 -- 172.21.15.67:0/2031117288 >> 172.21.15.67:0/2031117288 conn(0x7f41780cc6f0 msgr2=0x7f417806baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:06.879 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:06.875+0000 7f417c811700 1 -- 172.21.15.67:0/2031117288 shutdown_connections 2023-10-14T10:36:06.879 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:06.875+0000 7f417c811700 1 -- 172.21.15.67:0/2031117288 wait complete. 2023-10-14T10:36:06.891 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-14T10:36:06.892 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-14T10:36:07.023 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.021+0000 7fc353c56700 1 Processor -- start 2023-10-14T10:36:07.023 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.021+0000 7fc353c56700 1 -- start start 2023-10-14T10:36:07.026 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.022+0000 7fc353c56700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fc34c160e30 0x7fc34c0d4200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:07.026 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.022+0000 7fc353c56700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fc34c0d08a0 0x7fc34c0ce3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:07.026 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.022+0000 7fc353c56700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc34c0b6610 con 0x7fc34c161210 2023-10-14T10:36:07.026 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.022+0000 7fc353c56700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc34c0d1da0 con 0x7fc34c0d08a0 2023-10-14T10:36:07.027 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.022+0000 7fc353c56700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc34c0d2080 con 0x7fc34c160e30 2023-10-14T10:36:07.027 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.022+0000 7fc3521f3700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fc34c0d08a0 0x7fc34c0ce3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:07.027 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.022+0000 7fc3519f2700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fc34c160e30 0x7fc34c0d4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:07.027 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.022+0000 7fc3511f1700 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7fc34c161210 0x7fc34c0d0190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.67:59848/0 (socket says 172.21.15.67:59848) 2023-10-14T10:36:07.027 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.022+0000 7fc3511f1700 1 -- 172.21.15.67:0/199818745 learned_addr learned my addr 172.21.15.67:0/199818745 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:36:07.028 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.022+0000 7fc3509f0700 1 -- 172.21.15.67:0/199818745 <== mon.1 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2910902580 0 0) 0x7fc34c0b6610 con 0x7fc34c161210 2023-10-14T10:36:07.028 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.022+0000 7fc3509f0700 1 -- 172.21.15.67:0/199818745 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3380035b0 con 0x7fc34c161210 2023-10-14T10:36:07.028 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.022+0000 7fc3521f3700 1 -- 172.21.15.67:0/199818745 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fc34c160e30 msgr2=0x7fc34c0d4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:07.028 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.022+0000 7fc3521f3700 1 --2- 172.21.15.67:0/199818745 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fc34c160e30 0x7fc34c0d4200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:07.028 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.022+0000 7fc3521f3700 1 -- 172.21.15.67:0/199818745 >> v1:172.21.15.72:6789/0 conn(0x7fc34c161210 legacy=0x7fc34c0d0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:07.029 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.022+0000 7fc3521f3700 1 -- 172.21.15.67:0/199818745 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc34c0d2360 con 0x7fc34c0d08a0 2023-10-14T10:36:07.029 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.022+0000 7fc3519f2700 1 --2- 172.21.15.67:0/199818745 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fc34c160e30 0x7fc34c0d4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-14T10:36:07.029 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.023+0000 7fc3521f3700 1 --2- 172.21.15.67:0/199818745 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc34c0d08a0 0x7fc34c0ce3d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc344009d90 tx=0x7fc34400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58a9ef0a9672da14 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:07.029 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.023+0000 7fc3509f0700 1 -- 172.21.15.67:0/199818745 <== mon.1 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc34400e070 con 0x7fc34c0d08a0 2023-10-14T10:36:07.029 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.023+0000 7fc3509f0700 1 -- 172.21.15.67:0/199818745 <== mon.1 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc34400bb20 con 0x7fc34c0d08a0 2023-10-14T10:36:07.029 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.023+0000 7fc3509f0700 1 -- 172.21.15.67:0/199818745 <== mon.1 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc344012430 con 0x7fc34c0d08a0 2023-10-14T10:36:07.030 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.023+0000 7fc353c56700 1 -- 172.21.15.67:0/199818745 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc34c0d08a0 msgr2=0x7fc34c0ce3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:07.030 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.023+0000 7fc353c56700 1 --2- 172.21.15.67:0/199818745 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc34c0d08a0 0x7fc34c0ce3d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc344009d90 tx=0x7fc34400b330 comp rx=0 tx=0).stop 2023-10-14T10:36:07.030 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.023+0000 7fc353c56700 1 -- 172.21.15.67:0/199818745 shutdown_connections 2023-10-14T10:36:07.030 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.023+0000 7fc353c56700 1 --2- 172.21.15.67:0/199818745 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc34c0d08a0 0x7fc34c0ce3d0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:07.031 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.023+0000 7fc353c56700 1 --2- 172.21.15.67:0/199818745 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fc34c160e30 0x7fc34c0d4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:07.031 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.023+0000 7fc353c56700 1 -- 172.21.15.67:0/199818745 >> 172.21.15.67:0/199818745 conn(0x7fc34c09cb40 msgr2=0x7fc34c0d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:07.031 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.024+0000 7fc353c56700 1 -- 172.21.15.67:0/199818745 shutdown_connections 2023-10-14T10:36:07.031 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.024+0000 7fc353c56700 1 -- 172.21.15.67:0/199818745 wait complete. 2023-10-14T10:36:07.034 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.025+0000 7fc353c56700 1 Processor -- start 2023-10-14T10:36:07.034 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.025+0000 7fc353c56700 1 -- start start 2023-10-14T10:36:07.034 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.026+0000 7fc353c56700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fc34c0d08a0 0x7fc34c154c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:07.034 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.026+0000 7fc353c56700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc34c160e30 0x7fc34c0c84f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:07.035 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.026+0000 7fc353c56700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc34c161210 0x7fc34c09eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:07.035 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.026+0000 7fc353c56700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc34c0a0d60 con 0x7fc34c161210 2023-10-14T10:36:07.035 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.026+0000 7fc353c56700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc34c0c7450 con 0x7fc34c160e30 2023-10-14T10:36:07.035 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.026+0000 7fc353c56700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc34c0c7730 con 0x7fc34c0d08a0 2023-10-14T10:36:07.035 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.026+0000 7fc3521f3700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc34c161210 0x7fc34c09eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:07.036 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.026+0000 7fc3521f3700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc34c161210 0x7fc34c09eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43850/0 (socket says 172.21.15.67:43850) 2023-10-14T10:36:07.036 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.026+0000 7fc3521f3700 1 -- 172.21.15.67:0/3570621612 learned_addr learned my addr 172.21.15.67:0/3570621612 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:07.036 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.026+0000 7fc3519f2700 1 --2- 172.21.15.67:0/3570621612 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fc34c0d08a0 0x7fc34c154c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:07.036 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.026+0000 7fc3511f1700 1 --2- 172.21.15.67:0/3570621612 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc34c160e30 0x7fc34c0c84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:07.036 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.026+0000 7fc3521f3700 1 -- 172.21.15.67:0/3570621612 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fc34c0d08a0 msgr2=0x7fc34c154c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:07.037 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.026+0000 7fc3521f3700 1 --2- 172.21.15.67:0/3570621612 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fc34c0d08a0 0x7fc34c154c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:07.037 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.026+0000 7fc3521f3700 1 -- 172.21.15.67:0/3570621612 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc34c160e30 msgr2=0x7fc34c0c84f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:07.037 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.026+0000 7fc3521f3700 1 --2- 172.21.15.67:0/3570621612 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc34c160e30 0x7fc34c0c84f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:07.037 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.026+0000 7fc3521f3700 1 -- 172.21.15.67:0/3570621612 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc348009430 con 0x7fc34c161210 2023-10-14T10:36:07.037 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.026+0000 7fc3511f1700 1 --2- 172.21.15.67:0/3570621612 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc34c160e30 0x7fc34c0c84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-14T10:36:07.038 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.027+0000 7fc3521f3700 1 --2- 172.21.15.67:0/3570621612 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc34c161210 0x7fc34c09eed0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc344014040 tx=0x7fc344004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:07.038 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.027+0000 7fc3427fc700 1 -- 172.21.15.67:0/3570621612 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3440129f0 con 0x7fc34c161210 2023-10-14T10:36:07.038 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.027+0000 7fc3427fc700 1 -- 172.21.15.67:0/3570621612 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc344020070 con 0x7fc34c161210 2023-10-14T10:36:07.038 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.027+0000 7fc353c56700 1 -- 172.21.15.67:0/3570621612 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc344009a70 con 0x7fc34c161210 2023-10-14T10:36:07.038 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.027+0000 7fc3427fc700 1 -- 172.21.15.67:0/3570621612 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc344012db0 con 0x7fc34c161210 2023-10-14T10:36:07.039 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.027+0000 7fc353c56700 1 -- 172.21.15.67:0/3570621612 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc34c0c7cf0 con 0x7fc34c161210 2023-10-14T10:36:07.039 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.028+0000 7fc353c56700 1 -- 172.21.15.67:0/3570621612 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc34c067d20 con 0x7fc34c161210 2023-10-14T10:36:07.039 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.028+0000 7fc3427fc700 1 -- 172.21.15.67:0/3570621612 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fc344004040 con 0x7fc34c161210 2023-10-14T10:36:07.039 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.029+0000 7fc3427fc700 1 --2- 172.21.15.67:0/3570621612 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fc33006af00 0x7fc33006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:07.039 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.029+0000 7fc3427fc700 1 -- 172.21.15.67:0/3570621612 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc344022070 con 0x7fc34c161210 2023-10-14T10:36:07.039 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.029+0000 7fc3519f2700 1 --2- 172.21.15.67:0/3570621612 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fc33006af00 0x7fc33006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:07.040 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.029+0000 7fc3519f2700 1 --2- 172.21.15.67:0/3570621612 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fc33006af00 0x7fc33006d3c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc348009d50 tx=0x7fc3480091c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:07.040 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.033+0000 7fc3427fc700 1 -- 172.21.15.67:0/3570621612 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc3440566d0 con 0x7fc34c161210 2023-10-14T10:36:07.281 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.279+0000 7fc353c56700 1 -- 172.21.15.67:0/3570621612 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc34c0b5cd0 con 0x7fc34c161210 2023-10-14T10:36:07.282 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.279+0000 7fc3427fc700 1 -- 172.21.15.67:0/3570621612 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4901 (secure 0 0 0) 0x7fc344059cf0 con 0x7fc34c161210 2023-10-14T10:36:07.282 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-14T10:36:07.282 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4217,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6836/838298018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":838298018},{"type":"v1","addr":"172.21.15.72:6836","nonce":838298018}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4255,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/1687192726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6833","nonce":1687192726},{"type":"v1","addr":"172.21.15.67:6835","nonce":1687192726}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6834/3936231765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":3936231765},{"type":"v1","addr":"172.21.15.67:6834","nonce":3936231765}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-14T10:36:03.247976+0000","modified":"2023-10-14T10:36:06.283381+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4267},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4267":{"gid":4267,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.72:6837/757500960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6835","nonce":757500960},{"type":"v1","addr":"172.21.15.72:6837","nonce":757500960}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-14T10:36:07.284 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.282+0000 7fc353c56700 1 -- 172.21.15.67:0/3570621612 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fc33006af00 msgr2=0x7fc33006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:07.284 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.282+0000 7fc353c56700 1 --2- 172.21.15.67:0/3570621612 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fc33006af00 0x7fc33006d3c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc348009d50 tx=0x7fc3480091c0 comp rx=0 tx=0).stop 2023-10-14T10:36:07.284 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.282+0000 7fc353c56700 1 -- 172.21.15.67:0/3570621612 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc34c161210 msgr2=0x7fc34c09eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:07.285 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.282+0000 7fc353c56700 1 --2- 172.21.15.67:0/3570621612 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc34c161210 0x7fc34c09eed0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc344014040 tx=0x7fc344004ad0 comp rx=0 tx=0).stop 2023-10-14T10:36:07.285 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.282+0000 7fc353c56700 1 -- 172.21.15.67:0/3570621612 shutdown_connections 2023-10-14T10:36:07.285 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.282+0000 7fc353c56700 1 --2- 172.21.15.67:0/3570621612 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fc33006af00 0x7fc33006d3c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:07.285 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.282+0000 7fc353c56700 1 --2- 172.21.15.67:0/3570621612 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc34c161210 0x7fc34c09eed0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:07.285 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.282+0000 7fc353c56700 1 --2- 172.21.15.67:0/3570621612 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fc34c0d08a0 0x7fc34c154c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:07.285 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.282+0000 7fc353c56700 1 --2- 172.21.15.67:0/3570621612 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc34c160e30 0x7fc34c0c84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:07.286 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.282+0000 7fc353c56700 1 -- 172.21.15.67:0/3570621612 >> 172.21.15.67:0/3570621612 conn(0x7fc34c09cb40 msgr2=0x7fc34c06d190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:07.286 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.282+0000 7fc353c56700 1 -- 172.21.15.67:0/3570621612 shutdown_connections 2023-10-14T10:36:07.286 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.282+0000 7fc353c56700 1 -- 172.21.15.67:0/3570621612 wait complete. 2023-10-14T10:36:07.286 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 8 2023-10-14T10:36:07.296 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-14T10:36:07.421 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.419+0000 7f541f59d700 1 Processor -- start 2023-10-14T10:36:07.421 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.419+0000 7f541f59d700 1 -- start start 2023-10-14T10:36:07.423 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.420+0000 7f541f59d700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f54180ca8d0 0x7f5418161460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:07.424 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.420+0000 7f541f59d700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f54180ce560 0x7f54181619a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:07.424 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.420+0000 7f541f59d700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54180ceb60 con 0x7f5418160c40 2023-10-14T10:36:07.424 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.420+0000 7f541f59d700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54180cf4b0 con 0x7f54180ca8d0 2023-10-14T10:36:07.424 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.420+0000 7f541f59d700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5418071970 con 0x7f54180ce560 2023-10-14T10:36:07.424 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.420+0000 7f541dd9a700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f54180ca8d0 0x7f5418161460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:07.425 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.420+0000 7f541e59b700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f5418160c40 0x7f54180cde50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:51622/0 (socket says 172.21.15.67:51622) 2023-10-14T10:36:07.425 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.420+0000 7f541e59b700 1 -- 172.21.15.67:0/3847100524 learned_addr learned my addr 172.21.15.67:0/3847100524 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:36:07.425 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.420+0000 7f541ed9c700 1 --2- 172.21.15.67:0/3847100524 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f54180ce560 0x7f54181619a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:07.425 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.420+0000 7f541dd9a700 1 -- 172.21.15.67:0/3847100524 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f54180ce560 msgr2=0x7f54181619a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:07.425 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.420+0000 7f541dd9a700 1 --2- 172.21.15.67:0/3847100524 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f54180ce560 0x7f54181619a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:07.426 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.420+0000 7f541dd9a700 1 -- 172.21.15.67:0/3847100524 >> v1:172.21.15.67:6789/0 conn(0x7f5418160c40 legacy=0x7f54180cde50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:07.426 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.420+0000 7f541dd9a700 1 -- 172.21.15.67:0/3847100524 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5418071c50 con 0x7f54180ca8d0 2023-10-14T10:36:07.426 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.420+0000 7f541d599700 1 -- 172.21.15.67:0/3847100524 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1651516098 0 0) 0x7f54180ceb60 con 0x7f5418160c40 2023-10-14T10:36:07.426 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.420+0000 7f541dd9a700 1 --2- 172.21.15.67:0/3847100524 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54180ca8d0 0x7f5418161460 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5408009840 tx=0x7f5408009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3589ff71979ba586 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:07.426 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.420+0000 7f541d599700 1 -- 172.21.15.67:0/3847100524 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5408011830 con 0x7f54180ca8d0 2023-10-14T10:36:07.427 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.420+0000 7f541d599700 1 -- 172.21.15.67:0/3847100524 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5408011990 con 0x7f54180ca8d0 2023-10-14T10:36:07.427 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.421+0000 7f541d599700 1 -- 172.21.15.67:0/3847100524 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5408011d20 con 0x7f54180ca8d0 2023-10-14T10:36:07.427 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.421+0000 7f541f59d700 1 -- 172.21.15.67:0/3847100524 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54180ca8d0 msgr2=0x7f5418161460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:07.427 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.421+0000 7f541f59d700 1 --2- 172.21.15.67:0/3847100524 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54180ca8d0 0x7f5418161460 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5408009840 tx=0x7f5408009b50 comp rx=0 tx=0).stop 2023-10-14T10:36:07.427 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.421+0000 7f541f59d700 1 -- 172.21.15.67:0/3847100524 shutdown_connections 2023-10-14T10:36:07.428 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.421+0000 7f541f59d700 1 --2- 172.21.15.67:0/3847100524 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f54180ce560 0x7f54181619a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:07.428 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.421+0000 7f541f59d700 1 --2- 172.21.15.67:0/3847100524 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54180ca8d0 0x7f5418161460 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:07.428 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.421+0000 7f541f59d700 1 -- 172.21.15.67:0/3847100524 >> 172.21.15.67:0/3847100524 conn(0x7f54180b1d70 msgr2=0x7f54180b2150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:07.428 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.421+0000 7f541f59d700 1 -- 172.21.15.67:0/3847100524 shutdown_connections 2023-10-14T10:36:07.428 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.421+0000 7f541f59d700 1 -- 172.21.15.67:0/3847100524 wait complete. 2023-10-14T10:36:07.431 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.422+0000 7f541f59d700 1 Processor -- start 2023-10-14T10:36:07.431 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.422+0000 7f541f59d700 1 -- start start 2023-10-14T10:36:07.432 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.423+0000 7f541f59d700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f54180ca8d0 0x7f54181f3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:07.432 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.423+0000 7f541f59d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54180ce560 0x7f541815ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:07.432 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.423+0000 7f541f59d700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5418160c40 0x7f54181f6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:07.432 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.423+0000 7f541f59d700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f541815f2b0 con 0x7f54180ce560 2023-10-14T10:36:07.432 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.423+0000 7f541f59d700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54181f6c00 con 0x7f5418160c40 2023-10-14T10:36:07.433 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.423+0000 7f541f59d700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54181f6ee0 con 0x7f54180ca8d0 2023-10-14T10:36:07.433 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.423+0000 7f541dd9a700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54180ce560 0x7f541815ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:07.433 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.423+0000 7f541dd9a700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54180ce560 0x7f541815ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43862/0 (socket says 172.21.15.67:43862) 2023-10-14T10:36:07.433 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.423+0000 7f541dd9a700 1 -- 172.21.15.67:0/4002943897 learned_addr learned my addr 172.21.15.67:0/4002943897 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:07.433 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.423+0000 7f541e59b700 1 --2- 172.21.15.67:0/4002943897 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f54180ca8d0 0x7f54181f3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:07.433 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.423+0000 7f541ed9c700 1 --2- 172.21.15.67:0/4002943897 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5418160c40 0x7f54181f6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:07.434 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.423+0000 7f541dd9a700 1 -- 172.21.15.67:0/4002943897 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f54180ca8d0 msgr2=0x7f54181f3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:07.434 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.423+0000 7f541dd9a700 1 --2- 172.21.15.67:0/4002943897 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f54180ca8d0 0x7f54181f3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:07.434 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.423+0000 7f541dd9a700 1 -- 172.21.15.67:0/4002943897 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5418160c40 msgr2=0x7f54181f6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:07.434 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.423+0000 7f541dd9a700 1 --2- 172.21.15.67:0/4002943897 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5418160c40 0x7f54181f6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:07.434 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.423+0000 7f541dd9a700 1 -- 172.21.15.67:0/4002943897 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54080094a0 con 0x7f54180ce560 2023-10-14T10:36:07.435 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.424+0000 7f541dd9a700 1 --2- 172.21.15.67:0/4002943897 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54180ce560 0x7f541815ed70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f54080090b0 tx=0x7f5408004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:07.435 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.424+0000 7f540effd700 1 -- 172.21.15.67:0/4002943897 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54080036a0 con 0x7f54180ce560 2023-10-14T10:36:07.435 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.424+0000 7f540effd700 1 -- 172.21.15.67:0/4002943897 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f540801f070 con 0x7f54180ce560 2023-10-14T10:36:07.435 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.424+0000 7f541f59d700 1 -- 172.21.15.67:0/4002943897 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54181f2e80 con 0x7f54180ce560 2023-10-14T10:36:07.435 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.424+0000 7f540effd700 1 -- 172.21.15.67:0/4002943897 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5408003a60 con 0x7f54180ce560 2023-10-14T10:36:07.436 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.424+0000 7f541f59d700 1 -- 172.21.15.67:0/4002943897 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5418152750 con 0x7f54180ce560 2023-10-14T10:36:07.436 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.425+0000 7f541f59d700 1 -- 172.21.15.67:0/4002943897 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54181f3340 con 0x7f54180ce560 2023-10-14T10:36:07.436 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.425+0000 7f540effd700 1 -- 172.21.15.67:0/4002943897 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f5408003bc0 con 0x7f54180ce560 2023-10-14T10:36:07.436 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.426+0000 7f540effd700 1 --2- 172.21.15.67:0/4002943897 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f53fc06af30 0x7f53fc06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:07.436 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.426+0000 7f540effd700 1 -- 172.21.15.67:0/4002943897 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5408021070 con 0x7f54180ce560 2023-10-14T10:36:07.436 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.426+0000 7f541e59b700 1 --2- 172.21.15.67:0/4002943897 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f53fc06af30 0x7f53fc06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:07.437 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.426+0000 7f541e59b700 1 --2- 172.21.15.67:0/4002943897 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f53fc06af30 0x7f53fc06d3f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5414000f60 tx=0x7f5414007e70 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:07.437 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.430+0000 7f540effd700 1 -- 172.21.15.67:0/4002943897 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5408055e00 con 0x7f54180ce560 2023-10-14T10:36:07.632 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.630+0000 7f541f59d700 1 -- 172.21.15.67:0/4002943897 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f541804f400 con 0x7f54180ce560 2023-10-14T10:36:07.633 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.631+0000 7f540effd700 1 -- 172.21.15.67:0/4002943897 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16457 (secure 0 0 0) 0x7f540802b090 con 0x7f54180ce560 2023-10-14T10:36:07.633 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-14T10:36:07.634 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":22,"fsid":"37f9ab46-3082-4b71-8b24-08d39897b008","created":"2023-10-14T10:35:46.067554+0000","modified":"2023-10-14T10:36:04.837296+0000","last_up_change":"2023-10-14T10:35:56.668142+0000","last_in_change":"2023-10-14T10:35:48.551490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-14T10:35:57.809566+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-14T10:36:01.667146+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-14T10:36:02.219700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2b59a07e-11ef-440c-b766-9f64ac885838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6801","nonce":1287255499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6803","nonce":1287255499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6807","nonce":1287255499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6805","nonce":1287255499}]},"public_addr":"172.21.15.67:6801/1287255499","cluster_addr":"172.21.15.67:6803/1287255499","heartbeat_back_addr":"172.21.15.67:6807/1287255499","heartbeat_front_addr":"172.21.15.67:6805/1287255499","state":["exists","up"]},{"osd":1,"uuid":"6f813bd6-fa0b-487d-a204-431eb1276a60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6818","nonce":2435538787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6822","nonce":2435538787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6830","nonce":2435538787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6826","nonce":2435538787}]},"public_addr":"172.21.15.67:6818/2435538787","cluster_addr":"172.21.15.67:6822/2435538787","heartbeat_back_addr":"172.21.15.67:6830/2435538787","heartbeat_front_addr":"172.21.15.67:6826/2435538787","state":["exists","up"]},{"osd":2,"uuid":"6af42c6a-1b59-4669-9c04-b6876788446d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6809","nonce":2827491791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6811","nonce":2827491791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6815","nonce":2827491791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6813","nonce":2827491791}]},"public_addr":"172.21.15.67:6809/2827491791","cluster_addr":"172.21.15.67:6811/2827491791","heartbeat_back_addr":"172.21.15.67:6815/2827491791","heartbeat_front_addr":"172.21.15.67:6813/2827491791","state":["exists","up"]},{"osd":3,"uuid":"1d889d76-a646-499d-bfff-ed5bae5434ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6819","nonce":1848001176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6821","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6823","nonce":1848001176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6831","nonce":1848001176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6825","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6827","nonce":1848001176}]},"public_addr":"172.21.15.67:6819/1848001176","cluster_addr":"172.21.15.67:6823/1848001176","heartbeat_back_addr":"172.21.15.67:6831/1848001176","heartbeat_front_addr":"172.21.15.67:6827/1848001176","state":["exists","up"]},{"osd":4,"uuid":"c3271dff-b437-4fdd-b13b-6e49a35473af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6804","nonce":1891230785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6808","nonce":1891230785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6815","nonce":1891230785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6811","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6813","nonce":1891230785}]},"public_addr":"172.21.15.72:6804/1891230785","cluster_addr":"172.21.15.72:6808/1891230785","heartbeat_back_addr":"172.21.15.72:6815/1891230785","heartbeat_front_addr":"172.21.15.72:6813/1891230785","state":["exists","up"]},{"osd":5,"uuid":"0337bb18-4b68-4eff-b528-a667275e6926","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6801","nonce":569686991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6803","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6805","nonce":569686991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6812","nonce":569686991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6807","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6809","nonce":569686991}]},"public_addr":"172.21.15.72:6801/569686991","cluster_addr":"172.21.15.72:6805/569686991","heartbeat_back_addr":"172.21.15.72:6812/569686991","heartbeat_front_addr":"172.21.15.72:6809/569686991","state":["exists","up"]},{"osd":6,"uuid":"8e92d9cf-0c8c-46eb-89a7-db2023ef1509","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6817","nonce":3767685934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6819","nonce":3767685934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6823","nonce":3767685934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6821","nonce":3767685934}]},"public_addr":"172.21.15.72:6817/3767685934","cluster_addr":"172.21.15.72:6819/3767685934","heartbeat_back_addr":"172.21.15.72:6823/3767685934","heartbeat_front_addr":"172.21.15.72:6821/3767685934","state":["exists","up"]},{"osd":7,"uuid":"3fd6aa7d-50a9-45c9-9561-5902a044a119","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6825","nonce":3567465926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6827","nonce":3567465926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6831","nonce":3567465926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6829","nonce":3567465926}]},"public_addr":"172.21.15.72:6825/3567465926","cluster_addr":"172.21.15.72:6827/3567465926","heartbeat_back_addr":"172.21.15.72:6831/3567465926","heartbeat_front_addr":"172.21.15.72:6829/3567465926","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.511466+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.795107+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.858674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.728895+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.794842+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.750064+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.732122+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.765185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-14T10:36:07.635 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.633+0000 7f541f59d700 1 -- 172.21.15.67:0/4002943897 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f53fc06af30 msgr2=0x7f53fc06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:07.635 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.633+0000 7f541f59d700 1 --2- 172.21.15.67:0/4002943897 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f53fc06af30 0x7f53fc06d3f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5414000f60 tx=0x7f5414007e70 comp rx=0 tx=0).stop 2023-10-14T10:36:07.636 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.633+0000 7f541f59d700 1 -- 172.21.15.67:0/4002943897 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54180ce560 msgr2=0x7f541815ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:07.636 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.633+0000 7f541f59d700 1 --2- 172.21.15.67:0/4002943897 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54180ce560 0x7f541815ed70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f54080090b0 tx=0x7f5408004750 comp rx=0 tx=0).stop 2023-10-14T10:36:07.636 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.633+0000 7f541f59d700 1 -- 172.21.15.67:0/4002943897 shutdown_connections 2023-10-14T10:36:07.636 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.633+0000 7f541f59d700 1 --2- 172.21.15.67:0/4002943897 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f53fc06af30 0x7f53fc06d3f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:07.636 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.633+0000 7f541f59d700 1 --2- 172.21.15.67:0/4002943897 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5418160c40 0x7f54181f6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:07.637 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.633+0000 7f541f59d700 1 --2- 172.21.15.67:0/4002943897 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f54180ca8d0 0x7f54181f3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:07.637 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.633+0000 7f541f59d700 1 --2- 172.21.15.67:0/4002943897 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54180ce560 0x7f541815ed70 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:07.637 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.633+0000 7f541f59d700 1 -- 172.21.15.67:0/4002943897 >> 172.21.15.67:0/4002943897 conn(0x7f54180b1d70 msgr2=0x7f54181e7d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:07.637 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.633+0000 7f541f59d700 1 -- 172.21.15.67:0/4002943897 shutdown_connections 2023-10-14T10:36:07.637 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.633+0000 7f541f59d700 1 -- 172.21.15.67:0/4002943897 wait complete. 2023-10-14T10:36:07.649 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-14T10:36:07.650 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T10:36:07.775 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.773+0000 7fbdcbdbc700 1 Processor -- start 2023-10-14T10:36:07.775 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.773+0000 7fbdcbdbc700 1 -- start start 2023-10-14T10:36:07.777 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.774+0000 7fbdcbdbc700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fbdc415ab10 0x7fbdc415f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:07.777 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.774+0000 7fbdcbdbc700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fbdc40b1f50 0x7fbdc40cf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:07.778 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.774+0000 7fbdcbdbc700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdc415e7a0 con 0x7fbdc40b2400 2023-10-14T10:36:07.778 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.774+0000 7fbdcbdbc700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdc415f730 con 0x7fbdc415ab10 2023-10-14T10:36:07.778 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.774+0000 7fbdcbdbc700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdc40b29a0 con 0x7fbdc40b1f50 2023-10-14T10:36:07.778 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.774+0000 7fbdc9b58700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fbdc415ab10 0x7fbdc415f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:07.778 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.774+0000 7fbdca359700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fbdc40b2400 0x7fbdc40cbf40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:51638/0 (socket says 172.21.15.67:51638) 2023-10-14T10:36:07.779 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.774+0000 7fbdca359700 1 -- 172.21.15.67:0/198593550 learned_addr learned my addr 172.21.15.67:0/198593550 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:36:07.779 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.774+0000 7fbdc9357700 1 --2- 172.21.15.67:0/198593550 >> v2:172.21.15.72:3300/0 conn(0x7fbdc40b1f50 0x7fbdc40cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:07.779 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.774+0000 7fbdc9b58700 1 -- 172.21.15.67:0/198593550 >> v2:172.21.15.72:3300/0 conn(0x7fbdc40b1f50 msgr2=0x7fbdc40cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:07.779 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.774+0000 7fbdc9b58700 1 --2- 172.21.15.67:0/198593550 >> v2:172.21.15.72:3300/0 conn(0x7fbdc40b1f50 0x7fbdc40cf720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:07.779 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.774+0000 7fbdc9b58700 1 -- 172.21.15.67:0/198593550 >> v1:172.21.15.67:6789/0 conn(0x7fbdc40b2400 legacy=0x7fbdc40cbf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:07.780 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.774+0000 7fbdc8b56700 1 -- 172.21.15.67:0/198593550 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 444084796 0 0) 0x7fbdc415e7a0 con 0x7fbdc40b2400 2023-10-14T10:36:07.780 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.774+0000 7fbdc9b58700 1 -- 172.21.15.67:0/198593550 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdc40b2c80 con 0x7fbdc415ab10 2023-10-14T10:36:07.780 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.774+0000 7fbdc9b58700 1 --2- 172.21.15.67:0/198593550 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbdc415ab10 0x7fbdc415f1f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbdc406d470 tx=0x7fbdc0009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b98b0efa4adf4f6c server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:07.780 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.774+0000 7fbdc8b56700 1 -- 172.21.15.67:0/198593550 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdc0011830 con 0x7fbdc415ab10 2023-10-14T10:36:07.780 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.774+0000 7fbdc8b56700 1 -- 172.21.15.67:0/198593550 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdc0011990 con 0x7fbdc415ab10 2023-10-14T10:36:07.781 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.774+0000 7fbdc8b56700 1 -- 172.21.15.67:0/198593550 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdc0011d20 con 0x7fbdc415ab10 2023-10-14T10:36:07.781 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.775+0000 7fbdcbdbc700 1 -- 172.21.15.67:0/198593550 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbdc415ab10 msgr2=0x7fbdc415f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:07.781 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.775+0000 7fbdcbdbc700 1 --2- 172.21.15.67:0/198593550 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbdc415ab10 0x7fbdc415f1f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbdc406d470 tx=0x7fbdc0009b20 comp rx=0 tx=0).stop 2023-10-14T10:36:07.781 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.775+0000 7fbdcbdbc700 1 -- 172.21.15.67:0/198593550 shutdown_connections 2023-10-14T10:36:07.782 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.775+0000 7fbdcbdbc700 1 --2- 172.21.15.67:0/198593550 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbdc415ab10 0x7fbdc415f1f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:07.782 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.775+0000 7fbdcbdbc700 1 --2- 172.21.15.67:0/198593550 >> v2:172.21.15.72:3300/0 conn(0x7fbdc40b1f50 0x7fbdc40cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:07.782 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.775+0000 7fbdcbdbc700 1 -- 172.21.15.67:0/198593550 >> 172.21.15.67:0/198593550 conn(0x7fbdc406dae0 msgr2=0x7fbdc406def0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:07.782 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.775+0000 7fbdcbdbc700 1 -- 172.21.15.67:0/198593550 shutdown_connections 2023-10-14T10:36:07.782 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.775+0000 7fbdcbdbc700 1 -- 172.21.15.67:0/198593550 wait complete. 2023-10-14T10:36:07.785 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.776+0000 7fbdcbdbc700 1 Processor -- start 2023-10-14T10:36:07.785 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.776+0000 7fbdcbdbc700 1 -- start start 2023-10-14T10:36:07.785 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.777+0000 7fbdcbdbc700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbdc40b1f50 0x7fbdc40c92d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:07.786 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.777+0000 7fbdcbdbc700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fbdc40b2400 0x7fbdc4155de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:07.786 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.777+0000 7fbdcbdbc700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbdc415ab10 0x7fbdc4158300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:07.786 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.777+0000 7fbdcbdbc700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc40c8500 con 0x7fbdc40b1f50 2023-10-14T10:36:07.786 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.777+0000 7fbdcbdbc700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc40c75a0 con 0x7fbdc415ab10 2023-10-14T10:36:07.786 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.777+0000 7fbdcbdbc700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc40c7880 con 0x7fbdc40b2400 2023-10-14T10:36:07.787 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.777+0000 7fbdc9b58700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbdc40b1f50 0x7fbdc40c92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:07.787 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.777+0000 7fbdc9b58700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbdc40b1f50 0x7fbdc40c92d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43872/0 (socket says 172.21.15.67:43872) 2023-10-14T10:36:07.787 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.777+0000 7fbdc9b58700 1 -- 172.21.15.67:0/3473735625 learned_addr learned my addr 172.21.15.67:0/3473735625 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:07.787 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.777+0000 7fbdc9357700 1 --2- 172.21.15.67:0/3473735625 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fbdc40b2400 0x7fbdc4155de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:07.787 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.777+0000 7fbdca359700 1 --2- 172.21.15.67:0/3473735625 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbdc415ab10 0x7fbdc4158300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:07.788 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.777+0000 7fbdc9b58700 1 -- 172.21.15.67:0/3473735625 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fbdc40b2400 msgr2=0x7fbdc4155de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:07.788 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.777+0000 7fbdc9b58700 1 --2- 172.21.15.67:0/3473735625 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fbdc40b2400 0x7fbdc4155de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:07.788 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.777+0000 7fbdc9b58700 1 -- 172.21.15.67:0/3473735625 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbdc415ab10 msgr2=0x7fbdc4158300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:07.788 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.777+0000 7fbdc9b58700 1 --2- 172.21.15.67:0/3473735625 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbdc415ab10 0x7fbdc4158300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:07.788 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.777+0000 7fbdc9b58700 1 -- 172.21.15.67:0/3473735625 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdc00094a0 con 0x7fbdc40b1f50 2023-10-14T10:36:07.789 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.777+0000 7fbdc9b58700 1 --2- 172.21.15.67:0/3473735625 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbdc40b1f50 0x7fbdc40c92d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbdc00090b0 tx=0x7fbdc0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:07.789 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.778+0000 7fbdba7fc700 1 -- 172.21.15.67:0/3473735625 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdc00036a0 con 0x7fbdc40b1f50 2023-10-14T10:36:07.789 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.778+0000 7fbdba7fc700 1 -- 172.21.15.67:0/3473735625 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdc001f070 con 0x7fbdc40b1f50 2023-10-14T10:36:07.789 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.778+0000 7fbdcbdbc700 1 -- 172.21.15.67:0/3473735625 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdc40c7b60 con 0x7fbdc40b1f50 2023-10-14T10:36:07.789 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.778+0000 7fbdba7fc700 1 -- 172.21.15.67:0/3473735625 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdc0003a60 con 0x7fbdc40b1f50 2023-10-14T10:36:07.790 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.778+0000 7fbdcbdbc700 1 -- 172.21.15.67:0/3473735625 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdc40c8120 con 0x7fbdc40b1f50 2023-10-14T10:36:07.790 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.779+0000 7fbdcbdbc700 1 -- 172.21.15.67:0/3473735625 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdc4067d20 con 0x7fbdc40b1f50 2023-10-14T10:36:07.790 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.779+0000 7fbdba7fc700 1 -- 172.21.15.67:0/3473735625 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fbdc0003bc0 con 0x7fbdc40b1f50 2023-10-14T10:36:07.790 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.779+0000 7fbdba7fc700 1 --2- 172.21.15.67:0/3473735625 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fbda806af00 0x7fbda806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:07.790 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.779+0000 7fbdba7fc700 1 -- 172.21.15.67:0/3473735625 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbdc0021070 con 0x7fbdc40b1f50 2023-10-14T10:36:07.791 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.780+0000 7fbdc9357700 1 --2- 172.21.15.67:0/3473735625 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fbda806af00 0x7fbda806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:07.791 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.780+0000 7fbdc9357700 1 --2- 172.21.15.67:0/3473735625 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fbda806af00 0x7fbda806d3c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fbdc40cc240 tx=0x7fbdb40091c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:07.791 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.784+0000 7fbdba7fc700 1 -- 172.21.15.67:0/3473735625 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbdc0055e90 con 0x7fbdc40b1f50 2023-10-14T10:36:07.983 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.981+0000 7fbdcbdbc700 1 -- 172.21.15.67:0/3473735625 --> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbdc415ecd0 con 0x7fbda806af00 2023-10-14T10:36:07.992 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.990+0000 7fbdba7fc700 1 -- 172.21.15.67:0/3473735625 <== mgr.4100 v2:172.21.15.72:6832/703488382 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264716 (secure 0 0 0) 0x7fbdb4005a20 con 0x7fbda806af00 2023-10-14T10:36:07.995 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-14T10:36:07.998 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.993+0000 7fbdcbdbc700 1 -- 172.21.15.67:0/3473735625 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fbda806af00 msgr2=0x7fbda806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:07.998 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.993+0000 7fbdcbdbc700 1 --2- 172.21.15.67:0/3473735625 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fbda806af00 0x7fbda806d3c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fbdc40cc240 tx=0x7fbdb40091c0 comp rx=0 tx=0).stop 2023-10-14T10:36:07.998 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.993+0000 7fbdcbdbc700 1 -- 172.21.15.67:0/3473735625 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbdc40b1f50 msgr2=0x7fbdc40c92d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:07.998 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.993+0000 7fbdcbdbc700 1 --2- 172.21.15.67:0/3473735625 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbdc40b1f50 0x7fbdc40c92d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbdc00090b0 tx=0x7fbdc0004750 comp rx=0 tx=0).stop 2023-10-14T10:36:07.999 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.993+0000 7fbdcbdbc700 1 -- 172.21.15.67:0/3473735625 shutdown_connections 2023-10-14T10:36:07.999 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.993+0000 7fbdcbdbc700 1 --2- 172.21.15.67:0/3473735625 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fbda806af00 0x7fbda806d3c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:07.999 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.993+0000 7fbdcbdbc700 1 --2- 172.21.15.67:0/3473735625 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbdc415ab10 0x7fbdc4158300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:07.999 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.993+0000 7fbdcbdbc700 1 --2- 172.21.15.67:0/3473735625 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbdc40b1f50 0x7fbdc40c92d0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:07.999 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.993+0000 7fbdcbdbc700 1 --2- 172.21.15.67:0/3473735625 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fbdc40b2400 0x7fbdc4155de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:07.999 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.993+0000 7fbdcbdbc700 1 -- 172.21.15.67:0/3473735625 >> 172.21.15.67:0/3473735625 conn(0x7fbdc406dae0 msgr2=0x7fbdc40ccfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:08.000 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.993+0000 7fbdcbdbc700 1 -- 172.21.15.67:0/3473735625 shutdown_connections 2023-10-14T10:36:08.000 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:07.993+0000 7fbdcbdbc700 1 -- 172.21.15.67:0/3473735625 wait complete. 2023-10-14T10:36:08.000 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2023-10-14T10:36:08.007 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2023-10-14T10:36:07.755633+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219164,"kb_used_data":2984,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755556,"statfs":{"total":773094113280,"available":772869689344,"internally_reserved":0,"allocated":3055616,"data_stored":1816468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12742,"internal_metadata":221236794},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.930067"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.847283+0000","last_change":"2023-10-14T10:36:03.869543+0000","last_active":"2023-10-14T10:36:04.847283+0000","last_peered":"2023-10-14T10:36:04.847283+0000","last_clean":"2023-10-14T10:36:04.847283+0000","last_became_active":"2023-10-14T10:36:03.868884+0000","last_became_peered":"2023-10-14T10:36:03.868884+0000","last_unstale":"2023-10-14T10:36:04.847283+0000","last_undegraded":"2023-10-14T10:36:04.847283+0000","last_fullsized":"2023-10-14T10:36:04.847283+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:31:25.399614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.841189+0000","last_change":"2023-10-14T10:36:03.867145+0000","last_active":"2023-10-14T10:36:04.841189+0000","last_peered":"2023-10-14T10:36:04.841189+0000","last_clean":"2023-10-14T10:36:04.841189+0000","last_became_active":"2023-10-14T10:36:03.866743+0000","last_became_peered":"2023-10-14T10:36:03.866743+0000","last_unstale":"2023-10-14T10:36:04.841189+0000","last_undegraded":"2023-10-14T10:36:04.841189+0000","last_fullsized":"2023-10-14T10:36:04.841189+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T18:49:18.106622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.483814+0000","last_change":"2023-10-14T10:36:03.861171+0000","last_active":"2023-10-14T10:36:05.483814+0000","last_peered":"2023-10-14T10:36:05.483814+0000","last_clean":"2023-10-14T10:36:05.483814+0000","last_became_active":"2023-10-14T10:36:03.860860+0000","last_became_peered":"2023-10-14T10:36:03.860860+0000","last_unstale":"2023-10-14T10:36:05.483814+0000","last_undegraded":"2023-10-14T10:36:05.483814+0000","last_fullsized":"2023-10-14T10:36:05.483814+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T17:28:51.336361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.845699+0000","last_change":"2023-10-14T10:36:03.861631+0000","last_active":"2023-10-14T10:36:04.845699+0000","last_peered":"2023-10-14T10:36:04.845699+0000","last_clean":"2023-10-14T10:36:04.845699+0000","last_became_active":"2023-10-14T10:36:03.861291+0000","last_became_peered":"2023-10-14T10:36:03.861291+0000","last_unstale":"2023-10-14T10:36:04.845699+0000","last_undegraded":"2023-10-14T10:36:04.845699+0000","last_fullsized":"2023-10-14T10:36:04.845699+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T17:52:45.149486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.846931+0000","last_change":"2023-10-14T10:36:03.870469+0000","last_active":"2023-10-14T10:36:04.846931+0000","last_peered":"2023-10-14T10:36:04.846931+0000","last_clean":"2023-10-14T10:36:04.846931+0000","last_became_active":"2023-10-14T10:36:03.865743+0000","last_became_peered":"2023-10-14T10:36:03.865743+0000","last_unstale":"2023-10-14T10:36:04.846931+0000","last_undegraded":"2023-10-14T10:36:04.846931+0000","last_fullsized":"2023-10-14T10:36:04.846931+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:33:09.576282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.845553+0000","last_change":"2023-10-14T10:36:03.862386+0000","last_active":"2023-10-14T10:36:04.845553+0000","last_peered":"2023-10-14T10:36:04.845553+0000","last_clean":"2023-10-14T10:36:04.845553+0000","last_became_active":"2023-10-14T10:36:03.861705+0000","last_became_peered":"2023-10-14T10:36:03.861705+0000","last_unstale":"2023-10-14T10:36:04.845553+0000","last_undegraded":"2023-10-14T10:36:04.845553+0000","last_fullsized":"2023-10-14T10:36:04.845553+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:47:42.089047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.846829+0000","last_change":"2023-10-14T10:36:03.865502+0000","last_active":"2023-10-14T10:36:04.846829+0000","last_peered":"2023-10-14T10:36:04.846829+0000","last_clean":"2023-10-14T10:36:04.846829+0000","last_became_active":"2023-10-14T10:36:03.865238+0000","last_became_peered":"2023-10-14T10:36:03.865238+0000","last_unstale":"2023-10-14T10:36:04.846829+0000","last_undegraded":"2023-10-14T10:36:04.846829+0000","last_fullsized":"2023-10-14T10:36:04.846829+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T20:52:07.681036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.482589+0000","last_change":"2023-10-14T10:36:03.859238+0000","last_active":"2023-10-14T10:36:05.482589+0000","last_peered":"2023-10-14T10:36:05.482589+0000","last_clean":"2023-10-14T10:36:05.482589+0000","last_became_active":"2023-10-14T10:36:03.858981+0000","last_became_peered":"2023-10-14T10:36:03.858981+0000","last_unstale":"2023-10-14T10:36:05.482589+0000","last_undegraded":"2023-10-14T10:36:05.482589+0000","last_fullsized":"2023-10-14T10:36:05.482589+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T12:18:09.680616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.847259+0000","last_change":"2023-10-14T10:36:03.868883+0000","last_active":"2023-10-14T10:36:04.847259+0000","last_peered":"2023-10-14T10:36:04.847259+0000","last_clean":"2023-10-14T10:36:04.847259+0000","last_became_active":"2023-10-14T10:36:03.868019+0000","last_became_peered":"2023-10-14T10:36:03.868019+0000","last_unstale":"2023-10-14T10:36:04.847259+0000","last_undegraded":"2023-10-14T10:36:04.847259+0000","last_fullsized":"2023-10-14T10:36:04.847259+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T17:08:17.694068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.482896+0000","last_change":"2023-10-14T10:36:03.861523+0000","last_active":"2023-10-14T10:36:05.482896+0000","last_peered":"2023-10-14T10:36:05.482896+0000","last_clean":"2023-10-14T10:36:05.482896+0000","last_became_active":"2023-10-14T10:36:03.860708+0000","last_became_peered":"2023-10-14T10:36:03.860708+0000","last_unstale":"2023-10-14T10:36:05.482896+0000","last_undegraded":"2023-10-14T10:36:05.482896+0000","last_fullsized":"2023-10-14T10:36:05.482896+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:07:32.070142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.482672+0000","last_change":"2023-10-14T10:36:03.861278+0000","last_active":"2023-10-14T10:36:05.482672+0000","last_peered":"2023-10-14T10:36:05.482672+0000","last_clean":"2023-10-14T10:36:05.482672+0000","last_became_active":"2023-10-14T10:36:03.860994+0000","last_became_peered":"2023-10-14T10:36:03.860994+0000","last_unstale":"2023-10-14T10:36:05.482672+0000","last_undegraded":"2023-10-14T10:36:05.482672+0000","last_fullsized":"2023-10-14T10:36:05.482672+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T12:18:09.680616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.482785+0000","last_change":"2023-10-14T10:36:03.871181+0000","last_active":"2023-10-14T10:36:05.482785+0000","last_peered":"2023-10-14T10:36:05.482785+0000","last_clean":"2023-10-14T10:36:05.482785+0000","last_became_active":"2023-10-14T10:36:03.870875+0000","last_became_peered":"2023-10-14T10:36:03.870875+0000","last_unstale":"2023-10-14T10:36:05.482785+0000","last_undegraded":"2023-10-14T10:36:05.482785+0000","last_fullsized":"2023-10-14T10:36:05.482785+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:50:35.924528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.482529+0000","last_change":"2023-10-14T10:36:03.857251+0000","last_active":"2023-10-14T10:36:05.482529+0000","last_peered":"2023-10-14T10:36:05.482529+0000","last_clean":"2023-10-14T10:36:05.482529+0000","last_became_active":"2023-10-14T10:36:03.856908+0000","last_became_peered":"2023-10-14T10:36:03.856908+0000","last_unstale":"2023-10-14T10:36:05.482529+0000","last_undegraded":"2023-10-14T10:36:05.482529+0000","last_fullsized":"2023-10-14T10:36:05.482529+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T18:50:26.625744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.484165+0000","last_change":"2023-10-14T10:36:03.858760+0000","last_active":"2023-10-14T10:36:05.484165+0000","last_peered":"2023-10-14T10:36:05.484165+0000","last_clean":"2023-10-14T10:36:05.484165+0000","last_became_active":"2023-10-14T10:36:03.854779+0000","last_became_peered":"2023-10-14T10:36:03.854779+0000","last_unstale":"2023-10-14T10:36:05.484165+0000","last_undegraded":"2023-10-14T10:36:05.484165+0000","last_fullsized":"2023-10-14T10:36:05.484165+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T18:49:18.106622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.840868+0000","last_change":"2023-10-14T10:36:03.861476+0000","last_active":"2023-10-14T10:36:04.840868+0000","last_peered":"2023-10-14T10:36:04.840868+0000","last_clean":"2023-10-14T10:36:04.840868+0000","last_became_active":"2023-10-14T10:36:03.859641+0000","last_became_peered":"2023-10-14T10:36:03.859641+0000","last_unstale":"2023-10-14T10:36:04.840868+0000","last_undegraded":"2023-10-14T10:36:04.840868+0000","last_fullsized":"2023-10-14T10:36:04.840868+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T21:40:43.364841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.845130+0000","last_change":"2023-10-14T10:36:03.858710+0000","last_active":"2023-10-14T10:36:04.845130+0000","last_peered":"2023-10-14T10:36:04.845130+0000","last_clean":"2023-10-14T10:36:04.845130+0000","last_became_active":"2023-10-14T10:36:03.858456+0000","last_became_peered":"2023-10-14T10:36:03.858456+0000","last_unstale":"2023-10-14T10:36:04.845130+0000","last_undegraded":"2023-10-14T10:36:04.845130+0000","last_fullsized":"2023-10-14T10:36:04.845130+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:07:32.070142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.845836+0000","last_change":"2023-10-14T10:36:03.867067+0000","last_active":"2023-10-14T10:36:04.845836+0000","last_peered":"2023-10-14T10:36:04.845836+0000","last_clean":"2023-10-14T10:36:04.845836+0000","last_became_active":"2023-10-14T10:36:03.866802+0000","last_became_peered":"2023-10-14T10:36:03.866802+0000","last_unstale":"2023-10-14T10:36:04.845836+0000","last_undegraded":"2023-10-14T10:36:04.845836+0000","last_fullsized":"2023-10-14T10:36:04.845836+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:09:22.818455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.482788+0000","last_change":"2023-10-14T10:36:03.869409+0000","last_active":"2023-10-14T10:36:05.482788+0000","last_peered":"2023-10-14T10:36:05.482788+0000","last_clean":"2023-10-14T10:36:05.482788+0000","last_became_active":"2023-10-14T10:36:03.868961+0000","last_became_peered":"2023-10-14T10:36:03.868961+0000","last_unstale":"2023-10-14T10:36:05.482788+0000","last_undegraded":"2023-10-14T10:36:05.482788+0000","last_fullsized":"2023-10-14T10:36:05.482788+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:10:10.517040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.847011+0000","last_change":"2023-10-14T10:36:03.864794+0000","last_active":"2023-10-14T10:36:04.847011+0000","last_peered":"2023-10-14T10:36:04.847011+0000","last_clean":"2023-10-14T10:36:04.847011+0000","last_became_active":"2023-10-14T10:36:03.864195+0000","last_became_peered":"2023-10-14T10:36:03.864195+0000","last_unstale":"2023-10-14T10:36:04.847011+0000","last_undegraded":"2023-10-14T10:36:04.847011+0000","last_fullsized":"2023-10-14T10:36:04.847011+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T17:52:45.149486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.483618+0000","last_change":"2023-10-14T10:36:02.855169+0000","last_active":"2023-10-14T10:36:05.483618+0000","last_peered":"2023-10-14T10:36:05.483618+0000","last_clean":"2023-10-14T10:36:05.483618+0000","last_became_active":"2023-10-14T10:36:02.854817+0000","last_became_peered":"2023-10-14T10:36:02.854817+0000","last_unstale":"2023-10-14T10:36:05.483618+0000","last_undegraded":"2023-10-14T10:36:05.483618+0000","last_fullsized":"2023-10-14T10:36:05.483618+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T14:12:52.738066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.484369+0000","last_change":"2023-10-14T10:36:03.865118+0000","last_active":"2023-10-14T10:36:05.484369+0000","last_peered":"2023-10-14T10:36:05.484369+0000","last_clean":"2023-10-14T10:36:05.484369+0000","last_became_active":"2023-10-14T10:36:03.864579+0000","last_became_peered":"2023-10-14T10:36:03.864579+0000","last_unstale":"2023-10-14T10:36:05.484369+0000","last_undegraded":"2023-10-14T10:36:05.484369+0000","last_fullsized":"2023-10-14T10:36:05.484369+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:27:25.021169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.484313+0000","last_change":"2023-10-14T10:36:02.851587+0000","last_active":"2023-10-14T10:36:05.484313+0000","last_peered":"2023-10-14T10:36:05.484313+0000","last_clean":"2023-10-14T10:36:05.484313+0000","last_became_active":"2023-10-14T10:36:02.851276+0000","last_became_peered":"2023-10-14T10:36:02.851276+0000","last_unstale":"2023-10-14T10:36:05.484313+0000","last_undegraded":"2023-10-14T10:36:05.484313+0000","last_fullsized":"2023-10-14T10:36:05.484313+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T14:04:44.127519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.483564+0000","last_change":"2023-10-14T10:36:03.861320+0000","last_active":"2023-10-14T10:36:05.483564+0000","last_peered":"2023-10-14T10:36:05.483564+0000","last_clean":"2023-10-14T10:36:05.483564+0000","last_became_active":"2023-10-14T10:36:03.860684+0000","last_became_peered":"2023-10-14T10:36:03.860684+0000","last_unstale":"2023-10-14T10:36:05.483564+0000","last_undegraded":"2023-10-14T10:36:05.483564+0000","last_fullsized":"2023-10-14T10:36:05.483564+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:22:58.149187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.847230+0000","last_change":"2023-10-14T10:36:02.854569+0000","last_active":"2023-10-14T10:36:04.847230+0000","last_peered":"2023-10-14T10:36:04.847230+0000","last_clean":"2023-10-14T10:36:04.847230+0000","last_became_active":"2023-10-14T10:36:02.853518+0000","last_became_peered":"2023-10-14T10:36:02.853518+0000","last_unstale":"2023-10-14T10:36:04.847230+0000","last_undegraded":"2023-10-14T10:36:04.847230+0000","last_fullsized":"2023-10-14T10:36:04.847230+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:27:24.015552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.845887+0000","last_change":"2023-10-14T10:36:03.872077+0000","last_active":"2023-10-14T10:36:04.845887+0000","last_peered":"2023-10-14T10:36:04.845887+0000","last_clean":"2023-10-14T10:36:04.845887+0000","last_became_active":"2023-10-14T10:36:03.871740+0000","last_became_peered":"2023-10-14T10:36:03.871740+0000","last_unstale":"2023-10-14T10:36:04.845887+0000","last_undegraded":"2023-10-14T10:36:04.845887+0000","last_fullsized":"2023-10-14T10:36:04.845887+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T12:31:06.179885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.846883+0000","last_change":"2023-10-14T10:36:02.853780+0000","last_active":"2023-10-14T10:36:04.846883+0000","last_peered":"2023-10-14T10:36:04.846883+0000","last_clean":"2023-10-14T10:36:04.846883+0000","last_became_active":"2023-10-14T10:36:02.853450+0000","last_became_peered":"2023-10-14T10:36:02.853450+0000","last_unstale":"2023-10-14T10:36:04.846883+0000","last_undegraded":"2023-10-14T10:36:04.846883+0000","last_fullsized":"2023-10-14T10:36:04.846883+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T10:42:23.021229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.845567+0000","last_change":"2023-10-14T10:36:03.858379+0000","last_active":"2023-10-14T10:36:04.845567+0000","last_peered":"2023-10-14T10:36:04.845567+0000","last_clean":"2023-10-14T10:36:04.845567+0000","last_became_active":"2023-10-14T10:36:03.857904+0000","last_became_peered":"2023-10-14T10:36:03.857904+0000","last_unstale":"2023-10-14T10:36:04.845567+0000","last_undegraded":"2023-10-14T10:36:04.845567+0000","last_fullsized":"2023-10-14T10:36:04.845567+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:33:09.576282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.482737+0000","last_change":"2023-10-14T10:36:02.854396+0000","last_active":"2023-10-14T10:36:05.482737+0000","last_peered":"2023-10-14T10:36:05.482737+0000","last_clean":"2023-10-14T10:36:05.482737+0000","last_became_active":"2023-10-14T10:36:02.854085+0000","last_became_peered":"2023-10-14T10:36:02.854085+0000","last_unstale":"2023-10-14T10:36:05.482737+0000","last_undegraded":"2023-10-14T10:36:05.482737+0000","last_fullsized":"2023-10-14T10:36:05.482737+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T14:12:52.738066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.483060+0000","last_change":"2023-10-14T10:36:03.855010+0000","last_active":"2023-10-14T10:36:05.483060+0000","last_peered":"2023-10-14T10:36:05.483060+0000","last_clean":"2023-10-14T10:36:05.483060+0000","last_became_active":"2023-10-14T10:36:03.854714+0000","last_became_peered":"2023-10-14T10:36:03.854714+0000","last_unstale":"2023-10-14T10:36:05.483060+0000","last_undegraded":"2023-10-14T10:36:05.483060+0000","last_fullsized":"2023-10-14T10:36:05.483060+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:29:51.596547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-14T10:36:03.865241+0000","last_change":"2023-10-14T10:36:02.847573+0000","last_active":"2023-10-14T10:36:03.865241+0000","last_peered":"2023-10-14T10:36:03.865241+0000","last_clean":"2023-10-14T10:36:03.865241+0000","last_became_active":"2023-10-14T10:36:02.847061+0000","last_became_peered":"2023-10-14T10:36:02.847061+0000","last_unstale":"2023-10-14T10:36:03.865241+0000","last_undegraded":"2023-10-14T10:36:03.865241+0000","last_fullsized":"2023-10-14T10:36:03.865241+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:50:34.918912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.484012+0000","last_change":"2023-10-14T10:36:03.866134+0000","last_active":"2023-10-14T10:36:05.484012+0000","last_peered":"2023-10-14T10:36:05.484012+0000","last_clean":"2023-10-14T10:36:05.484012+0000","last_became_active":"2023-10-14T10:36:03.865614+0000","last_became_peered":"2023-10-14T10:36:03.865614+0000","last_unstale":"2023-10-14T10:36:05.484012+0000","last_undegraded":"2023-10-14T10:36:05.484012+0000","last_fullsized":"2023-10-14T10:36:05.484012+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T18:42:51.749474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.482644+0000","last_change":"2023-10-14T10:36:02.846515+0000","last_active":"2023-10-14T10:36:05.482644+0000","last_peered":"2023-10-14T10:36:05.482644+0000","last_clean":"2023-10-14T10:36:05.482644+0000","last_became_active":"2023-10-14T10:36:02.845841+0000","last_became_peered":"2023-10-14T10:36:02.845841+0000","last_unstale":"2023-10-14T10:36:05.482644+0000","last_undegraded":"2023-10-14T10:36:05.482644+0000","last_fullsized":"2023-10-14T10:36:05.482644+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:22:57.143570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.483031+0000","last_change":"2023-10-14T10:36:03.865280+0000","last_active":"2023-10-14T10:36:05.483031+0000","last_peered":"2023-10-14T10:36:05.483031+0000","last_clean":"2023-10-14T10:36:05.483031+0000","last_became_active":"2023-10-14T10:36:03.864926+0000","last_became_peered":"2023-10-14T10:36:03.864926+0000","last_unstale":"2023-10-14T10:36:05.483031+0000","last_undegraded":"2023-10-14T10:36:05.483031+0000","last_fullsized":"2023-10-14T10:36:05.483031+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:05:52.086793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.845177+0000","last_change":"2023-10-14T10:36:02.843680+0000","last_active":"2023-10-14T10:36:04.845177+0000","last_peered":"2023-10-14T10:36:04.845177+0000","last_clean":"2023-10-14T10:36:04.845177+0000","last_became_active":"2023-10-14T10:36:02.843364+0000","last_became_peered":"2023-10-14T10:36:02.843364+0000","last_unstale":"2023-10-14T10:36:04.845177+0000","last_undegraded":"2023-10-14T10:36:04.845177+0000","last_fullsized":"2023-10-14T10:36:04.845177+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T10:59:16.502697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.841021+0000","last_change":"2023-10-14T10:36:03.862822+0000","last_active":"2023-10-14T10:36:04.841021+0000","last_peered":"2023-10-14T10:36:04.841021+0000","last_clean":"2023-10-14T10:36:04.841021+0000","last_became_active":"2023-10-14T10:36:03.862547+0000","last_became_peered":"2023-10-14T10:36:03.862547+0000","last_unstale":"2023-10-14T10:36:04.841021+0000","last_undegraded":"2023-10-14T10:36:04.841021+0000","last_fullsized":"2023-10-14T10:36:04.841021+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T18:42:51.749474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-14T10:36:03.851585+0000","last_change":"2023-10-14T10:36:02.846160+0000","last_active":"2023-10-14T10:36:03.851585+0000","last_peered":"2023-10-14T10:36:03.851585+0000","last_clean":"2023-10-14T10:36:03.851585+0000","last_became_active":"2023-10-14T10:36:02.845784+0000","last_became_peered":"2023-10-14T10:36:02.845784+0000","last_unstale":"2023-10-14T10:36:03.851585+0000","last_undegraded":"2023-10-14T10:36:03.851585+0000","last_fullsized":"2023-10-14T10:36:03.851585+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T12:29:01.996039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.845511+0000","last_change":"2023-10-14T10:36:03.864605+0000","last_active":"2023-10-14T10:36:04.845511+0000","last_peered":"2023-10-14T10:36:04.845511+0000","last_clean":"2023-10-14T10:36:04.845511+0000","last_became_active":"2023-10-14T10:36:03.864160+0000","last_became_peered":"2023-10-14T10:36:03.864160+0000","last_unstale":"2023-10-14T10:36:04.845511+0000","last_undegraded":"2023-10-14T10:36:04.845511+0000","last_fullsized":"2023-10-14T10:36:04.845511+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T20:52:07.681036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.845294+0000","last_change":"2023-10-14T10:36:02.845777+0000","last_active":"2023-10-14T10:36:04.845294+0000","last_peered":"2023-10-14T10:36:04.845294+0000","last_clean":"2023-10-14T10:36:04.845294+0000","last_became_active":"2023-10-14T10:36:02.845482+0000","last_became_peered":"2023-10-14T10:36:02.845482+0000","last_unstale":"2023-10-14T10:36:04.845294+0000","last_undegraded":"2023-10-14T10:36:04.845294+0000","last_fullsized":"2023-10-14T10:36:04.845294+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T14:04:44.127519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.483717+0000","last_change":"2023-10-14T10:36:03.861072+0000","last_active":"2023-10-14T10:36:05.483717+0000","last_peered":"2023-10-14T10:36:05.483717+0000","last_clean":"2023-10-14T10:36:05.483717+0000","last_became_active":"2023-10-14T10:36:03.860335+0000","last_became_peered":"2023-10-14T10:36:03.860335+0000","last_unstale":"2023-10-14T10:36:05.483717+0000","last_undegraded":"2023-10-14T10:36:05.483717+0000","last_fullsized":"2023-10-14T10:36:05.483717+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T12:18:09.680616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.841103+0000","last_change":"2023-10-14T10:36:02.845951+0000","last_active":"2023-10-14T10:36:04.841103+0000","last_peered":"2023-10-14T10:36:04.841103+0000","last_clean":"2023-10-14T10:36:04.841103+0000","last_became_active":"2023-10-14T10:36:02.845667+0000","last_became_peered":"2023-10-14T10:36:02.845667+0000","last_unstale":"2023-10-14T10:36:04.841103+0000","last_undegraded":"2023-10-14T10:36:04.841103+0000","last_fullsized":"2023-10-14T10:36:04.841103+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:00:17.008189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.482746+0000","last_change":"2023-10-14T10:36:03.871035+0000","last_active":"2023-10-14T10:36:05.482746+0000","last_peered":"2023-10-14T10:36:05.482746+0000","last_clean":"2023-10-14T10:36:05.482746+0000","last_became_active":"2023-10-14T10:36:03.870542+0000","last_became_peered":"2023-10-14T10:36:03.870542+0000","last_unstale":"2023-10-14T10:36:05.482746+0000","last_undegraded":"2023-10-14T10:36:05.482746+0000","last_fullsized":"2023-10-14T10:36:05.482746+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T14:05:05.847145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-14T10:36:03.874236+0000","last_change":"2023-10-14T10:36:02.847268+0000","last_active":"2023-10-14T10:36:03.874236+0000","last_peered":"2023-10-14T10:36:03.874236+0000","last_clean":"2023-10-14T10:36:03.874236+0000","last_became_active":"2023-10-14T10:36:02.846840+0000","last_became_peered":"2023-10-14T10:36:02.846840+0000","last_unstale":"2023-10-14T10:36:03.874236+0000","last_undegraded":"2023-10-14T10:36:03.874236+0000","last_fullsized":"2023-10-14T10:36:03.874236+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:22:57.143570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.847057+0000","last_change":"2023-10-14T10:36:03.864955+0000","last_active":"2023-10-14T10:36:04.847057+0000","last_peered":"2023-10-14T10:36:04.847057+0000","last_clean":"2023-10-14T10:36:04.847057+0000","last_became_active":"2023-10-14T10:36:03.864505+0000","last_became_peered":"2023-10-14T10:36:03.864505+0000","last_unstale":"2023-10-14T10:36:04.847057+0000","last_undegraded":"2023-10-14T10:36:04.847057+0000","last_fullsized":"2023-10-14T10:36:04.847057+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:00:36.181878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.483879+0000","last_change":"2023-10-14T10:36:02.856271+0000","last_active":"2023-10-14T10:36:05.483879+0000","last_peered":"2023-10-14T10:36:05.483879+0000","last_clean":"2023-10-14T10:36:05.483879+0000","last_became_active":"2023-10-14T10:36:02.856039+0000","last_became_peered":"2023-10-14T10:36:02.856039+0000","last_unstale":"2023-10-14T10:36:05.483879+0000","last_undegraded":"2023-10-14T10:36:05.483879+0000","last_fullsized":"2023-10-14T10:36:05.483879+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:29:50.590931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.484124+0000","last_change":"2023-10-14T10:36:03.868295+0000","last_active":"2023-10-14T10:36:05.484124+0000","last_peered":"2023-10-14T10:36:05.484124+0000","last_clean":"2023-10-14T10:36:05.484124+0000","last_became_active":"2023-10-14T10:36:03.867969+0000","last_became_peered":"2023-10-14T10:36:03.867969+0000","last_unstale":"2023-10-14T10:36:05.484124+0000","last_undegraded":"2023-10-14T10:36:05.484124+0000","last_fullsized":"2023-10-14T10:36:05.484124+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:05:52.086793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.845748+0000","last_change":"2023-10-14T10:36:02.845900+0000","last_active":"2023-10-14T10:36:04.845748+0000","last_peered":"2023-10-14T10:36:04.845748+0000","last_clean":"2023-10-14T10:36:04.845748+0000","last_became_active":"2023-10-14T10:36:02.845596+0000","last_became_peered":"2023-10-14T10:36:02.845596+0000","last_unstale":"2023-10-14T10:36:04.845748+0000","last_undegraded":"2023-10-14T10:36:04.845748+0000","last_fullsized":"2023-10-14T10:36:04.845748+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T21:40:42.359224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.846941+0000","last_change":"2023-10-14T10:36:03.870846+0000","last_active":"2023-10-14T10:36:04.846941+0000","last_peered":"2023-10-14T10:36:04.846941+0000","last_clean":"2023-10-14T10:36:04.846941+0000","last_became_active":"2023-10-14T10:36:03.869379+0000","last_became_peered":"2023-10-14T10:36:03.869379+0000","last_unstale":"2023-10-14T10:36:04.846941+0000","last_undegraded":"2023-10-14T10:36:04.846941+0000","last_fullsized":"2023-10-14T10:36:04.846941+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T10:36:37.449003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-14T10:36:03.851688+0000","last_change":"2023-10-14T10:36:02.847080+0000","last_active":"2023-10-14T10:36:03.851688+0000","last_peered":"2023-10-14T10:36:03.851688+0000","last_clean":"2023-10-14T10:36:03.851688+0000","last_became_active":"2023-10-14T10:36:02.846784+0000","last_became_peered":"2023-10-14T10:36:02.846784+0000","last_unstale":"2023-10-14T10:36:03.851688+0000","last_undegraded":"2023-10-14T10:36:03.851688+0000","last_fullsized":"2023-10-14T10:36:03.851688+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:29:50.590931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.482758+0000","last_change":"2023-10-14T10:36:03.862316+0000","last_active":"2023-10-14T10:36:05.482758+0000","last_peered":"2023-10-14T10:36:05.482758+0000","last_clean":"2023-10-14T10:36:05.482758+0000","last_became_active":"2023-10-14T10:36:03.862022+0000","last_became_peered":"2023-10-14T10:36:03.862022+0000","last_unstale":"2023-10-14T10:36:05.482758+0000","last_undegraded":"2023-10-14T10:36:05.482758+0000","last_fullsized":"2023-10-14T10:36:05.482758+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T21:40:43.364841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-14T10:36:03.867586+0000","last_change":"2023-10-14T10:36:02.846811+0000","last_active":"2023-10-14T10:36:03.867586+0000","last_peered":"2023-10-14T10:36:03.867586+0000","last_clean":"2023-10-14T10:36:03.867586+0000","last_became_active":"2023-10-14T10:36:02.846487+0000","last_became_peered":"2023-10-14T10:36:02.846487+0000","last_unstale":"2023-10-14T10:36:03.867586+0000","last_undegraded":"2023-10-14T10:36:03.867586+0000","last_fullsized":"2023-10-14T10:36:03.867586+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T18:47:26.337837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.846699+0000","last_change":"2023-10-14T10:36:03.872480+0000","last_active":"2023-10-14T10:36:04.846699+0000","last_peered":"2023-10-14T10:36:04.846699+0000","last_clean":"2023-10-14T10:36:04.846699+0000","last_became_active":"2023-10-14T10:36:03.872231+0000","last_became_peered":"2023-10-14T10:36:03.872231+0000","last_unstale":"2023-10-14T10:36:04.846699+0000","last_undegraded":"2023-10-14T10:36:04.846699+0000","last_fullsized":"2023-10-14T10:36:04.846699+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:47:42.089047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.482611+0000","last_change":"2023-10-14T10:36:02.851223+0000","last_active":"2023-10-14T10:36:05.482611+0000","last_peered":"2023-10-14T10:36:05.482611+0000","last_clean":"2023-10-14T10:36:05.482611+0000","last_became_active":"2023-10-14T10:36:02.850955+0000","last_became_peered":"2023-10-14T10:36:02.850955+0000","last_unstale":"2023-10-14T10:36:05.482611+0000","last_undegraded":"2023-10-14T10:36:05.482611+0000","last_fullsized":"2023-10-14T10:36:05.482611+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:08:47.187941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.847107+0000","last_change":"2023-10-14T10:36:03.871637+0000","last_active":"2023-10-14T10:36:04.847107+0000","last_peered":"2023-10-14T10:36:04.847107+0000","last_clean":"2023-10-14T10:36:04.847107+0000","last_became_active":"2023-10-14T10:36:03.871362+0000","last_became_peered":"2023-10-14T10:36:03.871362+0000","last_unstale":"2023-10-14T10:36:04.847107+0000","last_undegraded":"2023-10-14T10:36:04.847107+0000","last_fullsized":"2023-10-14T10:36:04.847107+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:09:22.818455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.841156+0000","last_change":"2023-10-14T10:36:02.851439+0000","last_active":"2023-10-14T10:36:04.841156+0000","last_peered":"2023-10-14T10:36:04.841156+0000","last_clean":"2023-10-14T10:36:04.841156+0000","last_became_active":"2023-10-14T10:36:02.851121+0000","last_became_peered":"2023-10-14T10:36:02.851121+0000","last_unstale":"2023-10-14T10:36:04.841156+0000","last_undegraded":"2023-10-14T10:36:04.841156+0000","last_fullsized":"2023-10-14T10:36:04.841156+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T10:42:23.021229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.845753+0000","last_change":"2023-10-14T10:36:03.863345+0000","last_active":"2023-10-14T10:36:04.845753+0000","last_peered":"2023-10-14T10:36:04.845753+0000","last_clean":"2023-10-14T10:36:04.845753+0000","last_became_active":"2023-10-14T10:36:03.861227+0000","last_became_peered":"2023-10-14T10:36:03.861227+0000","last_unstale":"2023-10-14T10:36:04.845753+0000","last_undegraded":"2023-10-14T10:36:04.845753+0000","last_fullsized":"2023-10-14T10:36:04.845753+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:00:36.181878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.845649+0000","last_change":"2023-10-14T10:36:02.846354+0000","last_active":"2023-10-14T10:36:04.845649+0000","last_peered":"2023-10-14T10:36:04.845649+0000","last_clean":"2023-10-14T10:36:04.845649+0000","last_became_active":"2023-10-14T10:36:02.845950+0000","last_became_peered":"2023-10-14T10:36:02.845950+0000","last_unstale":"2023-10-14T10:36:04.845649+0000","last_undegraded":"2023-10-14T10:36:04.845649+0000","last_fullsized":"2023-10-14T10:36:04.845649+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T17:28:50.330745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.482785+0000","last_change":"2023-10-14T10:36:03.860805+0000","last_active":"2023-10-14T10:36:05.482785+0000","last_peered":"2023-10-14T10:36:05.482785+0000","last_clean":"2023-10-14T10:36:05.482785+0000","last_became_active":"2023-10-14T10:36:03.860218+0000","last_became_peered":"2023-10-14T10:36:03.860218+0000","last_unstale":"2023-10-14T10:36:05.482785+0000","last_undegraded":"2023-10-14T10:36:05.482785+0000","last_fullsized":"2023-10-14T10:36:05.482785+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T12:29:03.001656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.840832+0000","last_change":"2023-10-14T10:36:03.857512+0000","last_active":"2023-10-14T10:36:04.840832+0000","last_peered":"2023-10-14T10:36:04.840832+0000","last_clean":"2023-10-14T10:36:04.840832+0000","last_became_active":"2023-10-14T10:36:03.855687+0000","last_became_peered":"2023-10-14T10:36:03.855687+0000","last_unstale":"2023-10-14T10:36:04.840832+0000","last_undegraded":"2023-10-14T10:36:04.840832+0000","last_fullsized":"2023-10-14T10:36:04.840832+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T17:28:51.336361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-14T10:36:03.843979+0000","last_change":"2023-10-14T10:36:02.843620+0000","last_active":"2023-10-14T10:36:03.843979+0000","last_peered":"2023-10-14T10:36:03.843979+0000","last_clean":"2023-10-14T10:36:03.843979+0000","last_became_active":"2023-10-14T10:36:02.843324+0000","last_became_peered":"2023-10-14T10:36:02.843324+0000","last_unstale":"2023-10-14T10:36:03.843979+0000","last_undegraded":"2023-10-14T10:36:03.843979+0000","last_fullsized":"2023-10-14T10:36:03.843979+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T14:12:49.629312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.482850+0000","last_change":"2023-10-14T10:36:03.858691+0000","last_active":"2023-10-14T10:36:05.482850+0000","last_peered":"2023-10-14T10:36:05.482850+0000","last_clean":"2023-10-14T10:36:05.482850+0000","last_became_active":"2023-10-14T10:36:03.858420+0000","last_became_peered":"2023-10-14T10:36:03.858420+0000","last_unstale":"2023-10-14T10:36:05.482850+0000","last_undegraded":"2023-10-14T10:36:05.482850+0000","last_fullsized":"2023-10-14T10:36:05.482850+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T18:42:51.749474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.845260+0000","last_change":"2023-10-14T10:36:02.840533+0000","last_active":"2023-10-14T10:36:04.845260+0000","last_peered":"2023-10-14T10:36:04.845260+0000","last_clean":"2023-10-14T10:36:04.845260+0000","last_became_active":"2023-10-14T10:36:02.839919+0000","last_became_peered":"2023-10-14T10:36:02.839919+0000","last_unstale":"2023-10-14T10:36:04.845260+0000","last_undegraded":"2023-10-14T10:36:04.845260+0000","last_fullsized":"2023-10-14T10:36:04.845260+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:29:50.590931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":165,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-14T10:36:03.847417+0000","last_change":"2023-10-14T10:35:59.770163+0000","last_active":"2023-10-14T10:36:03.847417+0000","last_peered":"2023-10-14T10:36:03.847417+0000","last_clean":"2023-10-14T10:36:03.847417+0000","last_became_active":"2023-10-14T10:35:59.769199+0000","last_became_peered":"2023-10-14T10:35:59.769199+0000","last_unstale":"2023-10-14T10:36:03.847417+0000","last_undegraded":"2023-10-14T10:36:03.847417+0000","last_fullsized":"2023-10-14T10:36:03.847417+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:35:58.675572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:35:58.675572+0000","last_clean_scrub_stamp":"2023-10-14T10:35:58.675572+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:03:25.281187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.484102+0000","last_change":"2023-10-14T10:36:03.872244+0000","last_active":"2023-10-14T10:36:05.484102+0000","last_peered":"2023-10-14T10:36:05.484102+0000","last_clean":"2023-10-14T10:36:05.484102+0000","last_became_active":"2023-10-14T10:36:03.871841+0000","last_became_peered":"2023-10-14T10:36:03.871841+0000","last_unstale":"2023-10-14T10:36:05.484102+0000","last_undegraded":"2023-10-14T10:36:05.484102+0000","last_fullsized":"2023-10-14T10:36:05.484102+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:07:32.070142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.846600+0000","last_change":"2023-10-14T10:36:02.854777+0000","last_active":"2023-10-14T10:36:04.846600+0000","last_peered":"2023-10-14T10:36:04.846600+0000","last_clean":"2023-10-14T10:36:04.846600+0000","last_became_active":"2023-10-14T10:36:02.854126+0000","last_became_peered":"2023-10-14T10:36:02.854126+0000","last_unstale":"2023-10-14T10:36:04.846600+0000","last_undegraded":"2023-10-14T10:36:04.846600+0000","last_fullsized":"2023-10-14T10:36:04.846600+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:22:57.143570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.847151+0000","last_change":"2023-10-14T10:36:02.853800+0000","last_active":"2023-10-14T10:36:04.847151+0000","last_peered":"2023-10-14T10:36:04.847151+0000","last_clean":"2023-10-14T10:36:04.847151+0000","last_became_active":"2023-10-14T10:36:02.853508+0000","last_became_peered":"2023-10-14T10:36:02.853508+0000","last_unstale":"2023-10-14T10:36:04.847151+0000","last_undegraded":"2023-10-14T10:36:04.847151+0000","last_fullsized":"2023-10-14T10:36:04.847151+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:05:51.081176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.841103+0000","last_change":"2023-10-14T10:36:03.856928+0000","last_active":"2023-10-14T10:36:04.841103+0000","last_peered":"2023-10-14T10:36:04.841103+0000","last_clean":"2023-10-14T10:36:04.841103+0000","last_became_active":"2023-10-14T10:36:03.856616+0000","last_became_peered":"2023-10-14T10:36:03.856616+0000","last_unstale":"2023-10-14T10:36:04.841103+0000","last_undegraded":"2023-10-14T10:36:04.841103+0000","last_fullsized":"2023-10-14T10:36:04.841103+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:07:32.070142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.840950+0000","last_change":"2023-10-14T10:36:02.847787+0000","last_active":"2023-10-14T10:36:04.840950+0000","last_peered":"2023-10-14T10:36:04.840950+0000","last_clean":"2023-10-14T10:36:04.840950+0000","last_became_active":"2023-10-14T10:36:02.847468+0000","last_became_peered":"2023-10-14T10:36:02.847468+0000","last_unstale":"2023-10-14T10:36:04.840950+0000","last_undegraded":"2023-10-14T10:36:04.840950+0000","last_fullsized":"2023-10-14T10:36:04.840950+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T18:50:25.620127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.484430+0000","last_change":"2023-10-14T10:36:03.869759+0000","last_active":"2023-10-14T10:36:05.484430+0000","last_peered":"2023-10-14T10:36:05.484430+0000","last_clean":"2023-10-14T10:36:05.484430+0000","last_became_active":"2023-10-14T10:36:03.869404+0000","last_became_peered":"2023-10-14T10:36:03.869404+0000","last_unstale":"2023-10-14T10:36:05.484430+0000","last_undegraded":"2023-10-14T10:36:05.484430+0000","last_fullsized":"2023-10-14T10:36:05.484430+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:47:42.089047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-14T10:36:03.868372+0000","last_change":"2023-10-14T10:36:02.853796+0000","last_active":"2023-10-14T10:36:03.868372+0000","last_peered":"2023-10-14T10:36:03.868372+0000","last_clean":"2023-10-14T10:36:03.868372+0000","last_became_active":"2023-10-14T10:36:02.853468+0000","last_became_peered":"2023-10-14T10:36:02.853468+0000","last_unstale":"2023-10-14T10:36:03.868372+0000","last_undegraded":"2023-10-14T10:36:03.868372+0000","last_fullsized":"2023-10-14T10:36:03.868372+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:14:10.501109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.483634+0000","last_change":"2023-10-14T10:36:03.854935+0000","last_active":"2023-10-14T10:36:05.483634+0000","last_peered":"2023-10-14T10:36:05.483634+0000","last_clean":"2023-10-14T10:36:05.483634+0000","last_became_active":"2023-10-14T10:36:03.854648+0000","last_became_peered":"2023-10-14T10:36:03.854648+0000","last_unstale":"2023-10-14T10:36:05.483634+0000","last_undegraded":"2023-10-14T10:36:05.483634+0000","last_fullsized":"2023-10-14T10:36:05.483634+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T10:42:24.026846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.846475+0000","last_change":"2023-10-14T10:36:02.854850+0000","last_active":"2023-10-14T10:36:04.846475+0000","last_peered":"2023-10-14T10:36:04.846475+0000","last_clean":"2023-10-14T10:36:04.846475+0000","last_became_active":"2023-10-14T10:36:02.854442+0000","last_became_peered":"2023-10-14T10:36:02.854442+0000","last_unstale":"2023-10-14T10:36:04.846475+0000","last_undegraded":"2023-10-14T10:36:04.846475+0000","last_fullsized":"2023-10-14T10:36:04.846475+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T12:29:01.996039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.482660+0000","last_change":"2023-10-14T10:36:03.859500+0000","last_active":"2023-10-14T10:36:05.482660+0000","last_peered":"2023-10-14T10:36:05.482660+0000","last_clean":"2023-10-14T10:36:05.482660+0000","last_became_active":"2023-10-14T10:36:03.859247+0000","last_became_peered":"2023-10-14T10:36:03.859247+0000","last_unstale":"2023-10-14T10:36:05.482660+0000","last_undegraded":"2023-10-14T10:36:05.482660+0000","last_fullsized":"2023-10-14T10:36:05.482660+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:14:11.506725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.840826+0000","last_change":"2023-10-14T10:36:02.854036+0000","last_active":"2023-10-14T10:36:04.840826+0000","last_peered":"2023-10-14T10:36:04.840826+0000","last_clean":"2023-10-14T10:36:04.840826+0000","last_became_active":"2023-10-14T10:36:02.853723+0000","last_became_peered":"2023-10-14T10:36:02.853723+0000","last_unstale":"2023-10-14T10:36:04.840826+0000","last_undegraded":"2023-10-14T10:36:04.840826+0000","last_fullsized":"2023-10-14T10:36:04.840826+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:29:50.590931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.482772+0000","last_change":"2023-10-14T10:36:03.856047+0000","last_active":"2023-10-14T10:36:05.482772+0000","last_peered":"2023-10-14T10:36:05.482772+0000","last_clean":"2023-10-14T10:36:05.482772+0000","last_became_active":"2023-10-14T10:36:03.855747+0000","last_became_peered":"2023-10-14T10:36:03.855747+0000","last_unstale":"2023-10-14T10:36:05.482772+0000","last_undegraded":"2023-10-14T10:36:05.482772+0000","last_fullsized":"2023-10-14T10:36:05.482772+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T21:40:43.364841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.841238+0000","last_change":"2023-10-14T10:36:02.845255+0000","last_active":"2023-10-14T10:36:04.841238+0000","last_peered":"2023-10-14T10:36:04.841238+0000","last_clean":"2023-10-14T10:36:04.841238+0000","last_became_active":"2023-10-14T10:36:02.844939+0000","last_became_peered":"2023-10-14T10:36:02.844939+0000","last_unstale":"2023-10-14T10:36:04.841238+0000","last_undegraded":"2023-10-14T10:36:04.841238+0000","last_fullsized":"2023-10-14T10:36:04.841238+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T12:29:01.996039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.482888+0000","last_change":"2023-10-14T10:36:03.865197+0000","last_active":"2023-10-14T10:36:05.482888+0000","last_peered":"2023-10-14T10:36:05.482888+0000","last_clean":"2023-10-14T10:36:05.482888+0000","last_became_active":"2023-10-14T10:36:03.864885+0000","last_became_peered":"2023-10-14T10:36:03.864885+0000","last_unstale":"2023-10-14T10:36:05.482888+0000","last_undegraded":"2023-10-14T10:36:05.482888+0000","last_fullsized":"2023-10-14T10:36:05.482888+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:07:32.070142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.847061+0000","last_change":"2023-10-14T10:36:02.853880+0000","last_active":"2023-10-14T10:36:04.847061+0000","last_peered":"2023-10-14T10:36:04.847061+0000","last_clean":"2023-10-14T10:36:04.847061+0000","last_became_active":"2023-10-14T10:36:02.853574+0000","last_became_peered":"2023-10-14T10:36:02.853574+0000","last_unstale":"2023-10-14T10:36:04.847061+0000","last_undegraded":"2023-10-14T10:36:04.847061+0000","last_fullsized":"2023-10-14T10:36:04.847061+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T10:59:16.502697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.482245+0000","last_change":"2023-10-14T10:36:03.865872+0000","last_active":"2023-10-14T10:36:05.482245+0000","last_peered":"2023-10-14T10:36:05.482245+0000","last_clean":"2023-10-14T10:36:05.482245+0000","last_became_active":"2023-10-14T10:36:03.865624+0000","last_became_peered":"2023-10-14T10:36:03.865624+0000","last_unstale":"2023-10-14T10:36:05.482245+0000","last_undegraded":"2023-10-14T10:36:05.482245+0000","last_fullsized":"2023-10-14T10:36:05.482245+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T18:50:26.625744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-14T10:36:03.842738+0000","last_change":"2023-10-14T10:36:02.840500+0000","last_active":"2023-10-14T10:36:03.842738+0000","last_peered":"2023-10-14T10:36:03.842738+0000","last_clean":"2023-10-14T10:36:03.842738+0000","last_became_active":"2023-10-14T10:36:02.839996+0000","last_became_peered":"2023-10-14T10:36:02.839996+0000","last_unstale":"2023-10-14T10:36:03.842738+0000","last_undegraded":"2023-10-14T10:36:03.842738+0000","last_fullsized":"2023-10-14T10:36:03.842738+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T21:06:31.726029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.847199+0000","last_change":"2023-10-14T10:36:03.868941+0000","last_active":"2023-10-14T10:36:04.847199+0000","last_peered":"2023-10-14T10:36:04.847199+0000","last_clean":"2023-10-14T10:36:04.847199+0000","last_became_active":"2023-10-14T10:36:03.868609+0000","last_became_peered":"2023-10-14T10:36:03.868609+0000","last_unstale":"2023-10-14T10:36:04.847199+0000","last_undegraded":"2023-10-14T10:36:04.847199+0000","last_fullsized":"2023-10-14T10:36:04.847199+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:14:41.057137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.841048+0000","last_change":"2023-10-14T10:36:02.848004+0000","last_active":"2023-10-14T10:36:04.841048+0000","last_peered":"2023-10-14T10:36:04.841048+0000","last_clean":"2023-10-14T10:36:04.841048+0000","last_became_active":"2023-10-14T10:36:02.847494+0000","last_became_peered":"2023-10-14T10:36:02.847494+0000","last_unstale":"2023-10-14T10:36:04.841048+0000","last_undegraded":"2023-10-14T10:36:04.841048+0000","last_fullsized":"2023-10-14T10:36:04.841048+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T14:12:52.738066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.845506+0000","last_change":"2023-10-14T10:36:03.863211+0000","last_active":"2023-10-14T10:36:04.845506+0000","last_peered":"2023-10-14T10:36:04.845506+0000","last_clean":"2023-10-14T10:36:04.845506+0000","last_became_active":"2023-10-14T10:36:03.862894+0000","last_became_peered":"2023-10-14T10:36:03.862894+0000","last_unstale":"2023-10-14T10:36:04.845506+0000","last_undegraded":"2023-10-14T10:36:04.845506+0000","last_fullsized":"2023-10-14T10:36:04.845506+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:27:25.021169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-14T10:36:03.864535+0000","last_change":"2023-10-14T10:36:02.842899+0000","last_active":"2023-10-14T10:36:03.864535+0000","last_peered":"2023-10-14T10:36:03.864535+0000","last_clean":"2023-10-14T10:36:03.864535+0000","last_became_active":"2023-10-14T10:36:02.842620+0000","last_became_peered":"2023-10-14T10:36:02.842620+0000","last_unstale":"2023-10-14T10:36:03.864535+0000","last_undegraded":"2023-10-14T10:36:03.864535+0000","last_fullsized":"2023-10-14T10:36:03.864535+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T12:44:16.134070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.847179+0000","last_change":"2023-10-14T10:36:03.872451+0000","last_active":"2023-10-14T10:36:04.847179+0000","last_peered":"2023-10-14T10:36:04.847179+0000","last_clean":"2023-10-14T10:36:04.847179+0000","last_became_active":"2023-10-14T10:36:03.872212+0000","last_became_peered":"2023-10-14T10:36:03.872212+0000","last_unstale":"2023-10-14T10:36:04.847179+0000","last_undegraded":"2023-10-14T10:36:04.847179+0000","last_fullsized":"2023-10-14T10:36:04.847179+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T12:31:06.179885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.482245+0000","last_change":"2023-10-14T10:36:03.862676+0000","last_active":"2023-10-14T10:36:05.482245+0000","last_peered":"2023-10-14T10:36:05.482245+0000","last_clean":"2023-10-14T10:36:05.482245+0000","last_became_active":"2023-10-14T10:36:03.862364+0000","last_became_peered":"2023-10-14T10:36:03.862364+0000","last_unstale":"2023-10-14T10:36:05.482245+0000","last_undegraded":"2023-10-14T10:36:05.482245+0000","last_fullsized":"2023-10-14T10:36:05.482245+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T10:42:24.026846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.483021+0000","last_change":"2023-10-14T10:36:03.856448+0000","last_active":"2023-10-14T10:36:05.483021+0000","last_peered":"2023-10-14T10:36:05.483021+0000","last_clean":"2023-10-14T10:36:05.483021+0000","last_became_active":"2023-10-14T10:36:03.856165+0000","last_became_peered":"2023-10-14T10:36:03.856165+0000","last_unstale":"2023-10-14T10:36:05.483021+0000","last_undegraded":"2023-10-14T10:36:05.483021+0000","last_fullsized":"2023-10-14T10:36:05.483021+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:21:06.827004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.845673+0000","last_change":"2023-10-14T10:36:03.862566+0000","last_active":"2023-10-14T10:36:04.845673+0000","last_peered":"2023-10-14T10:36:04.845673+0000","last_clean":"2023-10-14T10:36:04.845673+0000","last_became_active":"2023-10-14T10:36:03.862028+0000","last_became_peered":"2023-10-14T10:36:03.862028+0000","last_unstale":"2023-10-14T10:36:04.845673+0000","last_undegraded":"2023-10-14T10:36:04.845673+0000","last_fullsized":"2023-10-14T10:36:04.845673+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T10:36:37.449003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.483846+0000","last_change":"2023-10-14T10:36:03.865325+0000","last_active":"2023-10-14T10:36:05.483846+0000","last_peered":"2023-10-14T10:36:05.483846+0000","last_clean":"2023-10-14T10:36:05.483846+0000","last_became_active":"2023-10-14T10:36:03.865034+0000","last_became_peered":"2023-10-14T10:36:03.865034+0000","last_unstale":"2023-10-14T10:36:05.483846+0000","last_undegraded":"2023-10-14T10:36:05.483846+0000","last_fullsized":"2023-10-14T10:36:05.483846+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T21:40:43.364841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.483268+0000","last_change":"2023-10-14T10:36:03.857566+0000","last_active":"2023-10-14T10:36:05.483268+0000","last_peered":"2023-10-14T10:36:05.483268+0000","last_clean":"2023-10-14T10:36:05.483268+0000","last_became_active":"2023-10-14T10:36:03.857301+0000","last_became_peered":"2023-10-14T10:36:03.857301+0000","last_unstale":"2023-10-14T10:36:05.483268+0000","last_undegraded":"2023-10-14T10:36:05.483268+0000","last_fullsized":"2023-10-14T10:36:05.483268+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T18:49:18.106622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.841002+0000","last_change":"2023-10-14T10:36:03.863392+0000","last_active":"2023-10-14T10:36:04.841002+0000","last_peered":"2023-10-14T10:36:04.841002+0000","last_clean":"2023-10-14T10:36:04.841002+0000","last_became_active":"2023-10-14T10:36:03.863097+0000","last_became_peered":"2023-10-14T10:36:03.863097+0000","last_unstale":"2023-10-14T10:36:04.841002+0000","last_undegraded":"2023-10-14T10:36:04.841002+0000","last_fullsized":"2023-10-14T10:36:04.841002+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:05:52.086793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.482815+0000","last_change":"2023-10-14T10:36:03.864988+0000","last_active":"2023-10-14T10:36:05.482815+0000","last_peered":"2023-10-14T10:36:05.482815+0000","last_clean":"2023-10-14T10:36:05.482815+0000","last_became_active":"2023-10-14T10:36:03.864699+0000","last_became_peered":"2023-10-14T10:36:03.864699+0000","last_unstale":"2023-10-14T10:36:05.482815+0000","last_undegraded":"2023-10-14T10:36:05.482815+0000","last_fullsized":"2023-10-14T10:36:05.482815+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T18:42:51.749474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.484472+0000","last_change":"2023-10-14T10:36:03.855927+0000","last_active":"2023-10-14T10:36:05.484472+0000","last_peered":"2023-10-14T10:36:05.484472+0000","last_clean":"2023-10-14T10:36:05.484472+0000","last_became_active":"2023-10-14T10:36:03.855643+0000","last_became_peered":"2023-10-14T10:36:03.855643+0000","last_unstale":"2023-10-14T10:36:05.484472+0000","last_undegraded":"2023-10-14T10:36:05.484472+0000","last_fullsized":"2023-10-14T10:36:05.484472+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T20:52:07.681036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.845305+0000","last_change":"2023-10-14T10:36:03.864752+0000","last_active":"2023-10-14T10:36:04.845305+0000","last_peered":"2023-10-14T10:36:04.845305+0000","last_clean":"2023-10-14T10:36:04.845305+0000","last_became_active":"2023-10-14T10:36:03.864480+0000","last_became_peered":"2023-10-14T10:36:03.864480+0000","last_unstale":"2023-10-14T10:36:04.845305+0000","last_undegraded":"2023-10-14T10:36:04.845305+0000","last_fullsized":"2023-10-14T10:36:04.845305+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:05:52.086793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.482676+0000","last_change":"2023-10-14T10:36:03.858250+0000","last_active":"2023-10-14T10:36:05.482676+0000","last_peered":"2023-10-14T10:36:05.482676+0000","last_clean":"2023-10-14T10:36:05.482676+0000","last_became_active":"2023-10-14T10:36:03.857940+0000","last_became_peered":"2023-10-14T10:36:03.857940+0000","last_unstale":"2023-10-14T10:36:05.482676+0000","last_undegraded":"2023-10-14T10:36:05.482676+0000","last_fullsized":"2023-10-14T10:36:05.482676+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T17:28:51.336361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.845474+0000","last_change":"2023-10-14T10:36:03.866904+0000","last_active":"2023-10-14T10:36:04.845474+0000","last_peered":"2023-10-14T10:36:04.845474+0000","last_clean":"2023-10-14T10:36:04.845474+0000","last_became_active":"2023-10-14T10:36:03.866439+0000","last_became_peered":"2023-10-14T10:36:03.866439+0000","last_unstale":"2023-10-14T10:36:04.845474+0000","last_undegraded":"2023-10-14T10:36:04.845474+0000","last_fullsized":"2023-10-14T10:36:04.845474+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T18:49:18.106622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.483685+0000","last_change":"2023-10-14T10:36:03.868286+0000","last_active":"2023-10-14T10:36:05.483685+0000","last_peered":"2023-10-14T10:36:05.483685+0000","last_clean":"2023-10-14T10:36:05.483685+0000","last_became_active":"2023-10-14T10:36:03.868050+0000","last_became_peered":"2023-10-14T10:36:03.868050+0000","last_unstale":"2023-10-14T10:36:05.483685+0000","last_undegraded":"2023-10-14T10:36:05.483685+0000","last_fullsized":"2023-10-14T10:36:05.483685+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T18:50:26.625744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.482746+0000","last_change":"2023-10-14T10:36:03.872106+0000","last_active":"2023-10-14T10:36:05.482746+0000","last_peered":"2023-10-14T10:36:05.482746+0000","last_clean":"2023-10-14T10:36:05.482746+0000","last_became_active":"2023-10-14T10:36:03.871788+0000","last_became_peered":"2023-10-14T10:36:03.871788+0000","last_unstale":"2023-10-14T10:36:05.482746+0000","last_undegraded":"2023-10-14T10:36:05.482746+0000","last_fullsized":"2023-10-14T10:36:05.482746+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T17:28:51.336361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542149,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":823296,"data_stored":669814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94999999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80700000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0129999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91400000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44400000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542150,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":237568,"data_stored":80476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63800000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66800000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.372}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51400000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47199999999999998}]}]},{"osd":1,"up_from":14,"seq":60129542149,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":233472,"data_stored":79024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32900000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27600000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60199999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.629}]}]},{"osd":0,"up_from":14,"seq":60129542148,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27832,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344008,"statfs":{"total":96636764160,"available":96608264192,"internally_reserved":0,"allocated":831488,"data_stored":669870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40600000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60799999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64100000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542149,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":225280,"data_stored":78864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32800000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58699999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]}]},{"osd":3,"up_from":14,"seq":60129542149,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":237568,"data_stored":80476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76500000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]}]},{"osd":4,"up_from":14,"seq":60129542148,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":237568,"data_stored":78990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79300000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.872}]}]},{"osd":5,"up_from":14,"seq":60129542149,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":78954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56399999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66200000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.436}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41499999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-14T10:36:08.013 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T10:36:08.119 INFO:tasks.ceph.mds.b.smithi072.stderr:2023-10-14T10:36:08.116+0000 7efc1d174700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-14T10:36:08.173 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.171+0000 7f1d63b42700 1 Processor -- start 2023-10-14T10:36:08.173 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.171+0000 7f1d63b42700 1 -- start start 2023-10-14T10:36:08.176 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.172+0000 7f1d63b42700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f1d5c15ab10 0x7f1d5c15f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:08.176 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.172+0000 7f1d63b42700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f1d5c0cfe30 0x7f1d5c0cbf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:08.176 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.172+0000 7f1d63b42700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d5c15e7a0 con 0x7f1d5c0b1f50 2023-10-14T10:36:08.176 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.172+0000 7f1d63b42700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d5c15f730 con 0x7f1d5c15ab10 2023-10-14T10:36:08.177 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.172+0000 7f1d63b42700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d5c0b29a0 con 0x7f1d5c0cfe30 2023-10-14T10:36:08.177 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.172+0000 7f1d618de700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f1d5c15ab10 0x7f1d5c15f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:08.177 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.172+0000 7f1d618de700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f1d5c15ab10 0x7f1d5c15f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43880/0 (socket says 172.21.15.67:43880) 2023-10-14T10:36:08.177 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.172+0000 7f1d618de700 1 -- 172.21.15.67:0/1127853758 learned_addr learned my addr 172.21.15.67:0/1127853758 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:08.177 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.172+0000 7f1d620df700 1 --2- 172.21.15.67:0/1127853758 >> v2:172.21.15.72:3300/0 conn(0x7f1d5c0cfe30 0x7f1d5c0cbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:08.178 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.172+0000 7f1d618de700 1 -- 172.21.15.67:0/1127853758 >> v2:172.21.15.72:3300/0 conn(0x7f1d5c0cfe30 msgr2=0x7f1d5c0cbf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:08.178 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.172+0000 7f1d618de700 1 --2- 172.21.15.67:0/1127853758 >> v2:172.21.15.72:3300/0 conn(0x7f1d5c0cfe30 0x7f1d5c0cbf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:08.178 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.172+0000 7f1d618de700 1 -- 172.21.15.67:0/1127853758 >> v1:172.21.15.72:6789/0 conn(0x7f1d5c0b1f50 legacy=0x7f1d5c0cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:08.178 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.172+0000 7f1d618de700 1 -- 172.21.15.67:0/1127853758 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d5c0b2c80 con 0x7f1d5c15ab10 2023-10-14T10:36:08.178 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.173+0000 7f1d618de700 1 --2- 172.21.15.67:0/1127853758 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1d5c15ab10 0x7f1d5c15f1f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c0d03e0 tx=0x7f1d4c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ecbca910f921856 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:08.179 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.173+0000 7f1d608dc700 1 -- 172.21.15.67:0/1127853758 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d4c011830 con 0x7f1d5c15ab10 2023-10-14T10:36:08.179 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.173+0000 7f1d608dc700 1 -- 172.21.15.67:0/1127853758 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d4c011990 con 0x7f1d5c15ab10 2023-10-14T10:36:08.179 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.173+0000 7f1d608dc700 1 -- 172.21.15.67:0/1127853758 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d4c011d20 con 0x7f1d5c15ab10 2023-10-14T10:36:08.179 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.173+0000 7f1d63b42700 1 -- 172.21.15.67:0/1127853758 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1d5c15ab10 msgr2=0x7f1d5c15f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:08.179 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.173+0000 7f1d63b42700 1 --2- 172.21.15.67:0/1127853758 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1d5c15ab10 0x7f1d5c15f1f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c0d03e0 tx=0x7f1d4c009bd0 comp rx=0 tx=0).stop 2023-10-14T10:36:08.179 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.173+0000 7f1d63b42700 1 -- 172.21.15.67:0/1127853758 shutdown_connections 2023-10-14T10:36:08.180 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.173+0000 7f1d63b42700 1 --2- 172.21.15.67:0/1127853758 >> v2:172.21.15.72:3300/0 conn(0x7f1d5c0cfe30 0x7f1d5c0cbf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:08.180 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.173+0000 7f1d63b42700 1 --2- 172.21.15.67:0/1127853758 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1d5c15ab10 0x7f1d5c15f1f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:08.180 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.173+0000 7f1d63b42700 1 -- 172.21.15.67:0/1127853758 >> 172.21.15.67:0/1127853758 conn(0x7f1d5c06dae0 msgr2=0x7f1d5c06def0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:08.180 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.173+0000 7f1d63b42700 1 -- 172.21.15.67:0/1127853758 shutdown_connections 2023-10-14T10:36:08.180 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.173+0000 7f1d63b42700 1 -- 172.21.15.67:0/1127853758 wait complete. 2023-10-14T10:36:08.183 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.175+0000 7f1d63b42700 1 Processor -- start 2023-10-14T10:36:08.184 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.175+0000 7f1d63b42700 1 -- start start 2023-10-14T10:36:08.184 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.175+0000 7f1d63b42700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1d5c0b1f50 0x7f1d5c1f3760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:08.184 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.175+0000 7f1d63b42700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1d5c0cfe30 0x7f1d5c0b3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:08.184 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.175+0000 7f1d63b42700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f1d5c15ab10 0x7f1d5c0ad410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:08.184 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.175+0000 7f1d63b42700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d5c0b3ba0 con 0x7f1d5c0b1f50 2023-10-14T10:36:08.184 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.175+0000 7f1d63b42700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d5c0b4ee0 con 0x7f1d5c0cfe30 2023-10-14T10:36:08.185 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.175+0000 7f1d63b42700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d5c0b3e60 con 0x7f1d5c15ab10 2023-10-14T10:36:08.185 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.175+0000 7f1d618de700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1d5c0b1f50 0x7f1d5c1f3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:08.185 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.175+0000 7f1d618de700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1d5c0b1f50 0x7f1d5c1f3760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43886/0 (socket says 172.21.15.67:43886) 2023-10-14T10:36:08.185 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.175+0000 7f1d618de700 1 -- 172.21.15.67:0/3487501689 learned_addr learned my addr 172.21.15.67:0/3487501689 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:08.185 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.176+0000 7f1d610dd700 1 --2- 172.21.15.67:0/3487501689 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1d5c0cfe30 0x7f1d5c0b3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:08.186 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.176+0000 7f1d620df700 1 --2- 172.21.15.67:0/3487501689 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f1d5c15ab10 0x7f1d5c0ad410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:08.186 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.176+0000 7f1d618de700 1 -- 172.21.15.67:0/3487501689 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f1d5c15ab10 msgr2=0x7f1d5c0ad410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:08.186 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.176+0000 7f1d618de700 1 --2- 172.21.15.67:0/3487501689 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f1d5c15ab10 0x7f1d5c0ad410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:08.186 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.176+0000 7f1d618de700 1 -- 172.21.15.67:0/3487501689 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1d5c0cfe30 msgr2=0x7f1d5c0b3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:08.186 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.176+0000 7f1d618de700 1 --2- 172.21.15.67:0/3487501689 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1d5c0cfe30 0x7f1d5c0b3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:08.187 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.176+0000 7f1d618de700 1 -- 172.21.15.67:0/3487501689 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d4c009550 con 0x7f1d5c0b1f50 2023-10-14T10:36:08.187 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.176+0000 7f1d620df700 1 --2- 172.21.15.67:0/3487501689 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f1d5c15ab10 0x7f1d5c0ad410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-14T10:36:08.187 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.176+0000 7f1d618de700 1 --2- 172.21.15.67:0/3487501689 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1d5c0b1f50 0x7f1d5c1f3760 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c000c00 tx=0x7f1d4c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:08.188 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.176+0000 7f1d527fc700 1 -- 172.21.15.67:0/3487501689 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d4c004480 con 0x7f1d5c0b1f50 2023-10-14T10:36:08.188 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.176+0000 7f1d527fc700 1 -- 172.21.15.67:0/3487501689 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d4c01f070 con 0x7f1d5c0b1f50 2023-10-14T10:36:08.188 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.176+0000 7f1d63b42700 1 -- 172.21.15.67:0/3487501689 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d5c0b4140 con 0x7f1d5c0b1f50 2023-10-14T10:36:08.188 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.176+0000 7f1d527fc700 1 -- 172.21.15.67:0/3487501689 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d4c0038e0 con 0x7f1d5c0b1f50 2023-10-14T10:36:08.188 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.176+0000 7f1d63b42700 1 -- 172.21.15.67:0/3487501689 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d5c0b4700 con 0x7f1d5c0b1f50 2023-10-14T10:36:08.188 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.177+0000 7f1d63b42700 1 -- 172.21.15.67:0/3487501689 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d5c0b4d50 con 0x7f1d5c0b1f50 2023-10-14T10:36:08.189 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.178+0000 7f1d527fc700 1 -- 172.21.15.67:0/3487501689 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f1d4c021030 con 0x7f1d5c0b1f50 2023-10-14T10:36:08.189 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.178+0000 7f1d527fc700 1 --2- 172.21.15.67:0/3487501689 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f1d4006af00 0x7f1d4006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:08.189 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.178+0000 7f1d527fc700 1 -- 172.21.15.67:0/3487501689 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1d4c08a990 con 0x7f1d5c0b1f50 2023-10-14T10:36:08.189 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.178+0000 7f1d610dd700 1 --2- 172.21.15.67:0/3487501689 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f1d4006af00 0x7f1d4006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:08.189 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.179+0000 7f1d610dd700 1 --2- 172.21.15.67:0/3487501689 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f1d4006af00 0x7f1d4006d3c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1d58000fb0 tx=0x7f1d58009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:08.190 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.182+0000 7f1d527fc700 1 -- 172.21.15.67:0/3487501689 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1d4c055ad0 con 0x7f1d5c0b1f50 2023-10-14T10:36:08.380 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.377+0000 7f1d63b42700 1 -- 172.21.15.67:0/3487501689 --> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1d5c15ecd0 con 0x7f1d4006af00 2023-10-14T10:36:08.387 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.386+0000 7f1d527fc700 1 -- 172.21.15.67:0/3487501689 <== mgr.4100 v2:172.21.15.72:6832/703488382 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264716 (secure 0 0 0) 0x7f1d58001cd0 con 0x7f1d4006af00 2023-10-14T10:36:08.390 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-14T10:36:08.392 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.388+0000 7f1d63b42700 1 -- 172.21.15.67:0/3487501689 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f1d4006af00 msgr2=0x7f1d4006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:08.393 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.388+0000 7f1d63b42700 1 --2- 172.21.15.67:0/3487501689 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f1d4006af00 0x7f1d4006d3c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1d58000fb0 tx=0x7f1d58009040 comp rx=0 tx=0).stop 2023-10-14T10:36:08.393 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.388+0000 7f1d63b42700 1 -- 172.21.15.67:0/3487501689 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1d5c0b1f50 msgr2=0x7f1d5c1f3760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:08.393 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.388+0000 7f1d63b42700 1 --2- 172.21.15.67:0/3487501689 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1d5c0b1f50 0x7f1d5c1f3760 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c000c00 tx=0x7f1d4c004750 comp rx=0 tx=0).stop 2023-10-14T10:36:08.393 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.388+0000 7f1d63b42700 1 -- 172.21.15.67:0/3487501689 shutdown_connections 2023-10-14T10:36:08.394 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.388+0000 7f1d63b42700 1 --2- 172.21.15.67:0/3487501689 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f1d4006af00 0x7f1d4006d3c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:08.394 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.388+0000 7f1d63b42700 1 --2- 172.21.15.67:0/3487501689 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f1d5c15ab10 0x7f1d5c0ad410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:08.394 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.388+0000 7f1d63b42700 1 --2- 172.21.15.67:0/3487501689 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1d5c0b1f50 0x7f1d5c1f3760 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:08.394 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.388+0000 7f1d63b42700 1 --2- 172.21.15.67:0/3487501689 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1d5c0cfe30 0x7f1d5c0b3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:08.394 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.388+0000 7f1d63b42700 1 -- 172.21.15.67:0/3487501689 >> 172.21.15.67:0/3487501689 conn(0x7f1d5c06dae0 msgr2=0x7f1d5c0b5240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:08.395 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.388+0000 7f1d63b42700 1 -- 172.21.15.67:0/3487501689 shutdown_connections 2023-10-14T10:36:08.395 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.388+0000 7f1d63b42700 1 -- 172.21.15.67:0/3487501689 wait complete. 2023-10-14T10:36:08.395 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2023-10-14T10:36:08.402 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2023-10-14T10:36:07.755633+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219164,"kb_used_data":2984,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755556,"statfs":{"total":773094113280,"available":772869689344,"internally_reserved":0,"allocated":3055616,"data_stored":1816468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12742,"internal_metadata":221236794},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.930067"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.847283+0000","last_change":"2023-10-14T10:36:03.869543+0000","last_active":"2023-10-14T10:36:04.847283+0000","last_peered":"2023-10-14T10:36:04.847283+0000","last_clean":"2023-10-14T10:36:04.847283+0000","last_became_active":"2023-10-14T10:36:03.868884+0000","last_became_peered":"2023-10-14T10:36:03.868884+0000","last_unstale":"2023-10-14T10:36:04.847283+0000","last_undegraded":"2023-10-14T10:36:04.847283+0000","last_fullsized":"2023-10-14T10:36:04.847283+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:31:25.399614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.841189+0000","last_change":"2023-10-14T10:36:03.867145+0000","last_active":"2023-10-14T10:36:04.841189+0000","last_peered":"2023-10-14T10:36:04.841189+0000","last_clean":"2023-10-14T10:36:04.841189+0000","last_became_active":"2023-10-14T10:36:03.866743+0000","last_became_peered":"2023-10-14T10:36:03.866743+0000","last_unstale":"2023-10-14T10:36:04.841189+0000","last_undegraded":"2023-10-14T10:36:04.841189+0000","last_fullsized":"2023-10-14T10:36:04.841189+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T18:49:18.106622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.483814+0000","last_change":"2023-10-14T10:36:03.861171+0000","last_active":"2023-10-14T10:36:05.483814+0000","last_peered":"2023-10-14T10:36:05.483814+0000","last_clean":"2023-10-14T10:36:05.483814+0000","last_became_active":"2023-10-14T10:36:03.860860+0000","last_became_peered":"2023-10-14T10:36:03.860860+0000","last_unstale":"2023-10-14T10:36:05.483814+0000","last_undegraded":"2023-10-14T10:36:05.483814+0000","last_fullsized":"2023-10-14T10:36:05.483814+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T17:28:51.336361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.845699+0000","last_change":"2023-10-14T10:36:03.861631+0000","last_active":"2023-10-14T10:36:04.845699+0000","last_peered":"2023-10-14T10:36:04.845699+0000","last_clean":"2023-10-14T10:36:04.845699+0000","last_became_active":"2023-10-14T10:36:03.861291+0000","last_became_peered":"2023-10-14T10:36:03.861291+0000","last_unstale":"2023-10-14T10:36:04.845699+0000","last_undegraded":"2023-10-14T10:36:04.845699+0000","last_fullsized":"2023-10-14T10:36:04.845699+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T17:52:45.149486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.846931+0000","last_change":"2023-10-14T10:36:03.870469+0000","last_active":"2023-10-14T10:36:04.846931+0000","last_peered":"2023-10-14T10:36:04.846931+0000","last_clean":"2023-10-14T10:36:04.846931+0000","last_became_active":"2023-10-14T10:36:03.865743+0000","last_became_peered":"2023-10-14T10:36:03.865743+0000","last_unstale":"2023-10-14T10:36:04.846931+0000","last_undegraded":"2023-10-14T10:36:04.846931+0000","last_fullsized":"2023-10-14T10:36:04.846931+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:33:09.576282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.845553+0000","last_change":"2023-10-14T10:36:03.862386+0000","last_active":"2023-10-14T10:36:04.845553+0000","last_peered":"2023-10-14T10:36:04.845553+0000","last_clean":"2023-10-14T10:36:04.845553+0000","last_became_active":"2023-10-14T10:36:03.861705+0000","last_became_peered":"2023-10-14T10:36:03.861705+0000","last_unstale":"2023-10-14T10:36:04.845553+0000","last_undegraded":"2023-10-14T10:36:04.845553+0000","last_fullsized":"2023-10-14T10:36:04.845553+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:47:42.089047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.846829+0000","last_change":"2023-10-14T10:36:03.865502+0000","last_active":"2023-10-14T10:36:04.846829+0000","last_peered":"2023-10-14T10:36:04.846829+0000","last_clean":"2023-10-14T10:36:04.846829+0000","last_became_active":"2023-10-14T10:36:03.865238+0000","last_became_peered":"2023-10-14T10:36:03.865238+0000","last_unstale":"2023-10-14T10:36:04.846829+0000","last_undegraded":"2023-10-14T10:36:04.846829+0000","last_fullsized":"2023-10-14T10:36:04.846829+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T20:52:07.681036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.482589+0000","last_change":"2023-10-14T10:36:03.859238+0000","last_active":"2023-10-14T10:36:05.482589+0000","last_peered":"2023-10-14T10:36:05.482589+0000","last_clean":"2023-10-14T10:36:05.482589+0000","last_became_active":"2023-10-14T10:36:03.858981+0000","last_became_peered":"2023-10-14T10:36:03.858981+0000","last_unstale":"2023-10-14T10:36:05.482589+0000","last_undegraded":"2023-10-14T10:36:05.482589+0000","last_fullsized":"2023-10-14T10:36:05.482589+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T12:18:09.680616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.847259+0000","last_change":"2023-10-14T10:36:03.868883+0000","last_active":"2023-10-14T10:36:04.847259+0000","last_peered":"2023-10-14T10:36:04.847259+0000","last_clean":"2023-10-14T10:36:04.847259+0000","last_became_active":"2023-10-14T10:36:03.868019+0000","last_became_peered":"2023-10-14T10:36:03.868019+0000","last_unstale":"2023-10-14T10:36:04.847259+0000","last_undegraded":"2023-10-14T10:36:04.847259+0000","last_fullsized":"2023-10-14T10:36:04.847259+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T17:08:17.694068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.482896+0000","last_change":"2023-10-14T10:36:03.861523+0000","last_active":"2023-10-14T10:36:05.482896+0000","last_peered":"2023-10-14T10:36:05.482896+0000","last_clean":"2023-10-14T10:36:05.482896+0000","last_became_active":"2023-10-14T10:36:03.860708+0000","last_became_peered":"2023-10-14T10:36:03.860708+0000","last_unstale":"2023-10-14T10:36:05.482896+0000","last_undegraded":"2023-10-14T10:36:05.482896+0000","last_fullsized":"2023-10-14T10:36:05.482896+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:07:32.070142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.482672+0000","last_change":"2023-10-14T10:36:03.861278+0000","last_active":"2023-10-14T10:36:05.482672+0000","last_peered":"2023-10-14T10:36:05.482672+0000","last_clean":"2023-10-14T10:36:05.482672+0000","last_became_active":"2023-10-14T10:36:03.860994+0000","last_became_peered":"2023-10-14T10:36:03.860994+0000","last_unstale":"2023-10-14T10:36:05.482672+0000","last_undegraded":"2023-10-14T10:36:05.482672+0000","last_fullsized":"2023-10-14T10:36:05.482672+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T12:18:09.680616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.482785+0000","last_change":"2023-10-14T10:36:03.871181+0000","last_active":"2023-10-14T10:36:05.482785+0000","last_peered":"2023-10-14T10:36:05.482785+0000","last_clean":"2023-10-14T10:36:05.482785+0000","last_became_active":"2023-10-14T10:36:03.870875+0000","last_became_peered":"2023-10-14T10:36:03.870875+0000","last_unstale":"2023-10-14T10:36:05.482785+0000","last_undegraded":"2023-10-14T10:36:05.482785+0000","last_fullsized":"2023-10-14T10:36:05.482785+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:50:35.924528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.482529+0000","last_change":"2023-10-14T10:36:03.857251+0000","last_active":"2023-10-14T10:36:05.482529+0000","last_peered":"2023-10-14T10:36:05.482529+0000","last_clean":"2023-10-14T10:36:05.482529+0000","last_became_active":"2023-10-14T10:36:03.856908+0000","last_became_peered":"2023-10-14T10:36:03.856908+0000","last_unstale":"2023-10-14T10:36:05.482529+0000","last_undegraded":"2023-10-14T10:36:05.482529+0000","last_fullsized":"2023-10-14T10:36:05.482529+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T18:50:26.625744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.484165+0000","last_change":"2023-10-14T10:36:03.858760+0000","last_active":"2023-10-14T10:36:05.484165+0000","last_peered":"2023-10-14T10:36:05.484165+0000","last_clean":"2023-10-14T10:36:05.484165+0000","last_became_active":"2023-10-14T10:36:03.854779+0000","last_became_peered":"2023-10-14T10:36:03.854779+0000","last_unstale":"2023-10-14T10:36:05.484165+0000","last_undegraded":"2023-10-14T10:36:05.484165+0000","last_fullsized":"2023-10-14T10:36:05.484165+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T18:49:18.106622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.840868+0000","last_change":"2023-10-14T10:36:03.861476+0000","last_active":"2023-10-14T10:36:04.840868+0000","last_peered":"2023-10-14T10:36:04.840868+0000","last_clean":"2023-10-14T10:36:04.840868+0000","last_became_active":"2023-10-14T10:36:03.859641+0000","last_became_peered":"2023-10-14T10:36:03.859641+0000","last_unstale":"2023-10-14T10:36:04.840868+0000","last_undegraded":"2023-10-14T10:36:04.840868+0000","last_fullsized":"2023-10-14T10:36:04.840868+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T21:40:43.364841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.845130+0000","last_change":"2023-10-14T10:36:03.858710+0000","last_active":"2023-10-14T10:36:04.845130+0000","last_peered":"2023-10-14T10:36:04.845130+0000","last_clean":"2023-10-14T10:36:04.845130+0000","last_became_active":"2023-10-14T10:36:03.858456+0000","last_became_peered":"2023-10-14T10:36:03.858456+0000","last_unstale":"2023-10-14T10:36:04.845130+0000","last_undegraded":"2023-10-14T10:36:04.845130+0000","last_fullsized":"2023-10-14T10:36:04.845130+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:07:32.070142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.845836+0000","last_change":"2023-10-14T10:36:03.867067+0000","last_active":"2023-10-14T10:36:04.845836+0000","last_peered":"2023-10-14T10:36:04.845836+0000","last_clean":"2023-10-14T10:36:04.845836+0000","last_became_active":"2023-10-14T10:36:03.866802+0000","last_became_peered":"2023-10-14T10:36:03.866802+0000","last_unstale":"2023-10-14T10:36:04.845836+0000","last_undegraded":"2023-10-14T10:36:04.845836+0000","last_fullsized":"2023-10-14T10:36:04.845836+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:09:22.818455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.482788+0000","last_change":"2023-10-14T10:36:03.869409+0000","last_active":"2023-10-14T10:36:05.482788+0000","last_peered":"2023-10-14T10:36:05.482788+0000","last_clean":"2023-10-14T10:36:05.482788+0000","last_became_active":"2023-10-14T10:36:03.868961+0000","last_became_peered":"2023-10-14T10:36:03.868961+0000","last_unstale":"2023-10-14T10:36:05.482788+0000","last_undegraded":"2023-10-14T10:36:05.482788+0000","last_fullsized":"2023-10-14T10:36:05.482788+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:10:10.517040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.847011+0000","last_change":"2023-10-14T10:36:03.864794+0000","last_active":"2023-10-14T10:36:04.847011+0000","last_peered":"2023-10-14T10:36:04.847011+0000","last_clean":"2023-10-14T10:36:04.847011+0000","last_became_active":"2023-10-14T10:36:03.864195+0000","last_became_peered":"2023-10-14T10:36:03.864195+0000","last_unstale":"2023-10-14T10:36:04.847011+0000","last_undegraded":"2023-10-14T10:36:04.847011+0000","last_fullsized":"2023-10-14T10:36:04.847011+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T17:52:45.149486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.483618+0000","last_change":"2023-10-14T10:36:02.855169+0000","last_active":"2023-10-14T10:36:05.483618+0000","last_peered":"2023-10-14T10:36:05.483618+0000","last_clean":"2023-10-14T10:36:05.483618+0000","last_became_active":"2023-10-14T10:36:02.854817+0000","last_became_peered":"2023-10-14T10:36:02.854817+0000","last_unstale":"2023-10-14T10:36:05.483618+0000","last_undegraded":"2023-10-14T10:36:05.483618+0000","last_fullsized":"2023-10-14T10:36:05.483618+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T14:12:52.738066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.484369+0000","last_change":"2023-10-14T10:36:03.865118+0000","last_active":"2023-10-14T10:36:05.484369+0000","last_peered":"2023-10-14T10:36:05.484369+0000","last_clean":"2023-10-14T10:36:05.484369+0000","last_became_active":"2023-10-14T10:36:03.864579+0000","last_became_peered":"2023-10-14T10:36:03.864579+0000","last_unstale":"2023-10-14T10:36:05.484369+0000","last_undegraded":"2023-10-14T10:36:05.484369+0000","last_fullsized":"2023-10-14T10:36:05.484369+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:27:25.021169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.484313+0000","last_change":"2023-10-14T10:36:02.851587+0000","last_active":"2023-10-14T10:36:05.484313+0000","last_peered":"2023-10-14T10:36:05.484313+0000","last_clean":"2023-10-14T10:36:05.484313+0000","last_became_active":"2023-10-14T10:36:02.851276+0000","last_became_peered":"2023-10-14T10:36:02.851276+0000","last_unstale":"2023-10-14T10:36:05.484313+0000","last_undegraded":"2023-10-14T10:36:05.484313+0000","last_fullsized":"2023-10-14T10:36:05.484313+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T14:04:44.127519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.483564+0000","last_change":"2023-10-14T10:36:03.861320+0000","last_active":"2023-10-14T10:36:05.483564+0000","last_peered":"2023-10-14T10:36:05.483564+0000","last_clean":"2023-10-14T10:36:05.483564+0000","last_became_active":"2023-10-14T10:36:03.860684+0000","last_became_peered":"2023-10-14T10:36:03.860684+0000","last_unstale":"2023-10-14T10:36:05.483564+0000","last_undegraded":"2023-10-14T10:36:05.483564+0000","last_fullsized":"2023-10-14T10:36:05.483564+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:22:58.149187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.847230+0000","last_change":"2023-10-14T10:36:02.854569+0000","last_active":"2023-10-14T10:36:04.847230+0000","last_peered":"2023-10-14T10:36:04.847230+0000","last_clean":"2023-10-14T10:36:04.847230+0000","last_became_active":"2023-10-14T10:36:02.853518+0000","last_became_peered":"2023-10-14T10:36:02.853518+0000","last_unstale":"2023-10-14T10:36:04.847230+0000","last_undegraded":"2023-10-14T10:36:04.847230+0000","last_fullsized":"2023-10-14T10:36:04.847230+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:27:24.015552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.845887+0000","last_change":"2023-10-14T10:36:03.872077+0000","last_active":"2023-10-14T10:36:04.845887+0000","last_peered":"2023-10-14T10:36:04.845887+0000","last_clean":"2023-10-14T10:36:04.845887+0000","last_became_active":"2023-10-14T10:36:03.871740+0000","last_became_peered":"2023-10-14T10:36:03.871740+0000","last_unstale":"2023-10-14T10:36:04.845887+0000","last_undegraded":"2023-10-14T10:36:04.845887+0000","last_fullsized":"2023-10-14T10:36:04.845887+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T12:31:06.179885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.846883+0000","last_change":"2023-10-14T10:36:02.853780+0000","last_active":"2023-10-14T10:36:04.846883+0000","last_peered":"2023-10-14T10:36:04.846883+0000","last_clean":"2023-10-14T10:36:04.846883+0000","last_became_active":"2023-10-14T10:36:02.853450+0000","last_became_peered":"2023-10-14T10:36:02.853450+0000","last_unstale":"2023-10-14T10:36:04.846883+0000","last_undegraded":"2023-10-14T10:36:04.846883+0000","last_fullsized":"2023-10-14T10:36:04.846883+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T10:42:23.021229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.845567+0000","last_change":"2023-10-14T10:36:03.858379+0000","last_active":"2023-10-14T10:36:04.845567+0000","last_peered":"2023-10-14T10:36:04.845567+0000","last_clean":"2023-10-14T10:36:04.845567+0000","last_became_active":"2023-10-14T10:36:03.857904+0000","last_became_peered":"2023-10-14T10:36:03.857904+0000","last_unstale":"2023-10-14T10:36:04.845567+0000","last_undegraded":"2023-10-14T10:36:04.845567+0000","last_fullsized":"2023-10-14T10:36:04.845567+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:33:09.576282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.482737+0000","last_change":"2023-10-14T10:36:02.854396+0000","last_active":"2023-10-14T10:36:05.482737+0000","last_peered":"2023-10-14T10:36:05.482737+0000","last_clean":"2023-10-14T10:36:05.482737+0000","last_became_active":"2023-10-14T10:36:02.854085+0000","last_became_peered":"2023-10-14T10:36:02.854085+0000","last_unstale":"2023-10-14T10:36:05.482737+0000","last_undegraded":"2023-10-14T10:36:05.482737+0000","last_fullsized":"2023-10-14T10:36:05.482737+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T14:12:52.738066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.483060+0000","last_change":"2023-10-14T10:36:03.855010+0000","last_active":"2023-10-14T10:36:05.483060+0000","last_peered":"2023-10-14T10:36:05.483060+0000","last_clean":"2023-10-14T10:36:05.483060+0000","last_became_active":"2023-10-14T10:36:03.854714+0000","last_became_peered":"2023-10-14T10:36:03.854714+0000","last_unstale":"2023-10-14T10:36:05.483060+0000","last_undegraded":"2023-10-14T10:36:05.483060+0000","last_fullsized":"2023-10-14T10:36:05.483060+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:29:51.596547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-14T10:36:03.865241+0000","last_change":"2023-10-14T10:36:02.847573+0000","last_active":"2023-10-14T10:36:03.865241+0000","last_peered":"2023-10-14T10:36:03.865241+0000","last_clean":"2023-10-14T10:36:03.865241+0000","last_became_active":"2023-10-14T10:36:02.847061+0000","last_became_peered":"2023-10-14T10:36:02.847061+0000","last_unstale":"2023-10-14T10:36:03.865241+0000","last_undegraded":"2023-10-14T10:36:03.865241+0000","last_fullsized":"2023-10-14T10:36:03.865241+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:50:34.918912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.484012+0000","last_change":"2023-10-14T10:36:03.866134+0000","last_active":"2023-10-14T10:36:05.484012+0000","last_peered":"2023-10-14T10:36:05.484012+0000","last_clean":"2023-10-14T10:36:05.484012+0000","last_became_active":"2023-10-14T10:36:03.865614+0000","last_became_peered":"2023-10-14T10:36:03.865614+0000","last_unstale":"2023-10-14T10:36:05.484012+0000","last_undegraded":"2023-10-14T10:36:05.484012+0000","last_fullsized":"2023-10-14T10:36:05.484012+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T18:42:51.749474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.482644+0000","last_change":"2023-10-14T10:36:02.846515+0000","last_active":"2023-10-14T10:36:05.482644+0000","last_peered":"2023-10-14T10:36:05.482644+0000","last_clean":"2023-10-14T10:36:05.482644+0000","last_became_active":"2023-10-14T10:36:02.845841+0000","last_became_peered":"2023-10-14T10:36:02.845841+0000","last_unstale":"2023-10-14T10:36:05.482644+0000","last_undegraded":"2023-10-14T10:36:05.482644+0000","last_fullsized":"2023-10-14T10:36:05.482644+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:22:57.143570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.483031+0000","last_change":"2023-10-14T10:36:03.865280+0000","last_active":"2023-10-14T10:36:05.483031+0000","last_peered":"2023-10-14T10:36:05.483031+0000","last_clean":"2023-10-14T10:36:05.483031+0000","last_became_active":"2023-10-14T10:36:03.864926+0000","last_became_peered":"2023-10-14T10:36:03.864926+0000","last_unstale":"2023-10-14T10:36:05.483031+0000","last_undegraded":"2023-10-14T10:36:05.483031+0000","last_fullsized":"2023-10-14T10:36:05.483031+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:05:52.086793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.845177+0000","last_change":"2023-10-14T10:36:02.843680+0000","last_active":"2023-10-14T10:36:04.845177+0000","last_peered":"2023-10-14T10:36:04.845177+0000","last_clean":"2023-10-14T10:36:04.845177+0000","last_became_active":"2023-10-14T10:36:02.843364+0000","last_became_peered":"2023-10-14T10:36:02.843364+0000","last_unstale":"2023-10-14T10:36:04.845177+0000","last_undegraded":"2023-10-14T10:36:04.845177+0000","last_fullsized":"2023-10-14T10:36:04.845177+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T10:59:16.502697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.841021+0000","last_change":"2023-10-14T10:36:03.862822+0000","last_active":"2023-10-14T10:36:04.841021+0000","last_peered":"2023-10-14T10:36:04.841021+0000","last_clean":"2023-10-14T10:36:04.841021+0000","last_became_active":"2023-10-14T10:36:03.862547+0000","last_became_peered":"2023-10-14T10:36:03.862547+0000","last_unstale":"2023-10-14T10:36:04.841021+0000","last_undegraded":"2023-10-14T10:36:04.841021+0000","last_fullsized":"2023-10-14T10:36:04.841021+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T18:42:51.749474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-14T10:36:03.851585+0000","last_change":"2023-10-14T10:36:02.846160+0000","last_active":"2023-10-14T10:36:03.851585+0000","last_peered":"2023-10-14T10:36:03.851585+0000","last_clean":"2023-10-14T10:36:03.851585+0000","last_became_active":"2023-10-14T10:36:02.845784+0000","last_became_peered":"2023-10-14T10:36:02.845784+0000","last_unstale":"2023-10-14T10:36:03.851585+0000","last_undegraded":"2023-10-14T10:36:03.851585+0000","last_fullsized":"2023-10-14T10:36:03.851585+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T12:29:01.996039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.845511+0000","last_change":"2023-10-14T10:36:03.864605+0000","last_active":"2023-10-14T10:36:04.845511+0000","last_peered":"2023-10-14T10:36:04.845511+0000","last_clean":"2023-10-14T10:36:04.845511+0000","last_became_active":"2023-10-14T10:36:03.864160+0000","last_became_peered":"2023-10-14T10:36:03.864160+0000","last_unstale":"2023-10-14T10:36:04.845511+0000","last_undegraded":"2023-10-14T10:36:04.845511+0000","last_fullsized":"2023-10-14T10:36:04.845511+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T20:52:07.681036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.845294+0000","last_change":"2023-10-14T10:36:02.845777+0000","last_active":"2023-10-14T10:36:04.845294+0000","last_peered":"2023-10-14T10:36:04.845294+0000","last_clean":"2023-10-14T10:36:04.845294+0000","last_became_active":"2023-10-14T10:36:02.845482+0000","last_became_peered":"2023-10-14T10:36:02.845482+0000","last_unstale":"2023-10-14T10:36:04.845294+0000","last_undegraded":"2023-10-14T10:36:04.845294+0000","last_fullsized":"2023-10-14T10:36:04.845294+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T14:04:44.127519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.483717+0000","last_change":"2023-10-14T10:36:03.861072+0000","last_active":"2023-10-14T10:36:05.483717+0000","last_peered":"2023-10-14T10:36:05.483717+0000","last_clean":"2023-10-14T10:36:05.483717+0000","last_became_active":"2023-10-14T10:36:03.860335+0000","last_became_peered":"2023-10-14T10:36:03.860335+0000","last_unstale":"2023-10-14T10:36:05.483717+0000","last_undegraded":"2023-10-14T10:36:05.483717+0000","last_fullsized":"2023-10-14T10:36:05.483717+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T12:18:09.680616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.841103+0000","last_change":"2023-10-14T10:36:02.845951+0000","last_active":"2023-10-14T10:36:04.841103+0000","last_peered":"2023-10-14T10:36:04.841103+0000","last_clean":"2023-10-14T10:36:04.841103+0000","last_became_active":"2023-10-14T10:36:02.845667+0000","last_became_peered":"2023-10-14T10:36:02.845667+0000","last_unstale":"2023-10-14T10:36:04.841103+0000","last_undegraded":"2023-10-14T10:36:04.841103+0000","last_fullsized":"2023-10-14T10:36:04.841103+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:00:17.008189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.482746+0000","last_change":"2023-10-14T10:36:03.871035+0000","last_active":"2023-10-14T10:36:05.482746+0000","last_peered":"2023-10-14T10:36:05.482746+0000","last_clean":"2023-10-14T10:36:05.482746+0000","last_became_active":"2023-10-14T10:36:03.870542+0000","last_became_peered":"2023-10-14T10:36:03.870542+0000","last_unstale":"2023-10-14T10:36:05.482746+0000","last_undegraded":"2023-10-14T10:36:05.482746+0000","last_fullsized":"2023-10-14T10:36:05.482746+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T14:05:05.847145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-14T10:36:03.874236+0000","last_change":"2023-10-14T10:36:02.847268+0000","last_active":"2023-10-14T10:36:03.874236+0000","last_peered":"2023-10-14T10:36:03.874236+0000","last_clean":"2023-10-14T10:36:03.874236+0000","last_became_active":"2023-10-14T10:36:02.846840+0000","last_became_peered":"2023-10-14T10:36:02.846840+0000","last_unstale":"2023-10-14T10:36:03.874236+0000","last_undegraded":"2023-10-14T10:36:03.874236+0000","last_fullsized":"2023-10-14T10:36:03.874236+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:22:57.143570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.847057+0000","last_change":"2023-10-14T10:36:03.864955+0000","last_active":"2023-10-14T10:36:04.847057+0000","last_peered":"2023-10-14T10:36:04.847057+0000","last_clean":"2023-10-14T10:36:04.847057+0000","last_became_active":"2023-10-14T10:36:03.864505+0000","last_became_peered":"2023-10-14T10:36:03.864505+0000","last_unstale":"2023-10-14T10:36:04.847057+0000","last_undegraded":"2023-10-14T10:36:04.847057+0000","last_fullsized":"2023-10-14T10:36:04.847057+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:00:36.181878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.483879+0000","last_change":"2023-10-14T10:36:02.856271+0000","last_active":"2023-10-14T10:36:05.483879+0000","last_peered":"2023-10-14T10:36:05.483879+0000","last_clean":"2023-10-14T10:36:05.483879+0000","last_became_active":"2023-10-14T10:36:02.856039+0000","last_became_peered":"2023-10-14T10:36:02.856039+0000","last_unstale":"2023-10-14T10:36:05.483879+0000","last_undegraded":"2023-10-14T10:36:05.483879+0000","last_fullsized":"2023-10-14T10:36:05.483879+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:29:50.590931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.484124+0000","last_change":"2023-10-14T10:36:03.868295+0000","last_active":"2023-10-14T10:36:05.484124+0000","last_peered":"2023-10-14T10:36:05.484124+0000","last_clean":"2023-10-14T10:36:05.484124+0000","last_became_active":"2023-10-14T10:36:03.867969+0000","last_became_peered":"2023-10-14T10:36:03.867969+0000","last_unstale":"2023-10-14T10:36:05.484124+0000","last_undegraded":"2023-10-14T10:36:05.484124+0000","last_fullsized":"2023-10-14T10:36:05.484124+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:05:52.086793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.845748+0000","last_change":"2023-10-14T10:36:02.845900+0000","last_active":"2023-10-14T10:36:04.845748+0000","last_peered":"2023-10-14T10:36:04.845748+0000","last_clean":"2023-10-14T10:36:04.845748+0000","last_became_active":"2023-10-14T10:36:02.845596+0000","last_became_peered":"2023-10-14T10:36:02.845596+0000","last_unstale":"2023-10-14T10:36:04.845748+0000","last_undegraded":"2023-10-14T10:36:04.845748+0000","last_fullsized":"2023-10-14T10:36:04.845748+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T21:40:42.359224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.846941+0000","last_change":"2023-10-14T10:36:03.870846+0000","last_active":"2023-10-14T10:36:04.846941+0000","last_peered":"2023-10-14T10:36:04.846941+0000","last_clean":"2023-10-14T10:36:04.846941+0000","last_became_active":"2023-10-14T10:36:03.869379+0000","last_became_peered":"2023-10-14T10:36:03.869379+0000","last_unstale":"2023-10-14T10:36:04.846941+0000","last_undegraded":"2023-10-14T10:36:04.846941+0000","last_fullsized":"2023-10-14T10:36:04.846941+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T10:36:37.449003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-14T10:36:03.851688+0000","last_change":"2023-10-14T10:36:02.847080+0000","last_active":"2023-10-14T10:36:03.851688+0000","last_peered":"2023-10-14T10:36:03.851688+0000","last_clean":"2023-10-14T10:36:03.851688+0000","last_became_active":"2023-10-14T10:36:02.846784+0000","last_became_peered":"2023-10-14T10:36:02.846784+0000","last_unstale":"2023-10-14T10:36:03.851688+0000","last_undegraded":"2023-10-14T10:36:03.851688+0000","last_fullsized":"2023-10-14T10:36:03.851688+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:29:50.590931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.482758+0000","last_change":"2023-10-14T10:36:03.862316+0000","last_active":"2023-10-14T10:36:05.482758+0000","last_peered":"2023-10-14T10:36:05.482758+0000","last_clean":"2023-10-14T10:36:05.482758+0000","last_became_active":"2023-10-14T10:36:03.862022+0000","last_became_peered":"2023-10-14T10:36:03.862022+0000","last_unstale":"2023-10-14T10:36:05.482758+0000","last_undegraded":"2023-10-14T10:36:05.482758+0000","last_fullsized":"2023-10-14T10:36:05.482758+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T21:40:43.364841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-14T10:36:03.867586+0000","last_change":"2023-10-14T10:36:02.846811+0000","last_active":"2023-10-14T10:36:03.867586+0000","last_peered":"2023-10-14T10:36:03.867586+0000","last_clean":"2023-10-14T10:36:03.867586+0000","last_became_active":"2023-10-14T10:36:02.846487+0000","last_became_peered":"2023-10-14T10:36:02.846487+0000","last_unstale":"2023-10-14T10:36:03.867586+0000","last_undegraded":"2023-10-14T10:36:03.867586+0000","last_fullsized":"2023-10-14T10:36:03.867586+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T18:47:26.337837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.846699+0000","last_change":"2023-10-14T10:36:03.872480+0000","last_active":"2023-10-14T10:36:04.846699+0000","last_peered":"2023-10-14T10:36:04.846699+0000","last_clean":"2023-10-14T10:36:04.846699+0000","last_became_active":"2023-10-14T10:36:03.872231+0000","last_became_peered":"2023-10-14T10:36:03.872231+0000","last_unstale":"2023-10-14T10:36:04.846699+0000","last_undegraded":"2023-10-14T10:36:04.846699+0000","last_fullsized":"2023-10-14T10:36:04.846699+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:47:42.089047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.482611+0000","last_change":"2023-10-14T10:36:02.851223+0000","last_active":"2023-10-14T10:36:05.482611+0000","last_peered":"2023-10-14T10:36:05.482611+0000","last_clean":"2023-10-14T10:36:05.482611+0000","last_became_active":"2023-10-14T10:36:02.850955+0000","last_became_peered":"2023-10-14T10:36:02.850955+0000","last_unstale":"2023-10-14T10:36:05.482611+0000","last_undegraded":"2023-10-14T10:36:05.482611+0000","last_fullsized":"2023-10-14T10:36:05.482611+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:08:47.187941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.847107+0000","last_change":"2023-10-14T10:36:03.871637+0000","last_active":"2023-10-14T10:36:04.847107+0000","last_peered":"2023-10-14T10:36:04.847107+0000","last_clean":"2023-10-14T10:36:04.847107+0000","last_became_active":"2023-10-14T10:36:03.871362+0000","last_became_peered":"2023-10-14T10:36:03.871362+0000","last_unstale":"2023-10-14T10:36:04.847107+0000","last_undegraded":"2023-10-14T10:36:04.847107+0000","last_fullsized":"2023-10-14T10:36:04.847107+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:09:22.818455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.841156+0000","last_change":"2023-10-14T10:36:02.851439+0000","last_active":"2023-10-14T10:36:04.841156+0000","last_peered":"2023-10-14T10:36:04.841156+0000","last_clean":"2023-10-14T10:36:04.841156+0000","last_became_active":"2023-10-14T10:36:02.851121+0000","last_became_peered":"2023-10-14T10:36:02.851121+0000","last_unstale":"2023-10-14T10:36:04.841156+0000","last_undegraded":"2023-10-14T10:36:04.841156+0000","last_fullsized":"2023-10-14T10:36:04.841156+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T10:42:23.021229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.845753+0000","last_change":"2023-10-14T10:36:03.863345+0000","last_active":"2023-10-14T10:36:04.845753+0000","last_peered":"2023-10-14T10:36:04.845753+0000","last_clean":"2023-10-14T10:36:04.845753+0000","last_became_active":"2023-10-14T10:36:03.861227+0000","last_became_peered":"2023-10-14T10:36:03.861227+0000","last_unstale":"2023-10-14T10:36:04.845753+0000","last_undegraded":"2023-10-14T10:36:04.845753+0000","last_fullsized":"2023-10-14T10:36:04.845753+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:00:36.181878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.845649+0000","last_change":"2023-10-14T10:36:02.846354+0000","last_active":"2023-10-14T10:36:04.845649+0000","last_peered":"2023-10-14T10:36:04.845649+0000","last_clean":"2023-10-14T10:36:04.845649+0000","last_became_active":"2023-10-14T10:36:02.845950+0000","last_became_peered":"2023-10-14T10:36:02.845950+0000","last_unstale":"2023-10-14T10:36:04.845649+0000","last_undegraded":"2023-10-14T10:36:04.845649+0000","last_fullsized":"2023-10-14T10:36:04.845649+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T17:28:50.330745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.482785+0000","last_change":"2023-10-14T10:36:03.860805+0000","last_active":"2023-10-14T10:36:05.482785+0000","last_peered":"2023-10-14T10:36:05.482785+0000","last_clean":"2023-10-14T10:36:05.482785+0000","last_became_active":"2023-10-14T10:36:03.860218+0000","last_became_peered":"2023-10-14T10:36:03.860218+0000","last_unstale":"2023-10-14T10:36:05.482785+0000","last_undegraded":"2023-10-14T10:36:05.482785+0000","last_fullsized":"2023-10-14T10:36:05.482785+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T12:29:03.001656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.840832+0000","last_change":"2023-10-14T10:36:03.857512+0000","last_active":"2023-10-14T10:36:04.840832+0000","last_peered":"2023-10-14T10:36:04.840832+0000","last_clean":"2023-10-14T10:36:04.840832+0000","last_became_active":"2023-10-14T10:36:03.855687+0000","last_became_peered":"2023-10-14T10:36:03.855687+0000","last_unstale":"2023-10-14T10:36:04.840832+0000","last_undegraded":"2023-10-14T10:36:04.840832+0000","last_fullsized":"2023-10-14T10:36:04.840832+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T17:28:51.336361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-14T10:36:03.843979+0000","last_change":"2023-10-14T10:36:02.843620+0000","last_active":"2023-10-14T10:36:03.843979+0000","last_peered":"2023-10-14T10:36:03.843979+0000","last_clean":"2023-10-14T10:36:03.843979+0000","last_became_active":"2023-10-14T10:36:02.843324+0000","last_became_peered":"2023-10-14T10:36:02.843324+0000","last_unstale":"2023-10-14T10:36:03.843979+0000","last_undegraded":"2023-10-14T10:36:03.843979+0000","last_fullsized":"2023-10-14T10:36:03.843979+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T14:12:49.629312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.482850+0000","last_change":"2023-10-14T10:36:03.858691+0000","last_active":"2023-10-14T10:36:05.482850+0000","last_peered":"2023-10-14T10:36:05.482850+0000","last_clean":"2023-10-14T10:36:05.482850+0000","last_became_active":"2023-10-14T10:36:03.858420+0000","last_became_peered":"2023-10-14T10:36:03.858420+0000","last_unstale":"2023-10-14T10:36:05.482850+0000","last_undegraded":"2023-10-14T10:36:05.482850+0000","last_fullsized":"2023-10-14T10:36:05.482850+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T18:42:51.749474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.845260+0000","last_change":"2023-10-14T10:36:02.840533+0000","last_active":"2023-10-14T10:36:04.845260+0000","last_peered":"2023-10-14T10:36:04.845260+0000","last_clean":"2023-10-14T10:36:04.845260+0000","last_became_active":"2023-10-14T10:36:02.839919+0000","last_became_peered":"2023-10-14T10:36:02.839919+0000","last_unstale":"2023-10-14T10:36:04.845260+0000","last_undegraded":"2023-10-14T10:36:04.845260+0000","last_fullsized":"2023-10-14T10:36:04.845260+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:29:50.590931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":165,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-14T10:36:03.847417+0000","last_change":"2023-10-14T10:35:59.770163+0000","last_active":"2023-10-14T10:36:03.847417+0000","last_peered":"2023-10-14T10:36:03.847417+0000","last_clean":"2023-10-14T10:36:03.847417+0000","last_became_active":"2023-10-14T10:35:59.769199+0000","last_became_peered":"2023-10-14T10:35:59.769199+0000","last_unstale":"2023-10-14T10:36:03.847417+0000","last_undegraded":"2023-10-14T10:36:03.847417+0000","last_fullsized":"2023-10-14T10:36:03.847417+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:35:58.675572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:35:58.675572+0000","last_clean_scrub_stamp":"2023-10-14T10:35:58.675572+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:03:25.281187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.484102+0000","last_change":"2023-10-14T10:36:03.872244+0000","last_active":"2023-10-14T10:36:05.484102+0000","last_peered":"2023-10-14T10:36:05.484102+0000","last_clean":"2023-10-14T10:36:05.484102+0000","last_became_active":"2023-10-14T10:36:03.871841+0000","last_became_peered":"2023-10-14T10:36:03.871841+0000","last_unstale":"2023-10-14T10:36:05.484102+0000","last_undegraded":"2023-10-14T10:36:05.484102+0000","last_fullsized":"2023-10-14T10:36:05.484102+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:07:32.070142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.846600+0000","last_change":"2023-10-14T10:36:02.854777+0000","last_active":"2023-10-14T10:36:04.846600+0000","last_peered":"2023-10-14T10:36:04.846600+0000","last_clean":"2023-10-14T10:36:04.846600+0000","last_became_active":"2023-10-14T10:36:02.854126+0000","last_became_peered":"2023-10-14T10:36:02.854126+0000","last_unstale":"2023-10-14T10:36:04.846600+0000","last_undegraded":"2023-10-14T10:36:04.846600+0000","last_fullsized":"2023-10-14T10:36:04.846600+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:22:57.143570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.847151+0000","last_change":"2023-10-14T10:36:02.853800+0000","last_active":"2023-10-14T10:36:04.847151+0000","last_peered":"2023-10-14T10:36:04.847151+0000","last_clean":"2023-10-14T10:36:04.847151+0000","last_became_active":"2023-10-14T10:36:02.853508+0000","last_became_peered":"2023-10-14T10:36:02.853508+0000","last_unstale":"2023-10-14T10:36:04.847151+0000","last_undegraded":"2023-10-14T10:36:04.847151+0000","last_fullsized":"2023-10-14T10:36:04.847151+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:05:51.081176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.841103+0000","last_change":"2023-10-14T10:36:03.856928+0000","last_active":"2023-10-14T10:36:04.841103+0000","last_peered":"2023-10-14T10:36:04.841103+0000","last_clean":"2023-10-14T10:36:04.841103+0000","last_became_active":"2023-10-14T10:36:03.856616+0000","last_became_peered":"2023-10-14T10:36:03.856616+0000","last_unstale":"2023-10-14T10:36:04.841103+0000","last_undegraded":"2023-10-14T10:36:04.841103+0000","last_fullsized":"2023-10-14T10:36:04.841103+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:07:32.070142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.840950+0000","last_change":"2023-10-14T10:36:02.847787+0000","last_active":"2023-10-14T10:36:04.840950+0000","last_peered":"2023-10-14T10:36:04.840950+0000","last_clean":"2023-10-14T10:36:04.840950+0000","last_became_active":"2023-10-14T10:36:02.847468+0000","last_became_peered":"2023-10-14T10:36:02.847468+0000","last_unstale":"2023-10-14T10:36:04.840950+0000","last_undegraded":"2023-10-14T10:36:04.840950+0000","last_fullsized":"2023-10-14T10:36:04.840950+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T18:50:25.620127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.484430+0000","last_change":"2023-10-14T10:36:03.869759+0000","last_active":"2023-10-14T10:36:05.484430+0000","last_peered":"2023-10-14T10:36:05.484430+0000","last_clean":"2023-10-14T10:36:05.484430+0000","last_became_active":"2023-10-14T10:36:03.869404+0000","last_became_peered":"2023-10-14T10:36:03.869404+0000","last_unstale":"2023-10-14T10:36:05.484430+0000","last_undegraded":"2023-10-14T10:36:05.484430+0000","last_fullsized":"2023-10-14T10:36:05.484430+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:47:42.089047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-14T10:36:03.868372+0000","last_change":"2023-10-14T10:36:02.853796+0000","last_active":"2023-10-14T10:36:03.868372+0000","last_peered":"2023-10-14T10:36:03.868372+0000","last_clean":"2023-10-14T10:36:03.868372+0000","last_became_active":"2023-10-14T10:36:02.853468+0000","last_became_peered":"2023-10-14T10:36:02.853468+0000","last_unstale":"2023-10-14T10:36:03.868372+0000","last_undegraded":"2023-10-14T10:36:03.868372+0000","last_fullsized":"2023-10-14T10:36:03.868372+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:14:10.501109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.483634+0000","last_change":"2023-10-14T10:36:03.854935+0000","last_active":"2023-10-14T10:36:05.483634+0000","last_peered":"2023-10-14T10:36:05.483634+0000","last_clean":"2023-10-14T10:36:05.483634+0000","last_became_active":"2023-10-14T10:36:03.854648+0000","last_became_peered":"2023-10-14T10:36:03.854648+0000","last_unstale":"2023-10-14T10:36:05.483634+0000","last_undegraded":"2023-10-14T10:36:05.483634+0000","last_fullsized":"2023-10-14T10:36:05.483634+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T10:42:24.026846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.846475+0000","last_change":"2023-10-14T10:36:02.854850+0000","last_active":"2023-10-14T10:36:04.846475+0000","last_peered":"2023-10-14T10:36:04.846475+0000","last_clean":"2023-10-14T10:36:04.846475+0000","last_became_active":"2023-10-14T10:36:02.854442+0000","last_became_peered":"2023-10-14T10:36:02.854442+0000","last_unstale":"2023-10-14T10:36:04.846475+0000","last_undegraded":"2023-10-14T10:36:04.846475+0000","last_fullsized":"2023-10-14T10:36:04.846475+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T12:29:01.996039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.482660+0000","last_change":"2023-10-14T10:36:03.859500+0000","last_active":"2023-10-14T10:36:05.482660+0000","last_peered":"2023-10-14T10:36:05.482660+0000","last_clean":"2023-10-14T10:36:05.482660+0000","last_became_active":"2023-10-14T10:36:03.859247+0000","last_became_peered":"2023-10-14T10:36:03.859247+0000","last_unstale":"2023-10-14T10:36:05.482660+0000","last_undegraded":"2023-10-14T10:36:05.482660+0000","last_fullsized":"2023-10-14T10:36:05.482660+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:14:11.506725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.840826+0000","last_change":"2023-10-14T10:36:02.854036+0000","last_active":"2023-10-14T10:36:04.840826+0000","last_peered":"2023-10-14T10:36:04.840826+0000","last_clean":"2023-10-14T10:36:04.840826+0000","last_became_active":"2023-10-14T10:36:02.853723+0000","last_became_peered":"2023-10-14T10:36:02.853723+0000","last_unstale":"2023-10-14T10:36:04.840826+0000","last_undegraded":"2023-10-14T10:36:04.840826+0000","last_fullsized":"2023-10-14T10:36:04.840826+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:29:50.590931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.482772+0000","last_change":"2023-10-14T10:36:03.856047+0000","last_active":"2023-10-14T10:36:05.482772+0000","last_peered":"2023-10-14T10:36:05.482772+0000","last_clean":"2023-10-14T10:36:05.482772+0000","last_became_active":"2023-10-14T10:36:03.855747+0000","last_became_peered":"2023-10-14T10:36:03.855747+0000","last_unstale":"2023-10-14T10:36:05.482772+0000","last_undegraded":"2023-10-14T10:36:05.482772+0000","last_fullsized":"2023-10-14T10:36:05.482772+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T21:40:43.364841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.841238+0000","last_change":"2023-10-14T10:36:02.845255+0000","last_active":"2023-10-14T10:36:04.841238+0000","last_peered":"2023-10-14T10:36:04.841238+0000","last_clean":"2023-10-14T10:36:04.841238+0000","last_became_active":"2023-10-14T10:36:02.844939+0000","last_became_peered":"2023-10-14T10:36:02.844939+0000","last_unstale":"2023-10-14T10:36:04.841238+0000","last_undegraded":"2023-10-14T10:36:04.841238+0000","last_fullsized":"2023-10-14T10:36:04.841238+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T12:29:01.996039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.482888+0000","last_change":"2023-10-14T10:36:03.865197+0000","last_active":"2023-10-14T10:36:05.482888+0000","last_peered":"2023-10-14T10:36:05.482888+0000","last_clean":"2023-10-14T10:36:05.482888+0000","last_became_active":"2023-10-14T10:36:03.864885+0000","last_became_peered":"2023-10-14T10:36:03.864885+0000","last_unstale":"2023-10-14T10:36:05.482888+0000","last_undegraded":"2023-10-14T10:36:05.482888+0000","last_fullsized":"2023-10-14T10:36:05.482888+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:07:32.070142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.847061+0000","last_change":"2023-10-14T10:36:02.853880+0000","last_active":"2023-10-14T10:36:04.847061+0000","last_peered":"2023-10-14T10:36:04.847061+0000","last_clean":"2023-10-14T10:36:04.847061+0000","last_became_active":"2023-10-14T10:36:02.853574+0000","last_became_peered":"2023-10-14T10:36:02.853574+0000","last_unstale":"2023-10-14T10:36:04.847061+0000","last_undegraded":"2023-10-14T10:36:04.847061+0000","last_fullsized":"2023-10-14T10:36:04.847061+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T10:59:16.502697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.482245+0000","last_change":"2023-10-14T10:36:03.865872+0000","last_active":"2023-10-14T10:36:05.482245+0000","last_peered":"2023-10-14T10:36:05.482245+0000","last_clean":"2023-10-14T10:36:05.482245+0000","last_became_active":"2023-10-14T10:36:03.865624+0000","last_became_peered":"2023-10-14T10:36:03.865624+0000","last_unstale":"2023-10-14T10:36:05.482245+0000","last_undegraded":"2023-10-14T10:36:05.482245+0000","last_fullsized":"2023-10-14T10:36:05.482245+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T18:50:26.625744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-14T10:36:03.842738+0000","last_change":"2023-10-14T10:36:02.840500+0000","last_active":"2023-10-14T10:36:03.842738+0000","last_peered":"2023-10-14T10:36:03.842738+0000","last_clean":"2023-10-14T10:36:03.842738+0000","last_became_active":"2023-10-14T10:36:02.839996+0000","last_became_peered":"2023-10-14T10:36:02.839996+0000","last_unstale":"2023-10-14T10:36:03.842738+0000","last_undegraded":"2023-10-14T10:36:03.842738+0000","last_fullsized":"2023-10-14T10:36:03.842738+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T21:06:31.726029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.847199+0000","last_change":"2023-10-14T10:36:03.868941+0000","last_active":"2023-10-14T10:36:04.847199+0000","last_peered":"2023-10-14T10:36:04.847199+0000","last_clean":"2023-10-14T10:36:04.847199+0000","last_became_active":"2023-10-14T10:36:03.868609+0000","last_became_peered":"2023-10-14T10:36:03.868609+0000","last_unstale":"2023-10-14T10:36:04.847199+0000","last_undegraded":"2023-10-14T10:36:04.847199+0000","last_fullsized":"2023-10-14T10:36:04.847199+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:14:41.057137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.841048+0000","last_change":"2023-10-14T10:36:02.848004+0000","last_active":"2023-10-14T10:36:04.841048+0000","last_peered":"2023-10-14T10:36:04.841048+0000","last_clean":"2023-10-14T10:36:04.841048+0000","last_became_active":"2023-10-14T10:36:02.847494+0000","last_became_peered":"2023-10-14T10:36:02.847494+0000","last_unstale":"2023-10-14T10:36:04.841048+0000","last_undegraded":"2023-10-14T10:36:04.841048+0000","last_fullsized":"2023-10-14T10:36:04.841048+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T14:12:52.738066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.845506+0000","last_change":"2023-10-14T10:36:03.863211+0000","last_active":"2023-10-14T10:36:04.845506+0000","last_peered":"2023-10-14T10:36:04.845506+0000","last_clean":"2023-10-14T10:36:04.845506+0000","last_became_active":"2023-10-14T10:36:03.862894+0000","last_became_peered":"2023-10-14T10:36:03.862894+0000","last_unstale":"2023-10-14T10:36:04.845506+0000","last_undegraded":"2023-10-14T10:36:04.845506+0000","last_fullsized":"2023-10-14T10:36:04.845506+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:27:25.021169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-14T10:36:03.864535+0000","last_change":"2023-10-14T10:36:02.842899+0000","last_active":"2023-10-14T10:36:03.864535+0000","last_peered":"2023-10-14T10:36:03.864535+0000","last_clean":"2023-10-14T10:36:03.864535+0000","last_became_active":"2023-10-14T10:36:02.842620+0000","last_became_peered":"2023-10-14T10:36:02.842620+0000","last_unstale":"2023-10-14T10:36:03.864535+0000","last_undegraded":"2023-10-14T10:36:03.864535+0000","last_fullsized":"2023-10-14T10:36:03.864535+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T12:44:16.134070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.847179+0000","last_change":"2023-10-14T10:36:03.872451+0000","last_active":"2023-10-14T10:36:04.847179+0000","last_peered":"2023-10-14T10:36:04.847179+0000","last_clean":"2023-10-14T10:36:04.847179+0000","last_became_active":"2023-10-14T10:36:03.872212+0000","last_became_peered":"2023-10-14T10:36:03.872212+0000","last_unstale":"2023-10-14T10:36:04.847179+0000","last_undegraded":"2023-10-14T10:36:04.847179+0000","last_fullsized":"2023-10-14T10:36:04.847179+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T12:31:06.179885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.482245+0000","last_change":"2023-10-14T10:36:03.862676+0000","last_active":"2023-10-14T10:36:05.482245+0000","last_peered":"2023-10-14T10:36:05.482245+0000","last_clean":"2023-10-14T10:36:05.482245+0000","last_became_active":"2023-10-14T10:36:03.862364+0000","last_became_peered":"2023-10-14T10:36:03.862364+0000","last_unstale":"2023-10-14T10:36:05.482245+0000","last_undegraded":"2023-10-14T10:36:05.482245+0000","last_fullsized":"2023-10-14T10:36:05.482245+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T10:42:24.026846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.483021+0000","last_change":"2023-10-14T10:36:03.856448+0000","last_active":"2023-10-14T10:36:05.483021+0000","last_peered":"2023-10-14T10:36:05.483021+0000","last_clean":"2023-10-14T10:36:05.483021+0000","last_became_active":"2023-10-14T10:36:03.856165+0000","last_became_peered":"2023-10-14T10:36:03.856165+0000","last_unstale":"2023-10-14T10:36:05.483021+0000","last_undegraded":"2023-10-14T10:36:05.483021+0000","last_fullsized":"2023-10-14T10:36:05.483021+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:21:06.827004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.845673+0000","last_change":"2023-10-14T10:36:03.862566+0000","last_active":"2023-10-14T10:36:04.845673+0000","last_peered":"2023-10-14T10:36:04.845673+0000","last_clean":"2023-10-14T10:36:04.845673+0000","last_became_active":"2023-10-14T10:36:03.862028+0000","last_became_peered":"2023-10-14T10:36:03.862028+0000","last_unstale":"2023-10-14T10:36:04.845673+0000","last_undegraded":"2023-10-14T10:36:04.845673+0000","last_fullsized":"2023-10-14T10:36:04.845673+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T10:36:37.449003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.483846+0000","last_change":"2023-10-14T10:36:03.865325+0000","last_active":"2023-10-14T10:36:05.483846+0000","last_peered":"2023-10-14T10:36:05.483846+0000","last_clean":"2023-10-14T10:36:05.483846+0000","last_became_active":"2023-10-14T10:36:03.865034+0000","last_became_peered":"2023-10-14T10:36:03.865034+0000","last_unstale":"2023-10-14T10:36:05.483846+0000","last_undegraded":"2023-10-14T10:36:05.483846+0000","last_fullsized":"2023-10-14T10:36:05.483846+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T21:40:43.364841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.483268+0000","last_change":"2023-10-14T10:36:03.857566+0000","last_active":"2023-10-14T10:36:05.483268+0000","last_peered":"2023-10-14T10:36:05.483268+0000","last_clean":"2023-10-14T10:36:05.483268+0000","last_became_active":"2023-10-14T10:36:03.857301+0000","last_became_peered":"2023-10-14T10:36:03.857301+0000","last_unstale":"2023-10-14T10:36:05.483268+0000","last_undegraded":"2023-10-14T10:36:05.483268+0000","last_fullsized":"2023-10-14T10:36:05.483268+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T18:49:18.106622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.841002+0000","last_change":"2023-10-14T10:36:03.863392+0000","last_active":"2023-10-14T10:36:04.841002+0000","last_peered":"2023-10-14T10:36:04.841002+0000","last_clean":"2023-10-14T10:36:04.841002+0000","last_became_active":"2023-10-14T10:36:03.863097+0000","last_became_peered":"2023-10-14T10:36:03.863097+0000","last_unstale":"2023-10-14T10:36:04.841002+0000","last_undegraded":"2023-10-14T10:36:04.841002+0000","last_fullsized":"2023-10-14T10:36:04.841002+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:05:52.086793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.482815+0000","last_change":"2023-10-14T10:36:03.864988+0000","last_active":"2023-10-14T10:36:05.482815+0000","last_peered":"2023-10-14T10:36:05.482815+0000","last_clean":"2023-10-14T10:36:05.482815+0000","last_became_active":"2023-10-14T10:36:03.864699+0000","last_became_peered":"2023-10-14T10:36:03.864699+0000","last_unstale":"2023-10-14T10:36:05.482815+0000","last_undegraded":"2023-10-14T10:36:05.482815+0000","last_fullsized":"2023-10-14T10:36:05.482815+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T18:42:51.749474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.484472+0000","last_change":"2023-10-14T10:36:03.855927+0000","last_active":"2023-10-14T10:36:05.484472+0000","last_peered":"2023-10-14T10:36:05.484472+0000","last_clean":"2023-10-14T10:36:05.484472+0000","last_became_active":"2023-10-14T10:36:03.855643+0000","last_became_peered":"2023-10-14T10:36:03.855643+0000","last_unstale":"2023-10-14T10:36:05.484472+0000","last_undegraded":"2023-10-14T10:36:05.484472+0000","last_fullsized":"2023-10-14T10:36:05.484472+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T20:52:07.681036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.845305+0000","last_change":"2023-10-14T10:36:03.864752+0000","last_active":"2023-10-14T10:36:04.845305+0000","last_peered":"2023-10-14T10:36:04.845305+0000","last_clean":"2023-10-14T10:36:04.845305+0000","last_became_active":"2023-10-14T10:36:03.864480+0000","last_became_peered":"2023-10-14T10:36:03.864480+0000","last_unstale":"2023-10-14T10:36:04.845305+0000","last_undegraded":"2023-10-14T10:36:04.845305+0000","last_fullsized":"2023-10-14T10:36:04.845305+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:05:52.086793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.482676+0000","last_change":"2023-10-14T10:36:03.858250+0000","last_active":"2023-10-14T10:36:05.482676+0000","last_peered":"2023-10-14T10:36:05.482676+0000","last_clean":"2023-10-14T10:36:05.482676+0000","last_became_active":"2023-10-14T10:36:03.857940+0000","last_became_peered":"2023-10-14T10:36:03.857940+0000","last_unstale":"2023-10-14T10:36:05.482676+0000","last_undegraded":"2023-10-14T10:36:05.482676+0000","last_fullsized":"2023-10-14T10:36:05.482676+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T17:28:51.336361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.845474+0000","last_change":"2023-10-14T10:36:03.866904+0000","last_active":"2023-10-14T10:36:04.845474+0000","last_peered":"2023-10-14T10:36:04.845474+0000","last_clean":"2023-10-14T10:36:04.845474+0000","last_became_active":"2023-10-14T10:36:03.866439+0000","last_became_peered":"2023-10-14T10:36:03.866439+0000","last_unstale":"2023-10-14T10:36:04.845474+0000","last_undegraded":"2023-10-14T10:36:04.845474+0000","last_fullsized":"2023-10-14T10:36:04.845474+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T18:49:18.106622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.483685+0000","last_change":"2023-10-14T10:36:03.868286+0000","last_active":"2023-10-14T10:36:05.483685+0000","last_peered":"2023-10-14T10:36:05.483685+0000","last_clean":"2023-10-14T10:36:05.483685+0000","last_became_active":"2023-10-14T10:36:03.868050+0000","last_became_peered":"2023-10-14T10:36:03.868050+0000","last_unstale":"2023-10-14T10:36:05.483685+0000","last_undegraded":"2023-10-14T10:36:05.483685+0000","last_fullsized":"2023-10-14T10:36:05.483685+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T18:50:26.625744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.482746+0000","last_change":"2023-10-14T10:36:03.872106+0000","last_active":"2023-10-14T10:36:05.482746+0000","last_peered":"2023-10-14T10:36:05.482746+0000","last_clean":"2023-10-14T10:36:05.482746+0000","last_became_active":"2023-10-14T10:36:03.871788+0000","last_became_peered":"2023-10-14T10:36:03.871788+0000","last_unstale":"2023-10-14T10:36:05.482746+0000","last_undegraded":"2023-10-14T10:36:05.482746+0000","last_fullsized":"2023-10-14T10:36:05.482746+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T17:28:51.336361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542149,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":823296,"data_stored":669814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94999999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80700000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0129999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91400000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44400000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542150,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":237568,"data_stored":80476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63800000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66800000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.372}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51400000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47199999999999998}]}]},{"osd":1,"up_from":14,"seq":60129542149,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":233472,"data_stored":79024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32900000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27600000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60199999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.629}]}]},{"osd":0,"up_from":14,"seq":60129542148,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27832,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344008,"statfs":{"total":96636764160,"available":96608264192,"internally_reserved":0,"allocated":831488,"data_stored":669870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40600000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60799999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64100000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542149,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":225280,"data_stored":78864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32800000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58699999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]}]},{"osd":3,"up_from":14,"seq":60129542149,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":237568,"data_stored":80476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76500000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]}]},{"osd":4,"up_from":14,"seq":60129542148,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":237568,"data_stored":78990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79300000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.872}]}]},{"osd":5,"up_from":14,"seq":60129542149,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":78954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56399999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66200000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.436}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41499999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-14T10:36:08.407 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-14T10:36:08.408 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-10-14T10:36:08.408 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-10-14T10:36:08.409 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-10-14T10:36:08.409 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-10-14T10:36:08.567 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.565+0000 7f7faa2f6700 1 Processor -- start 2023-10-14T10:36:08.568 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.565+0000 7f7faa2f6700 1 -- start start 2023-10-14T10:36:08.571 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.566+0000 7f7faa2f6700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f7fa406af50 0x7f7fa40cede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:08.571 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.566+0000 7f7faa2f6700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fa40b3530 con 0x7f7fa40b28b0 2023-10-14T10:36:08.571 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.566+0000 7f7faa2f6700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fa40cd180 con 0x7f7fa415f130 2023-10-14T10:36:08.571 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.566+0000 7f7faa2f6700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fa40cde00 con 0x7f7fa406af50 2023-10-14T10:36:08.571 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.566+0000 7f7fa8893700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f7fa40b28b0 0x7f7fa40b3960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:51664/0 (socket says 172.21.15.67:51664) 2023-10-14T10:36:08.572 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.566+0000 7f7fa8893700 1 -- 172.21.15.67:0/1729629840 learned_addr learned my addr 172.21.15.67:0/1729629840 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:36:08.572 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.566+0000 7f7fa37fe700 1 --2- 172.21.15.67:0/1729629840 >> v2:172.21.15.72:3300/0 conn(0x7f7fa406af50 0x7f7fa40cede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:08.572 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.566+0000 7f7fa2ffd700 1 -- 172.21.15.67:0/1729629840 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2852563701 0 0) 0x7f7fa40b3530 con 0x7f7fa40b28b0 2023-10-14T10:36:08.572 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.567+0000 7f7fa2ffd700 1 -- 172.21.15.67:0/1729629840 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f8c0035b0 con 0x7f7fa40b28b0 2023-10-14T10:36:08.572 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.567+0000 7f7fa2ffd700 1 -- 172.21.15.67:0/1729629840 <== mon.1 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4217978335 0 0) 0x7f7fa40cd180 con 0x7f7fa415f130 2023-10-14T10:36:08.572 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.567+0000 7f7fa2ffd700 1 -- 172.21.15.67:0/1729629840 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7fa40b3530 con 0x7f7fa415f130 2023-10-14T10:36:08.573 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.567+0000 7f7fa2ffd700 1 -- 172.21.15.67:0/1729629840 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2559248228 0 0) 0x7f7f8c0035b0 con 0x7f7fa40b28b0 2023-10-14T10:36:08.573 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.567+0000 7f7fa37fe700 1 -- 172.21.15.67:0/1729629840 >> v1:172.21.15.72:6789/0 conn(0x7f7fa415f130 legacy=0x7f7fa406a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:08.573 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.567+0000 7f7fa37fe700 1 -- 172.21.15.67:0/1729629840 >> v1:172.21.15.67:6789/0 conn(0x7f7fa40b28b0 legacy=0x7f7fa40b3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:08.573 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.567+0000 7f7fa37fe700 1 -- 172.21.15.67:0/1729629840 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fa40cd460 con 0x7f7fa406af50 2023-10-14T10:36:08.573 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.567+0000 7f7fa37fe700 1 --2- 172.21.15.67:0/1729629840 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7fa406af50 0x7f7fa40cede0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7f98009820 tx=0x7f7f98009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a367e699670ba807 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:08.574 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.567+0000 7f7fa2ffd700 1 -- 172.21.15.67:0/1729629840 <== mon.1 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f9c0019f0 con 0x7f7fa406af50 2023-10-14T10:36:08.574 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.567+0000 7f7fa2ffd700 1 -- 172.21.15.67:0/1729629840 <== mon.1 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f98005c30 con 0x7f7fa406af50 2023-10-14T10:36:08.574 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.567+0000 7f7fa2ffd700 1 -- 172.21.15.67:0/1729629840 <== mon.1 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f98004030 con 0x7f7fa406af50 2023-10-14T10:36:08.574 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.568+0000 7f7faa2f6700 1 -- 172.21.15.67:0/1729629840 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7fa406af50 msgr2=0x7f7fa40cede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:08.574 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.568+0000 7f7faa2f6700 1 --2- 172.21.15.67:0/1729629840 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7fa406af50 0x7f7fa40cede0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7f98009820 tx=0x7f7f98009b30 comp rx=0 tx=0).stop 2023-10-14T10:36:08.574 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.568+0000 7f7faa2f6700 1 -- 172.21.15.67:0/1729629840 shutdown_connections 2023-10-14T10:36:08.575 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.568+0000 7f7faa2f6700 1 --2- 172.21.15.67:0/1729629840 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7fa406af50 0x7f7fa40cede0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:08.575 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.568+0000 7f7faa2f6700 1 -- 172.21.15.67:0/1729629840 >> 172.21.15.67:0/1729629840 conn(0x7f7fa406dae0 msgr2=0x7f7fa406def0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:08.575 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.568+0000 7f7faa2f6700 1 -- 172.21.15.67:0/1729629840 shutdown_connections 2023-10-14T10:36:08.575 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.568+0000 7f7faa2f6700 1 -- 172.21.15.67:0/1729629840 wait complete. 2023-10-14T10:36:08.575 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.569+0000 7f7faa2f6700 1 Processor -- start 2023-10-14T10:36:08.578 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.569+0000 7f7faa2f6700 1 -- start start 2023-10-14T10:36:08.578 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.570+0000 7f7faa2f6700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f7fa406af50 0x7f7fa4156b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:08.578 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.570+0000 7f7faa2f6700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7fa40b28b0 0x7f7fa40c9000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:08.579 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.570+0000 7f7faa2f6700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7fa415f130 0x7f7fa415ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:08.579 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.570+0000 7f7faa2f6700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa40c9540 con 0x7f7fa415f130 2023-10-14T10:36:08.579 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.570+0000 7f7faa2f6700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa40c97e0 con 0x7f7fa40b28b0 2023-10-14T10:36:08.579 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.570+0000 7f7faa2f6700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa40c9ac0 con 0x7f7fa406af50 2023-10-14T10:36:08.579 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.570+0000 7f7fa8893700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7fa415f130 0x7f7fa415ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:08.580 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.570+0000 7f7fa8893700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7fa415f130 0x7f7fa415ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43904/0 (socket says 172.21.15.67:43904) 2023-10-14T10:36:08.580 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.570+0000 7f7fa8893700 1 -- 172.21.15.67:0/2966022332 learned_addr learned my addr 172.21.15.67:0/2966022332 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:08.580 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.570+0000 7f7fa37fe700 1 --2- 172.21.15.67:0/2966022332 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7fa40b28b0 0x7f7fa40c9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:08.580 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.570+0000 7f7fa3fff700 1 --2- 172.21.15.67:0/2966022332 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f7fa406af50 0x7f7fa4156b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:08.580 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.570+0000 7f7fa8893700 1 -- 172.21.15.67:0/2966022332 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f7fa406af50 msgr2=0x7f7fa4156b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:08.581 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.570+0000 7f7fa8893700 1 --2- 172.21.15.67:0/2966022332 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f7fa406af50 0x7f7fa4156b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:08.581 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.570+0000 7f7fa8893700 1 -- 172.21.15.67:0/2966022332 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7fa40b28b0 msgr2=0x7f7fa40c9000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:08.581 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.570+0000 7f7fa8893700 1 --2- 172.21.15.67:0/2966022332 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7fa40b28b0 0x7f7fa40c9000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:08.581 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.570+0000 7f7fa8893700 1 -- 172.21.15.67:0/2966022332 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f98009430 con 0x7f7fa415f130 2023-10-14T10:36:08.581 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.570+0000 7f7fa37fe700 1 --2- 172.21.15.67:0/2966022332 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7fa40b28b0 0x7f7fa40c9000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-14T10:36:08.582 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.571+0000 7f7fa8893700 1 --2- 172.21.15.67:0/2966022332 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7fa415f130 0x7f7fa415ca00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c0017f0 tx=0x7f7f9c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:08.582 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.571+0000 7f7fa0ff9700 1 -- 172.21.15.67:0/2966022332 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f9c0019f0 con 0x7f7fa415f130 2023-10-14T10:36:08.582 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.571+0000 7f7fa0ff9700 1 -- 172.21.15.67:0/2966022332 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f9c00ade0 con 0x7f7fa415f130 2023-10-14T10:36:08.582 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.571+0000 7f7faa2f6700 1 -- 172.21.15.67:0/2966022332 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fa40c9da0 con 0x7f7fa415f130 2023-10-14T10:36:08.582 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.571+0000 7f7fa0ff9700 1 -- 172.21.15.67:0/2966022332 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f9c007310 con 0x7f7fa415f130 2023-10-14T10:36:08.583 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.571+0000 7f7faa2f6700 1 -- 172.21.15.67:0/2966022332 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fa415cf40 con 0x7f7fa415f130 2023-10-14T10:36:08.583 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.572+0000 7f7faa2f6700 1 -- 172.21.15.67:0/2966022332 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fa415a9f0 con 0x7f7fa415f130 2023-10-14T10:36:08.583 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.572+0000 7f7fa0ff9700 1 -- 172.21.15.67:0/2966022332 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f7f9c007470 con 0x7f7fa415f130 2023-10-14T10:36:08.583 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.573+0000 7f7fa0ff9700 1 --2- 172.21.15.67:0/2966022332 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f7f9006af00 0x7f7f9006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:08.583 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.573+0000 7f7fa0ff9700 1 -- 172.21.15.67:0/2966022332 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7f9c089520 con 0x7f7fa415f130 2023-10-14T10:36:08.584 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.573+0000 7f7fa3fff700 1 --2- 172.21.15.67:0/2966022332 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f7f9006af00 0x7f7f9006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:08.584 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.573+0000 7f7fa3fff700 1 --2- 172.21.15.67:0/2966022332 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f7f9006af00 0x7f7f9006d3c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7f94007fd0 tx=0x7f7f94009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:08.584 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.577+0000 7f7fa0ff9700 1 -- 172.21.15.67:0/2966022332 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7fa415a9f0 con 0x7f7fa415f130 2023-10-14T10:36:08.836 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.834+0000 7f7faa2f6700 1 -- 172.21.15.67:0/2966022332 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f7fa415a9f0 con 0x7f7fa415f130 2023-10-14T10:36:08.839 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.837+0000 7f7fa0ff9700 1 -- 172.21.15.67:0/2966022332 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+171026 (secure 0 0 0) 0x7f7fa415a9f0 con 0x7f7fa415f130 2023-10-14T10:36:08.842 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-14T10:36:08.843 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.840+0000 7f7faa2f6700 1 -- 172.21.15.67:0/2966022332 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f7f9006af00 msgr2=0x7f7f9006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:08.844 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.840+0000 7f7faa2f6700 1 --2- 172.21.15.67:0/2966022332 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f7f9006af00 0x7f7f9006d3c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7f94007fd0 tx=0x7f7f94009040 comp rx=0 tx=0).stop 2023-10-14T10:36:08.844 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.840+0000 7f7faa2f6700 1 -- 172.21.15.67:0/2966022332 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7fa415f130 msgr2=0x7f7fa415ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:08.844 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.840+0000 7f7faa2f6700 1 --2- 172.21.15.67:0/2966022332 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7fa415f130 0x7f7fa415ca00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c0017f0 tx=0x7f7f9c00a340 comp rx=0 tx=0).stop 2023-10-14T10:36:08.844 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.840+0000 7f7faa2f6700 1 -- 172.21.15.67:0/2966022332 shutdown_connections 2023-10-14T10:36:08.845 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.840+0000 7f7faa2f6700 1 --2- 172.21.15.67:0/2966022332 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f7f9006af00 0x7f7f9006d3c0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:08.845 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.840+0000 7f7faa2f6700 1 --2- 172.21.15.67:0/2966022332 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7fa415f130 0x7f7fa415ca00 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:08.845 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.840+0000 7f7faa2f6700 1 --2- 172.21.15.67:0/2966022332 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f7fa406af50 0x7f7fa4156b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:08.845 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.840+0000 7f7faa2f6700 1 --2- 172.21.15.67:0/2966022332 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7fa40b28b0 0x7f7fa40c9000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:08.845 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.840+0000 7f7faa2f6700 1 -- 172.21.15.67:0/2966022332 >> 172.21.15.67:0/2966022332 conn(0x7f7fa406dae0 msgr2=0x7f7fa40b5de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:08.846 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.840+0000 7f7faa2f6700 1 -- 172.21.15.67:0/2966022332 shutdown_connections 2023-10-14T10:36:08.846 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.841+0000 7f7faa2f6700 1 -- 172.21.15.67:0/2966022332 wait complete. 2023-10-14T10:36:08.855 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":5,"active_gid":4100,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":703488382},{"type":"v1","addr":"172.21.15.72:6833","nonce":703488382}]},"active_addr":"172.21.15.72:6833/703488382","active_change":"2023-10-14T10:35:51.747022+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4110,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.72:0","nonce":1855828278}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.72:0","nonce":1329697984}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.72:0","nonce":1231181351}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.72:0","nonce":401538891}]}]} 2023-10-14T10:36:08.859 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-10-14T10:36:08.859 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-10-14T10:36:08.860 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-14T10:36:08.985 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.983+0000 7fbd77ad1700 1 Processor -- start 2023-10-14T10:36:08.986 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.984+0000 7fbd77ad1700 1 -- start start 2023-10-14T10:36:08.988 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.984+0000 7fbd77ad1700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fbd700b54f0 0x7fbd7015ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:08.988 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.984+0000 7fbd77ad1700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd700b5c70 con 0x7fbd7006e750 2023-10-14T10:36:08.989 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.984+0000 7fbd77ad1700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd700b5990 con 0x7fbd7006eb30 2023-10-14T10:36:08.989 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.984+0000 7fbd77ad1700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd7015b2b0 con 0x7fbd700b54f0 2023-10-14T10:36:08.989 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.985+0000 7fbd7606e700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fbd700b54f0 0x7fbd7015ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:08.989 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.985+0000 7fbd7586d700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fbd7006e750 0x7fbd700ceec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:51676/0 (socket says 172.21.15.67:51676) 2023-10-14T10:36:08.989 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.985+0000 7fbd7586d700 1 -- 172.21.15.67:0/155886657 learned_addr learned my addr 172.21.15.67:0/155886657 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:36:08.990 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.985+0000 7fbd7606e700 1 -- 172.21.15.67:0/155886657 >> v1:172.21.15.72:6789/0 conn(0x7fbd7006eb30 legacy=0x7fbd7015e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:08.990 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.985+0000 7fbd7606e700 1 -- 172.21.15.67:0/155886657 >> v1:172.21.15.67:6789/0 conn(0x7fbd7006e750 legacy=0x7fbd700ceec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:08.990 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.985+0000 7fbd7486b700 1 -- 172.21.15.67:0/155886657 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 166813839 0 0) 0x7fbd700b5c70 con 0x7fbd7006e750 2023-10-14T10:36:08.990 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.985+0000 7fbd7606e700 1 -- 172.21.15.67:0/155886657 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd7015b590 con 0x7fbd700b54f0 2023-10-14T10:36:08.991 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.985+0000 7fbd7606e700 1 --2- 172.21.15.67:0/155886657 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd700b54f0 0x7fbd7015ee10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fbd68009dc0 tx=0x7fbd6800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be1ff550cfc82ca0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:08.991 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.985+0000 7fbd7486b700 1 -- 172.21.15.67:0/155886657 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd6800e070 con 0x7fbd700b54f0 2023-10-14T10:36:08.991 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.985+0000 7fbd7486b700 1 -- 172.21.15.67:0/155886657 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd6800bb90 con 0x7fbd700b54f0 2023-10-14T10:36:08.991 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.985+0000 7fbd7486b700 1 -- 172.21.15.67:0/155886657 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd68012430 con 0x7fbd700b54f0 2023-10-14T10:36:08.991 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.986+0000 7fbd77ad1700 1 -- 172.21.15.67:0/155886657 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd700b54f0 msgr2=0x7fbd7015ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:08.992 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.986+0000 7fbd77ad1700 1 --2- 172.21.15.67:0/155886657 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd700b54f0 0x7fbd7015ee10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fbd68009dc0 tx=0x7fbd6800b330 comp rx=0 tx=0).stop 2023-10-14T10:36:08.992 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.986+0000 7fbd77ad1700 1 -- 172.21.15.67:0/155886657 shutdown_connections 2023-10-14T10:36:08.992 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.986+0000 7fbd77ad1700 1 --2- 172.21.15.67:0/155886657 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd700b54f0 0x7fbd7015ee10 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:08.992 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.986+0000 7fbd77ad1700 1 -- 172.21.15.67:0/155886657 >> 172.21.15.67:0/155886657 conn(0x7fbd700cc6f0 msgr2=0x7fbd700ccb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:08.993 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.986+0000 7fbd77ad1700 1 -- 172.21.15.67:0/155886657 shutdown_connections 2023-10-14T10:36:08.993 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.986+0000 7fbd77ad1700 1 -- 172.21.15.67:0/155886657 wait complete. 2023-10-14T10:36:08.996 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.987+0000 7fbd77ad1700 1 Processor -- start 2023-10-14T10:36:08.996 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.987+0000 7fbd77ad1700 1 -- start start 2023-10-14T10:36:08.996 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.988+0000 7fbd77ad1700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd7006e750 0x7fbd7014c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:08.996 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.988+0000 7fbd77ad1700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fbd7006eb30 0x7fbd70150030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:08.997 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.988+0000 7fbd77ad1700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbd700b54f0 0x7fbd701497c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:08.997 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.988+0000 7fbd77ad1700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd7014c720 con 0x7fbd7006e750 2023-10-14T10:36:08.997 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.988+0000 7fbd77ad1700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd7014ca00 con 0x7fbd700b54f0 2023-10-14T10:36:08.997 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.988+0000 7fbd77ad1700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd7014cce0 con 0x7fbd7006eb30 2023-10-14T10:36:08.997 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.988+0000 7fbd7586d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd7006e750 0x7fbd7014c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:08.998 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.988+0000 7fbd7586d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd7006e750 0x7fbd7014c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43914/0 (socket says 172.21.15.67:43914) 2023-10-14T10:36:08.998 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.988+0000 7fbd7586d700 1 -- 172.21.15.67:0/2431678234 learned_addr learned my addr 172.21.15.67:0/2431678234 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:08.998 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.988+0000 7fbd7606e700 1 --2- 172.21.15.67:0/2431678234 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbd700b54f0 0x7fbd701497c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:08.998 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.988+0000 7fbd7586d700 1 -- 172.21.15.67:0/2431678234 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fbd7006eb30 msgr2=0x7fbd70150030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:08.998 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.988+0000 7fbd7586d700 1 --2- 172.21.15.67:0/2431678234 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fbd7006eb30 0x7fbd70150030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:08.999 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.988+0000 7fbd7586d700 1 -- 172.21.15.67:0/2431678234 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbd700b54f0 msgr2=0x7fbd701497c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:08.999 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.988+0000 7fbd7586d700 1 --2- 172.21.15.67:0/2431678234 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbd700b54f0 0x7fbd701497c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:08.999 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.988+0000 7fbd7586d700 1 -- 172.21.15.67:0/2431678234 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd68009a70 con 0x7fbd7006e750 2023-10-14T10:36:08.999 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.988+0000 7fbd7586d700 1 --2- 172.21.15.67:0/2431678234 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd7006e750 0x7fbd7014c140 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fbd6000c520 tx=0x7fbd6000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:08.999 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.988+0000 7fbd667fc700 1 -- 172.21.15.67:0/2431678234 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd60004c80 con 0x7fbd7006e750 2023-10-14T10:36:09.000 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.988+0000 7fbd667fc700 1 -- 172.21.15.67:0/2431678234 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd60004e20 con 0x7fbd7006e750 2023-10-14T10:36:09.000 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.989+0000 7fbd667fc700 1 -- 172.21.15.67:0/2431678234 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd60007310 con 0x7fbd7006e750 2023-10-14T10:36:09.000 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.989+0000 7fbd77ad1700 1 -- 172.21.15.67:0/2431678234 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd701f6dc0 con 0x7fbd7006e750 2023-10-14T10:36:09.000 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.989+0000 7fbd77ad1700 1 -- 172.21.15.67:0/2431678234 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd7014cfc0 con 0x7fbd7006e750 2023-10-14T10:36:09.000 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.989+0000 7fbd77ad1700 1 -- 172.21.15.67:0/2431678234 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd7006dd00 con 0x7fbd7006e750 2023-10-14T10:36:09.000 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.990+0000 7fbd667fc700 1 -- 172.21.15.67:0/2431678234 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fbd60007470 con 0x7fbd7006e750 2023-10-14T10:36:09.001 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.990+0000 7fbd667fc700 1 --2- 172.21.15.67:0/2431678234 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fbd5406af00 0x7fbd5406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:09.001 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.990+0000 7fbd667fc700 1 -- 172.21.15.67:0/2431678234 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbd60089420 con 0x7fbd7006e750 2023-10-14T10:36:09.001 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.991+0000 7fbd7506c700 1 --2- 172.21.15.67:0/2431678234 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fbd5406af00 0x7fbd5406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:09.001 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.991+0000 7fbd7506c700 1 --2- 172.21.15.67:0/2431678234 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fbd5406af00 0x7fbd5406d3c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbd700ada10 tx=0x7fbd6c007a20 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:09.002 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:08.995+0000 7fbd667fc700 1 -- 172.21.15.67:0/2431678234 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbd60054430 con 0x7fbd7006e750 2023-10-14T10:36:09.198 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.196+0000 7fbd77ad1700 1 -- 172.21.15.67:0/2431678234 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbd7006b950 con 0x7fbd7006e750 2023-10-14T10:36:09.199 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.197+0000 7fbd667fc700 1 -- 172.21.15.67:0/2431678234 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16457 (secure 0 0 0) 0x7fbd60057a50 con 0x7fbd7006e750 2023-10-14T10:36:09.199 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-14T10:36:09.199 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":22,"fsid":"37f9ab46-3082-4b71-8b24-08d39897b008","created":"2023-10-14T10:35:46.067554+0000","modified":"2023-10-14T10:36:04.837296+0000","last_up_change":"2023-10-14T10:35:56.668142+0000","last_in_change":"2023-10-14T10:35:48.551490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-14T10:35:57.809566+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-14T10:36:01.667146+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-14T10:36:02.219700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2b59a07e-11ef-440c-b766-9f64ac885838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6801","nonce":1287255499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6803","nonce":1287255499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6807","nonce":1287255499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6805","nonce":1287255499}]},"public_addr":"172.21.15.67:6801/1287255499","cluster_addr":"172.21.15.67:6803/1287255499","heartbeat_back_addr":"172.21.15.67:6807/1287255499","heartbeat_front_addr":"172.21.15.67:6805/1287255499","state":["exists","up"]},{"osd":1,"uuid":"6f813bd6-fa0b-487d-a204-431eb1276a60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6818","nonce":2435538787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6822","nonce":2435538787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6830","nonce":2435538787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6826","nonce":2435538787}]},"public_addr":"172.21.15.67:6818/2435538787","cluster_addr":"172.21.15.67:6822/2435538787","heartbeat_back_addr":"172.21.15.67:6830/2435538787","heartbeat_front_addr":"172.21.15.67:6826/2435538787","state":["exists","up"]},{"osd":2,"uuid":"6af42c6a-1b59-4669-9c04-b6876788446d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6809","nonce":2827491791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6811","nonce":2827491791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6815","nonce":2827491791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6813","nonce":2827491791}]},"public_addr":"172.21.15.67:6809/2827491791","cluster_addr":"172.21.15.67:6811/2827491791","heartbeat_back_addr":"172.21.15.67:6815/2827491791","heartbeat_front_addr":"172.21.15.67:6813/2827491791","state":["exists","up"]},{"osd":3,"uuid":"1d889d76-a646-499d-bfff-ed5bae5434ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6819","nonce":1848001176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6821","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6823","nonce":1848001176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6831","nonce":1848001176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6825","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6827","nonce":1848001176}]},"public_addr":"172.21.15.67:6819/1848001176","cluster_addr":"172.21.15.67:6823/1848001176","heartbeat_back_addr":"172.21.15.67:6831/1848001176","heartbeat_front_addr":"172.21.15.67:6827/1848001176","state":["exists","up"]},{"osd":4,"uuid":"c3271dff-b437-4fdd-b13b-6e49a35473af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6804","nonce":1891230785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6808","nonce":1891230785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6815","nonce":1891230785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6811","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6813","nonce":1891230785}]},"public_addr":"172.21.15.72:6804/1891230785","cluster_addr":"172.21.15.72:6808/1891230785","heartbeat_back_addr":"172.21.15.72:6815/1891230785","heartbeat_front_addr":"172.21.15.72:6813/1891230785","state":["exists","up"]},{"osd":5,"uuid":"0337bb18-4b68-4eff-b528-a667275e6926","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6801","nonce":569686991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6803","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6805","nonce":569686991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6812","nonce":569686991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6807","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6809","nonce":569686991}]},"public_addr":"172.21.15.72:6801/569686991","cluster_addr":"172.21.15.72:6805/569686991","heartbeat_back_addr":"172.21.15.72:6812/569686991","heartbeat_front_addr":"172.21.15.72:6809/569686991","state":["exists","up"]},{"osd":6,"uuid":"8e92d9cf-0c8c-46eb-89a7-db2023ef1509","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6817","nonce":3767685934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6819","nonce":3767685934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6823","nonce":3767685934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6821","nonce":3767685934}]},"public_addr":"172.21.15.72:6817/3767685934","cluster_addr":"172.21.15.72:6819/3767685934","heartbeat_back_addr":"172.21.15.72:6823/3767685934","heartbeat_front_addr":"172.21.15.72:6821/3767685934","state":["exists","up"]},{"osd":7,"uuid":"3fd6aa7d-50a9-45c9-9561-5902a044a119","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6825","nonce":3567465926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6827","nonce":3567465926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6831","nonce":3567465926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6829","nonce":3567465926}]},"public_addr":"172.21.15.72:6825/3567465926","cluster_addr":"172.21.15.72:6827/3567465926","heartbeat_back_addr":"172.21.15.72:6831/3567465926","heartbeat_front_addr":"172.21.15.72:6829/3567465926","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.511466+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.795107+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.858674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.728895+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.794842+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.750064+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.732122+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.765185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-14T10:36:09.201 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.199+0000 7fbd77ad1700 1 -- 172.21.15.67:0/2431678234 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fbd5406af00 msgr2=0x7fbd5406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:09.201 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.199+0000 7fbd77ad1700 1 --2- 172.21.15.67:0/2431678234 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fbd5406af00 0x7fbd5406d3c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbd700ada10 tx=0x7fbd6c007a20 comp rx=0 tx=0).stop 2023-10-14T10:36:09.202 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.199+0000 7fbd77ad1700 1 -- 172.21.15.67:0/2431678234 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd7006e750 msgr2=0x7fbd7014c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:09.202 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.199+0000 7fbd77ad1700 1 --2- 172.21.15.67:0/2431678234 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd7006e750 0x7fbd7014c140 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fbd6000c520 tx=0x7fbd6000c8e0 comp rx=0 tx=0).stop 2023-10-14T10:36:09.202 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.199+0000 7fbd77ad1700 1 -- 172.21.15.67:0/2431678234 shutdown_connections 2023-10-14T10:36:09.202 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.199+0000 7fbd77ad1700 1 --2- 172.21.15.67:0/2431678234 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fbd5406af00 0x7fbd5406d3c0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:09.202 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.199+0000 7fbd77ad1700 1 --2- 172.21.15.67:0/2431678234 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbd700b54f0 0x7fbd701497c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:09.203 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.199+0000 7fbd77ad1700 1 --2- 172.21.15.67:0/2431678234 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd7006e750 0x7fbd7014c140 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:09.203 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.199+0000 7fbd77ad1700 1 --2- 172.21.15.67:0/2431678234 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fbd7006eb30 0x7fbd70150030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:09.203 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.199+0000 7fbd77ad1700 1 -- 172.21.15.67:0/2431678234 >> 172.21.15.67:0/2431678234 conn(0x7fbd700cc6f0 msgr2=0x7fbd700af6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:09.203 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.199+0000 7fbd77ad1700 1 -- 172.21.15.67:0/2431678234 shutdown_connections 2023-10-14T10:36:09.203 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.199+0000 7fbd77ad1700 1 -- 172.21.15.67:0/2431678234 wait complete. 2023-10-14T10:36:09.213 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-10-14T10:36:09.214 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-14T10:36:09.285 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-14T10:36:09.288 DEBUG:teuthology.orchestra.run.smithi072:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-14T10:36:09.339 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.338+0000 7faf5df2d700 1 Processor -- start 2023-10-14T10:36:09.340 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.338+0000 7faf5df2d700 1 -- start start 2023-10-14T10:36:09.342 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.338+0000 7faf5df2d700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7faf5815f1f0 0x7faf580cf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:09.342 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.338+0000 7faf5df2d700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf5815e7a0 con 0x7faf580cfc60 2023-10-14T10:36:09.342 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.338+0000 7faf5df2d700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf580d0090 con 0x7faf5815ab10 2023-10-14T10:36:09.342 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.338+0000 7faf5df2d700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf580b29a0 con 0x7faf5815f1f0 2023-10-14T10:36:09.343 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.339+0000 7faf56ffd700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7faf5815f1f0 0x7faf580cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:09.343 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.339+0000 7faf57fff700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7faf580cfc60 0x7faf580cbf40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:51694/0 (socket says 172.21.15.67:51694) 2023-10-14T10:36:09.343 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.339+0000 7faf57fff700 1 -- 172.21.15.67:0/3641301453 learned_addr learned my addr 172.21.15.67:0/3641301453 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:36:09.343 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.339+0000 7faf56ffd700 1 -- 172.21.15.67:0/3641301453 >> v1:172.21.15.72:6789/0 conn(0x7faf5815ab10 legacy=0x7faf580b1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:09.343 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.339+0000 7faf56ffd700 1 -- 172.21.15.67:0/3641301453 >> v1:172.21.15.67:6789/0 conn(0x7faf580cfc60 legacy=0x7faf580cbf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:09.344 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.339+0000 7faf567fc700 1 -- 172.21.15.67:0/3641301453 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1556896498 0 0) 0x7faf5815e7a0 con 0x7faf580cfc60 2023-10-14T10:36:09.344 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.339+0000 7faf56ffd700 1 -- 172.21.15.67:0/3641301453 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf580b2c80 con 0x7faf5815f1f0 2023-10-14T10:36:09.344 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.339+0000 7faf56ffd700 1 --2- 172.21.15.67:0/3641301453 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf5815f1f0 0x7faf580cf720 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7faf4c009820 tx=0x7faf4c009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c53d8756800dbd50 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:09.344 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.339+0000 7faf567fc700 1 -- 172.21.15.67:0/3641301453 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf4c005e30 con 0x7faf5815f1f0 2023-10-14T10:36:09.344 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.339+0000 7faf567fc700 1 -- 172.21.15.67:0/3641301453 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf4c004030 con 0x7faf5815f1f0 2023-10-14T10:36:09.345 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.339+0000 7faf567fc700 1 -- 172.21.15.67:0/3641301453 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf4c0043c0 con 0x7faf5815f1f0 2023-10-14T10:36:09.345 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.340+0000 7faf5df2d700 1 -- 172.21.15.67:0/3641301453 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf5815f1f0 msgr2=0x7faf580cf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:09.345 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.340+0000 7faf5df2d700 1 --2- 172.21.15.67:0/3641301453 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf5815f1f0 0x7faf580cf720 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7faf4c009820 tx=0x7faf4c009b30 comp rx=0 tx=0).stop 2023-10-14T10:36:09.345 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.340+0000 7faf5df2d700 1 -- 172.21.15.67:0/3641301453 shutdown_connections 2023-10-14T10:36:09.345 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.340+0000 7faf5df2d700 1 --2- 172.21.15.67:0/3641301453 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf5815f1f0 0x7faf580cf720 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:09.345 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.340+0000 7faf5df2d700 1 -- 172.21.15.67:0/3641301453 >> 172.21.15.67:0/3641301453 conn(0x7faf5806dae0 msgr2=0x7faf5806def0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:09.346 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.340+0000 7faf5df2d700 1 -- 172.21.15.67:0/3641301453 shutdown_connections 2023-10-14T10:36:09.346 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.340+0000 7faf5df2d700 1 -- 172.21.15.67:0/3641301453 wait complete. 2023-10-14T10:36:09.349 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.341+0000 7faf5df2d700 1 Processor -- start 2023-10-14T10:36:09.349 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.341+0000 7faf5df2d700 1 -- start start 2023-10-14T10:36:09.349 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.342+0000 7faf5df2d700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faf580cfc60 0x7faf581f7930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:09.349 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.342+0000 7faf5df2d700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7faf5815ab10 0x7faf580cd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:09.349 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.342+0000 7faf5df2d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf5815f1f0 0x7faf580cd6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:09.350 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.342+0000 7faf5df2d700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf581f7e70 con 0x7faf5815f1f0 2023-10-14T10:36:09.350 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.342+0000 7faf5df2d700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf580ad2a0 con 0x7faf580cfc60 2023-10-14T10:36:09.350 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.342+0000 7faf5df2d700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf581572b0 con 0x7faf5815ab10 2023-10-14T10:36:09.350 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.342+0000 7faf57fff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf5815f1f0 0x7faf580cd6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:09.351 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.342+0000 7faf57fff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf5815f1f0 0x7faf580cd6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43930/0 (socket says 172.21.15.67:43930) 2023-10-14T10:36:09.351 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.342+0000 7faf57fff700 1 -- 172.21.15.67:0/517514401 learned_addr learned my addr 172.21.15.67:0/517514401 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:09.351 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.342+0000 7faf577fe700 1 --2- 172.21.15.67:0/517514401 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faf580cfc60 0x7faf581f7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:09.351 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.342+0000 7faf56ffd700 1 --2- 172.21.15.67:0/517514401 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7faf5815ab10 0x7faf580cd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:09.351 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.342+0000 7faf57fff700 1 -- 172.21.15.67:0/517514401 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7faf5815ab10 msgr2=0x7faf580cd1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:09.352 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.342+0000 7faf57fff700 1 --2- 172.21.15.67:0/517514401 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7faf5815ab10 0x7faf580cd1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:09.352 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.342+0000 7faf57fff700 1 -- 172.21.15.67:0/517514401 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faf580cfc60 msgr2=0x7faf581f7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:09.352 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.342+0000 7faf57fff700 1 --2- 172.21.15.67:0/517514401 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faf580cfc60 0x7faf581f7930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:09.352 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.342+0000 7faf57fff700 1 -- 172.21.15.67:0/517514401 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf4c009430 con 0x7faf5815f1f0 2023-10-14T10:36:09.352 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.342+0000 7faf56ffd700 1 --2- 172.21.15.67:0/517514401 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7faf5815ab10 0x7faf580cd1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-14T10:36:09.352 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.342+0000 7faf57fff700 1 --2- 172.21.15.67:0/517514401 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf5815f1f0 0x7faf580cd6e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7faf5000cb90 tx=0x7faf5000cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:09.353 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.343+0000 7faf3ffff700 1 -- 172.21.15.67:0/517514401 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf5000aad0 con 0x7faf5815f1f0 2023-10-14T10:36:09.353 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.343+0000 7faf3ffff700 1 -- 172.21.15.67:0/517514401 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf5000ac30 con 0x7faf5815f1f0 2023-10-14T10:36:09.353 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.343+0000 7faf5df2d700 1 -- 172.21.15.67:0/517514401 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf581564a0 con 0x7faf5815f1f0 2023-10-14T10:36:09.353 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.343+0000 7faf3ffff700 1 -- 172.21.15.67:0/517514401 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf500070d0 con 0x7faf5815f1f0 2023-10-14T10:36:09.354 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.343+0000 7faf5df2d700 1 -- 172.21.15.67:0/517514401 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf58156c60 con 0x7faf5815f1f0 2023-10-14T10:36:09.354 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.343+0000 7faf5df2d700 1 -- 172.21.15.67:0/517514401 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf58156ab0 con 0x7faf5815f1f0 2023-10-14T10:36:09.354 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.344+0000 7faf3ffff700 1 -- 172.21.15.67:0/517514401 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7faf50007230 con 0x7faf5815f1f0 2023-10-14T10:36:09.354 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.344+0000 7faf3ffff700 1 --2- 172.21.15.67:0/517514401 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7faf4406af00 0x7faf4406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:09.354 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.345+0000 7faf3ffff700 1 -- 172.21.15.67:0/517514401 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faf500890d0 con 0x7faf5815f1f0 2023-10-14T10:36:09.355 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.345+0000 7faf577fe700 1 --2- 172.21.15.67:0/517514401 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7faf4406af00 0x7faf4406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:09.355 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.345+0000 7faf577fe700 1 --2- 172.21.15.67:0/517514401 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7faf4406af00 0x7faf4406d3c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7faf580c7060 tx=0x7faf48008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:09.355 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.349+0000 7faf3ffff700 1 -- 172.21.15.67:0/517514401 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7faf50054180 con 0x7faf5815f1f0 2023-10-14T10:36:09.549 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.547+0000 7faf5df2d700 1 -- 172.21.15.67:0/517514401 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faf5806e8d0 con 0x7faf5815f1f0 2023-10-14T10:36:09.550 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.548+0000 7faf3ffff700 1 -- 172.21.15.67:0/517514401 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16457 (secure 0 0 0) 0x7faf500577a0 con 0x7faf5815f1f0 2023-10-14T10:36:09.551 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-14T10:36:09.551 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":22,"fsid":"37f9ab46-3082-4b71-8b24-08d39897b008","created":"2023-10-14T10:35:46.067554+0000","modified":"2023-10-14T10:36:04.837296+0000","last_up_change":"2023-10-14T10:35:56.668142+0000","last_in_change":"2023-10-14T10:35:48.551490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-14T10:35:57.809566+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-14T10:36:01.667146+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-14T10:36:02.219700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2b59a07e-11ef-440c-b766-9f64ac885838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6801","nonce":1287255499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6803","nonce":1287255499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6807","nonce":1287255499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6805","nonce":1287255499}]},"public_addr":"172.21.15.67:6801/1287255499","cluster_addr":"172.21.15.67:6803/1287255499","heartbeat_back_addr":"172.21.15.67:6807/1287255499","heartbeat_front_addr":"172.21.15.67:6805/1287255499","state":["exists","up"]},{"osd":1,"uuid":"6f813bd6-fa0b-487d-a204-431eb1276a60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6818","nonce":2435538787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6822","nonce":2435538787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6830","nonce":2435538787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6826","nonce":2435538787}]},"public_addr":"172.21.15.67:6818/2435538787","cluster_addr":"172.21.15.67:6822/2435538787","heartbeat_back_addr":"172.21.15.67:6830/2435538787","heartbeat_front_addr":"172.21.15.67:6826/2435538787","state":["exists","up"]},{"osd":2,"uuid":"6af42c6a-1b59-4669-9c04-b6876788446d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6809","nonce":2827491791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6811","nonce":2827491791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6815","nonce":2827491791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6813","nonce":2827491791}]},"public_addr":"172.21.15.67:6809/2827491791","cluster_addr":"172.21.15.67:6811/2827491791","heartbeat_back_addr":"172.21.15.67:6815/2827491791","heartbeat_front_addr":"172.21.15.67:6813/2827491791","state":["exists","up"]},{"osd":3,"uuid":"1d889d76-a646-499d-bfff-ed5bae5434ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6819","nonce":1848001176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6821","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6823","nonce":1848001176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6831","nonce":1848001176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6825","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6827","nonce":1848001176}]},"public_addr":"172.21.15.67:6819/1848001176","cluster_addr":"172.21.15.67:6823/1848001176","heartbeat_back_addr":"172.21.15.67:6831/1848001176","heartbeat_front_addr":"172.21.15.67:6827/1848001176","state":["exists","up"]},{"osd":4,"uuid":"c3271dff-b437-4fdd-b13b-6e49a35473af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6804","nonce":1891230785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6808","nonce":1891230785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6815","nonce":1891230785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6811","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6813","nonce":1891230785}]},"public_addr":"172.21.15.72:6804/1891230785","cluster_addr":"172.21.15.72:6808/1891230785","heartbeat_back_addr":"172.21.15.72:6815/1891230785","heartbeat_front_addr":"172.21.15.72:6813/1891230785","state":["exists","up"]},{"osd":5,"uuid":"0337bb18-4b68-4eff-b528-a667275e6926","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6801","nonce":569686991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6803","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6805","nonce":569686991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6812","nonce":569686991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6807","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6809","nonce":569686991}]},"public_addr":"172.21.15.72:6801/569686991","cluster_addr":"172.21.15.72:6805/569686991","heartbeat_back_addr":"172.21.15.72:6812/569686991","heartbeat_front_addr":"172.21.15.72:6809/569686991","state":["exists","up"]},{"osd":6,"uuid":"8e92d9cf-0c8c-46eb-89a7-db2023ef1509","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6817","nonce":3767685934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6819","nonce":3767685934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6823","nonce":3767685934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6821","nonce":3767685934}]},"public_addr":"172.21.15.72:6817/3767685934","cluster_addr":"172.21.15.72:6819/3767685934","heartbeat_back_addr":"172.21.15.72:6823/3767685934","heartbeat_front_addr":"172.21.15.72:6821/3767685934","state":["exists","up"]},{"osd":7,"uuid":"3fd6aa7d-50a9-45c9-9561-5902a044a119","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6825","nonce":3567465926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6827","nonce":3567465926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6831","nonce":3567465926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6829","nonce":3567465926}]},"public_addr":"172.21.15.72:6825/3567465926","cluster_addr":"172.21.15.72:6827/3567465926","heartbeat_back_addr":"172.21.15.72:6831/3567465926","heartbeat_front_addr":"172.21.15.72:6829/3567465926","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.511466+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.795107+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.858674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.728895+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.794842+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.750064+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.732122+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.765185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-14T10:36:09.553 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.550+0000 7faf5df2d700 1 -- 172.21.15.67:0/517514401 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7faf4406af00 msgr2=0x7faf4406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:09.553 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.551+0000 7faf5df2d700 1 --2- 172.21.15.67:0/517514401 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7faf4406af00 0x7faf4406d3c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7faf580c7060 tx=0x7faf48008040 comp rx=0 tx=0).stop 2023-10-14T10:36:09.553 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.551+0000 7faf5df2d700 1 -- 172.21.15.67:0/517514401 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf5815f1f0 msgr2=0x7faf580cd6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:09.554 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.551+0000 7faf5df2d700 1 --2- 172.21.15.67:0/517514401 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf5815f1f0 0x7faf580cd6e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7faf5000cb90 tx=0x7faf5000cf50 comp rx=0 tx=0).stop 2023-10-14T10:36:09.554 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.551+0000 7faf5df2d700 1 -- 172.21.15.67:0/517514401 shutdown_connections 2023-10-14T10:36:09.554 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.551+0000 7faf5df2d700 1 --2- 172.21.15.67:0/517514401 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7faf4406af00 0x7faf4406d3c0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:09.554 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.551+0000 7faf5df2d700 1 --2- 172.21.15.67:0/517514401 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf5815f1f0 0x7faf580cd6e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:09.555 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.551+0000 7faf5df2d700 1 --2- 172.21.15.67:0/517514401 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faf580cfc60 0x7faf581f7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:09.555 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.551+0000 7faf5df2d700 1 --2- 172.21.15.67:0/517514401 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7faf5815ab10 0x7faf580cd1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:09.555 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.551+0000 7faf5df2d700 1 -- 172.21.15.67:0/517514401 >> 172.21.15.67:0/517514401 conn(0x7faf5806dae0 msgr2=0x7faf58158970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:09.555 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.551+0000 7faf5df2d700 1 -- 172.21.15.67:0/517514401 shutdown_connections 2023-10-14T10:36:09.555 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.551+0000 7faf5df2d700 1 -- 172.21.15.67:0/517514401 wait complete. 2023-10-14T10:36:09.566 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-10-14T10:36:09.566 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-10-14T10:36:09.567 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-10-14T10:36:09.567 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-10-14T10:36:09.568 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-10-14T10:36:09.568 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-10-14T10:36:09.569 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-10-14T10:36:09.569 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-10-14T10:36:09.820 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.816+0000 7f8ffc94f700 1 Processor -- start 2023-10-14T10:36:09.821 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.816+0000 7f8ffc94f700 1 -- start start 2023-10-14T10:36:09.821 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.817+0000 7f8ffc94f700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f8ff808a700 0x7f8ff80ca450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:09.821 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.817+0000 7f8ffc94f700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ff80ca990 con 0x7f8ff806d0c0 2023-10-14T10:36:09.822 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.817+0000 7f8ffc94f700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ff80cf250 con 0x7f8ff80ceae0 2023-10-14T10:36:09.822 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.817+0000 7f8ffc94f700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ff816ebd0 con 0x7f8ff808a700 2023-10-14T10:36:09.823 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.818+0000 7f2ca82cc700 1 Processor -- start 2023-10-14T10:36:09.823 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.818+0000 7f2ca82cc700 1 -- start start 2023-10-14T10:36:09.824 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.819+0000 7f2ca82cc700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f2ca00ceae0 0x7f2ca008a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:09.824 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.819+0000 7f2ca82cc700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f2ca006d0c0 0x7f2ca008a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:09.824 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.819+0000 7f2ca82cc700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ca00cf250 con 0x7f2ca016f060 2023-10-14T10:36:09.824 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.819+0000 7f2ca82cc700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ca016ebd0 con 0x7f2ca00ceae0 2023-10-14T10:36:09.825 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.819+0000 7f2ca82cc700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ca016e580 con 0x7f2ca006d0c0 2023-10-14T10:36:09.825 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.820+0000 7f8ff659c700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f8ff808a700 0x7f8ff80ca450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:09.825 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.820+0000 7f8ff659c700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f8ff808a700 0x7f8ff80ca450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.67:38726/0 (socket says 172.21.15.67:38726) 2023-10-14T10:36:09.826 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.820+0000 7f8ff659c700 1 -- 172.21.15.67:0/3594213276 learned_addr learned my addr 172.21.15.67:0/3594213276 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:09.826 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.820+0000 7f2ca6869700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f2ca006d0c0 0x7f2ca008a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:09.826 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.820+0000 7f2ca6869700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f2ca006d0c0 0x7f2ca008a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.67:38732/0 (socket says 172.21.15.67:38732) 2023-10-14T10:36:09.827 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.820+0000 7f2ca6869700 1 -- 172.21.15.67:0/457802424 learned_addr learned my addr 172.21.15.67:0/457802424 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:09.827 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.820+0000 7f8399ae7700 1 Processor -- start 2023-10-14T10:36:09.827 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.820+0000 7f8399ae7700 1 -- start start 2023-10-14T10:36:09.828 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.820+0000 7f39a4a7a700 1 Processor -- start 2023-10-14T10:36:09.828 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.820+0000 7f39a4a7a700 1 -- start start 2023-10-14T10:36:09.828 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.821+0000 7f8399ae7700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f83940c2840 0x7f8394166620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:09.829 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.821+0000 7f8399ae7700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f839406c9b0 0x7f839406ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:09.829 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.821+0000 7f8399ae7700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83940ca9f0 con 0x7f83940c9c50 2023-10-14T10:36:09.829 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.821+0000 7f8399ae7700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f839406d380 con 0x7f839406c9b0 2023-10-14T10:36:09.829 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.821+0000 7f8399ae7700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8394162490 con 0x7f83940c2840 2023-10-14T10:36:09.829 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.822+0000 7f8392ffd700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f83940c9c50 0x7f83940cde60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:51712/0 (socket says 172.21.15.67:51712) 2023-10-14T10:36:09.830 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.822+0000 7f8392ffd700 1 -- 172.21.15.67:0/1767673864 learned_addr learned my addr 172.21.15.67:0/1767673864 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:36:09.830 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.822+0000 7f83937fe700 1 --2- 172.21.15.67:0/1767673864 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f83940c2840 0x7f8394166620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:09.830 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.820+0000 7f8ff659c700 1 -- 172.21.15.67:0/3594213276 >> v1:172.21.15.72:6789/0 conn(0x7f8ff80ceae0 legacy=0x7f8ff806c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:09.831 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.820+0000 7f8ff659c700 1 -- 172.21.15.67:0/3594213276 >> v1:172.21.15.67:6789/0 conn(0x7f8ff806d0c0 legacy=0x7f8ff80ccfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:09.831 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.820+0000 7f8ff659c700 1 -- 172.21.15.67:0/3594213276 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ff80cef70 con 0x7f8ff808a700 2023-10-14T10:36:09.831 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.822+0000 7f39a4a7a700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f39a0072430 0x7f39a016edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:09.831 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.822+0000 7f39a4a7a700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f39a00ccfc0 0x7f39a00cd450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:09.832 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.822+0000 7f39a4a7a700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39a00ca5d0 con 0x7f39a006c9b0 2023-10-14T10:36:09.832 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.822+0000 7f39a4a7a700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39a00ceae0 con 0x7f39a00ccfc0 2023-10-14T10:36:09.832 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.822+0000 7f39a4a7a700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39a00cf280 con 0x7f39a0072430 2023-10-14T10:36:09.832 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.822+0000 7f83937fe700 1 -- 172.21.15.67:0/1767673864 >> v2:172.21.15.67:3300/0 conn(0x7f839406c9b0 msgr2=0x7f839406ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:09.833 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.822+0000 7f83937fe700 1 --2- 172.21.15.67:0/1767673864 >> v2:172.21.15.67:3300/0 conn(0x7f839406c9b0 0x7f839406ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:09.833 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.822+0000 7f83937fe700 1 -- 172.21.15.67:0/1767673864 >> v1:172.21.15.67:6789/0 conn(0x7f83940c9c50 legacy=0x7f83940cde60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:09.833 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.822+0000 7f83937fe700 1 -- 172.21.15.67:0/1767673864 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83940ca430 con 0x7f83940c2840 2023-10-14T10:36:09.833 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.822+0000 7f399e59c700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f39a006c9b0 0x7f39a006cd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:51718/0 (socket says 172.21.15.67:51718) 2023-10-14T10:36:09.834 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.822+0000 7f399e59c700 1 -- 172.21.15.67:0/4153296344 learned_addr learned my addr 172.21.15.67:0/4153296344 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:36:09.834 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.822+0000 7f83937fe700 1 --2- 172.21.15.67:0/1767673864 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f83940c2840 0x7f8394166620 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8388009840 tx=0x7f8388009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9c63e45c52ef00c server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:09.835 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.822+0000 7f399ed9d700 1 --2- 172.21.15.67:0/4153296344 >> v2:172.21.15.67:3300/0 conn(0x7f39a00ccfc0 0x7f39a00cd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:09.835 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.822+0000 7f399dd9b700 1 --2- 172.21.15.67:0/4153296344 >> v2:172.21.15.72:3300/0 conn(0x7f39a0072430 0x7f39a016edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:09.836 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.823+0000 7f399ed9d700 1 -- 172.21.15.67:0/4153296344 >> v2:172.21.15.72:3300/0 conn(0x7f39a0072430 msgr2=0x7f39a016edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:09.836 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.823+0000 7f399ed9d700 1 --2- 172.21.15.67:0/4153296344 >> v2:172.21.15.72:3300/0 conn(0x7f39a0072430 0x7f39a016edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:09.836 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.823+0000 7f399ed9d700 1 -- 172.21.15.67:0/4153296344 >> v1:172.21.15.67:6789/0 conn(0x7f39a006c9b0 legacy=0x7f39a006cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:09.836 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.823+0000 7f399ed9d700 1 -- 172.21.15.67:0/4153296344 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39a00cecc0 con 0x7f39a00ccfc0 2023-10-14T10:36:09.836 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.823+0000 7f399ed9d700 1 --2- 172.21.15.67:0/4153296344 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f39a00ccfc0 0x7f39a00cd450 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f39a00cb550 tx=0x7f399800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f9215eae172ff8f server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:09.837 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.823+0000 7f399d59a700 1 -- 172.21.15.67:0/4153296344 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f399800e070 con 0x7f39a00ccfc0 2023-10-14T10:36:09.837 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.823+0000 7f2ca6068700 1 --2- 172.21.15.67:0/457802424 >> v2:172.21.15.67:3300/0 conn(0x7f2ca00ceae0 0x7f2ca008a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:09.837 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.823+0000 7f399d59a700 1 -- 172.21.15.67:0/4153296344 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f399800bbd0 con 0x7f39a00ccfc0 2023-10-14T10:36:09.838 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.823+0000 7f399d59a700 1 -- 172.21.15.67:0/4153296344 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3998012430 con 0x7f39a00ccfc0 2023-10-14T10:36:09.838 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.823+0000 7f2ca6068700 1 -- 172.21.15.67:0/457802424 >> v2:172.21.15.72:3300/0 conn(0x7f2ca006d0c0 msgr2=0x7f2ca008a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:09.839 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.823+0000 7f2ca6068700 1 --2- 172.21.15.67:0/457802424 >> v2:172.21.15.72:3300/0 conn(0x7f2ca006d0c0 0x7f2ca008a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:09.839 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.823+0000 7f2ca6068700 1 -- 172.21.15.67:0/457802424 >> v1:172.21.15.72:6789/0 conn(0x7f2ca016f060 legacy=0x7f2ca006c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:09.839 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.823+0000 7f2ca6068700 1 -- 172.21.15.67:0/457802424 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ca016e860 con 0x7f2ca00ceae0 2023-10-14T10:36:09.839 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.824+0000 7f2ca6068700 1 --2- 172.21.15.67:0/457802424 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2ca00ceae0 0x7f2ca008a490 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2ca006d740 tx=0x7f2c9800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6ebd35e8d573da3 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:09.840 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.824+0000 7f8ff659c700 1 --2- 172.21.15.67:0/3594213276 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8ff808a700 0x7f8ff80ca450 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8ff0007cb0 tx=0x7f8ff000d4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e49ad5755023f4a server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:09.840 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.824+0000 7f39a4a7a700 1 -- 172.21.15.67:0/4153296344 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f39a00ccfc0 msgr2=0x7f39a00cd450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:09.841 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.824+0000 7f39a4a7a700 1 --2- 172.21.15.67:0/4153296344 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f39a00ccfc0 0x7f39a00cd450 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f39a00cb550 tx=0x7f399800b330 comp rx=0 tx=0).stop 2023-10-14T10:36:09.841 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.824+0000 7f39a4a7a700 1 -- 172.21.15.67:0/4153296344 shutdown_connections 2023-10-14T10:36:09.841 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.824+0000 7f39a4a7a700 1 --2- 172.21.15.67:0/4153296344 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f39a00ccfc0 0x7f39a00cd450 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:09.841 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.824+0000 7f39a4a7a700 1 --2- 172.21.15.67:0/4153296344 >> v2:172.21.15.72:3300/0 conn(0x7f39a0072430 0x7f39a016edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:09.841 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.824+0000 7f39a4a7a700 1 -- 172.21.15.67:0/4153296344 >> 172.21.15.67:0/4153296344 conn(0x7f39a00cf3f0 msgr2=0x7f39a00cf800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:09.842 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.824+0000 7f8ff559a700 1 -- 172.21.15.67:0/3594213276 <== mon.1 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff000dde0 con 0x7f8ff808a700 2023-10-14T10:36:09.842 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.824+0000 7f8ff559a700 1 -- 172.21.15.67:0/3594213276 <== mon.1 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ff0003a20 con 0x7f8ff808a700 2023-10-14T10:36:09.842 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.824+0000 7f8ff559a700 1 -- 172.21.15.67:0/3594213276 <== mon.1 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff0003db0 con 0x7f8ff808a700 2023-10-14T10:36:09.843 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.825+0000 7f8ffc94f700 1 -- 172.21.15.67:0/3594213276 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8ff808a700 msgr2=0x7f8ff80ca450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:09.843 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.825+0000 7f8ffc94f700 1 --2- 172.21.15.67:0/3594213276 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8ff808a700 0x7f8ff80ca450 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8ff0007cb0 tx=0x7f8ff000d4a0 comp rx=0 tx=0).stop 2023-10-14T10:36:09.843 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.824+0000 7f2ca5066700 1 -- 172.21.15.67:0/457802424 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c9800e070 con 0x7f2ca00ceae0 2023-10-14T10:36:09.844 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.824+0000 7f2ca5066700 1 -- 172.21.15.67:0/457802424 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c9800bae0 con 0x7f2ca00ceae0 2023-10-14T10:36:09.844 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.824+0000 7f2ca5066700 1 -- 172.21.15.67:0/457802424 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c9800be70 con 0x7f2ca00ceae0 2023-10-14T10:36:09.844 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.825+0000 7f2ca82cc700 1 -- 172.21.15.67:0/457802424 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2ca00ceae0 msgr2=0x7f2ca008a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:09.844 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.825+0000 7f2ca82cc700 1 --2- 172.21.15.67:0/457802424 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2ca00ceae0 0x7f2ca008a490 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2ca006d740 tx=0x7f2c9800b330 comp rx=0 tx=0).stop 2023-10-14T10:36:09.845 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.825+0000 7f39a4a7a700 1 -- 172.21.15.67:0/4153296344 shutdown_connections 2023-10-14T10:36:09.845 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.825+0000 7f83927fc700 1 -- 172.21.15.67:0/1767673864 <== mon.2 v2:172.21.15.72:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8388011830 con 0x7f83940c2840 2023-10-14T10:36:09.846 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.825+0000 7f2ca82cc700 1 -- 172.21.15.67:0/457802424 shutdown_connections 2023-10-14T10:36:09.846 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.825+0000 7f2ca82cc700 1 --2- 172.21.15.67:0/457802424 >> v2:172.21.15.72:3300/0 conn(0x7f2ca006d0c0 0x7f2ca008a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:09.846 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.825+0000 7f2ca82cc700 1 --2- 172.21.15.67:0/457802424 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2ca00ceae0 0x7f2ca008a490 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:09.847 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.825+0000 7f2ca82cc700 1 -- 172.21.15.67:0/457802424 >> 172.21.15.67:0/457802424 conn(0x7f2ca00cf3f0 msgr2=0x7f2ca00cf800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:09.847 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.825+0000 7f2ca82cc700 1 -- 172.21.15.67:0/457802424 shutdown_connections 2023-10-14T10:36:09.847 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.825+0000 7f2ca82cc700 1 -- 172.21.15.67:0/457802424 wait complete. 2023-10-14T10:36:09.847 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.826+0000 7f8ffc94f700 1 -- 172.21.15.67:0/3594213276 shutdown_connections 2023-10-14T10:36:09.848 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.826+0000 7f8ffc94f700 1 --2- 172.21.15.67:0/3594213276 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8ff808a700 0x7f8ff80ca450 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:09.848 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.826+0000 7f8ffc94f700 1 -- 172.21.15.67:0/3594213276 >> 172.21.15.67:0/3594213276 conn(0x7f8ff80cf3f0 msgr2=0x7f8ff80cf800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:09.849 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.825+0000 7f83927fc700 1 -- 172.21.15.67:0/1767673864 <== mon.2 v2:172.21.15.72:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8388011990 con 0x7f83940c2840 2023-10-14T10:36:09.849 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.825+0000 7f83927fc700 1 -- 172.21.15.67:0/1767673864 <== mon.2 v2:172.21.15.72:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8388011d20 con 0x7f83940c2840 2023-10-14T10:36:09.849 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.826+0000 7f8399ae7700 1 -- 172.21.15.67:0/1767673864 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f83940c2840 msgr2=0x7f8394166620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:09.849 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.826+0000 7f8399ae7700 1 --2- 172.21.15.67:0/1767673864 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f83940c2840 0x7f8394166620 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8388009840 tx=0x7f8388009b50 comp rx=0 tx=0).stop 2023-10-14T10:36:09.850 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.826+0000 7f8399ae7700 1 -- 172.21.15.67:0/1767673864 shutdown_connections 2023-10-14T10:36:09.850 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.826+0000 7f8399ae7700 1 --2- 172.21.15.67:0/1767673864 >> v2:172.21.15.67:3300/0 conn(0x7f839406c9b0 0x7f839406ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:09.850 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.826+0000 7f8399ae7700 1 --2- 172.21.15.67:0/1767673864 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f83940c2840 0x7f8394166620 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:09.850 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.826+0000 7f8399ae7700 1 -- 172.21.15.67:0/1767673864 >> 172.21.15.67:0/1767673864 conn(0x7f83940cf3f0 msgr2=0x7f83940cf800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:09.850 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.826+0000 7f8399ae7700 1 -- 172.21.15.67:0/1767673864 shutdown_connections 2023-10-14T10:36:09.851 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.826+0000 7f8399ae7700 1 -- 172.21.15.67:0/1767673864 wait complete. 2023-10-14T10:36:09.851 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.826+0000 7f39a4a7a700 1 -- 172.21.15.67:0/4153296344 wait complete. 2023-10-14T10:36:09.851 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.827+0000 7f1805303700 1 Processor -- start 2023-10-14T10:36:09.852 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.827+0000 7f2ca82cc700 1 Processor -- start 2023-10-14T10:36:09.852 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.827+0000 7f2ca82cc700 1 -- start start 2023-10-14T10:36:09.852 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.828+0000 7f2ca82cc700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2ca006d0c0 0x7f2ca00b46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:09.852 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.828+0000 7f2ca82cc700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f2ca016f060 0x7f2ca00a84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:09.853 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.828+0000 7f2ca82cc700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2ca0088120 0x7f2ca00a89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:09.853 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.828+0000 7f2ca82cc700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ca00b3600 con 0x7f2ca0088120 2023-10-14T10:36:09.853 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.828+0000 7f2ca82cc700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ca00889c0 con 0x7f2ca006d0c0 2023-10-14T10:36:09.853 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.828+0000 7f2ca82cc700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ca00b4db0 con 0x7f2ca016f060 2023-10-14T10:36:09.853 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.828+0000 7f2ca6869700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2ca0088120 0x7f2ca00a89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:09.854 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.828+0000 7f2ca6869700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2ca0088120 0x7f2ca00a89e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43964/0 (socket says 172.21.15.67:43964) 2023-10-14T10:36:09.854 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.828+0000 7f2ca6869700 1 -- 172.21.15.67:0/1142749056 learned_addr learned my addr 172.21.15.67:0/1142749056 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:09.854 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.828+0000 7f2ca6068700 1 --2- 172.21.15.67:0/1142749056 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2ca006d0c0 0x7f2ca00b46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:09.854 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.828+0000 7f2ca5867700 1 --2- 172.21.15.67:0/1142749056 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f2ca016f060 0x7f2ca00a84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:09.855 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.828+0000 7f2ca6068700 1 -- 172.21.15.67:0/1142749056 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f2ca016f060 msgr2=0x7f2ca00a84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:09.855 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.828+0000 7f2ca6068700 1 --2- 172.21.15.67:0/1142749056 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f2ca016f060 0x7f2ca00a84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:09.855 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.828+0000 7f2ca6068700 1 -- 172.21.15.67:0/1142749056 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2ca0088120 msgr2=0x7f2ca00a89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:09.855 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.829+0000 7f2ca6068700 1 --2- 172.21.15.67:0/1142749056 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2ca0088120 0x7f2ca00a89e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:09.856 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.829+0000 7f8ffc94f700 1 -- 172.21.15.67:0/3594213276 shutdown_connections 2023-10-14T10:36:09.856 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.829+0000 7f2ca6068700 1 -- 172.21.15.67:0/1142749056 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c98009a70 con 0x7f2ca006d0c0 2023-10-14T10:36:09.856 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.828+0000 7f39a4a7a700 1 Processor -- start 2023-10-14T10:36:09.857 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.827+0000 7f1805303700 1 -- start start 2023-10-14T10:36:09.857 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.829+0000 7f1805303700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f18000c27d0 0x7f18000ce000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:09.857 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.829+0000 7f1805303700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f18000b0e70 0x7f18000b12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:09.857 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.829+0000 7f1805303700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1800165e40 con 0x7f18000c9a40 2023-10-14T10:36:09.858 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.828+0000 7f8399ae7700 1 Processor -- start 2023-10-14T10:36:09.858 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.829+0000 7f1805303700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18000cdbe0 con 0x7f18000b0e70 2023-10-14T10:36:09.859 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.829+0000 7f1805303700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18000b1830 con 0x7f18000c27d0 2023-10-14T10:36:09.859 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.829+0000 7f39a4a7a700 1 -- start start 2023-10-14T10:36:09.859 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.829+0000 7f8399ae7700 1 -- start start 2023-10-14T10:36:09.860 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.829+0000 7f17fe7fc700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f18000b0e70 0x7f18000b12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:09.860 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.829+0000 7f17fe7fc700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f18000b0e70 0x7f18000b12f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.67:38758/0 (socket says 172.21.15.67:38758) 2023-10-14T10:36:09.861 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.829+0000 7f17fe7fc700 1 -- 172.21.15.67:0/723757319 learned_addr learned my addr 172.21.15.67:0/723757319 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:09.861 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.830+0000 7f17fdffb700 1 -- 172.21.15.67:0/723757319 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 685307440 0 0) 0x7f1800165e40 con 0x7f18000c9a40 2023-10-14T10:36:09.861 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.830+0000 7f17fdffb700 1 -- 172.21.15.67:0/723757319 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f17e80035b0 con 0x7f18000c9a40 2023-10-14T10:36:09.862 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.830+0000 7f39a4a7a700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f39a006c9b0 0x7f39a00c9300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:09.862 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.830+0000 7f39a4a7a700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f39a00ccfc0 0x7f39a0169ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:09.862 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.830+0000 7f39a4a7a700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f39a02123d0 0x7f39a0212860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:09.862 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.830+0000 7f39a4a7a700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39a00b40b0 con 0x7f39a00ccfc0 2023-10-14T10:36:09.862 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.830+0000 7f39a4a7a700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39a0213370 con 0x7f39a02123d0 2023-10-14T10:36:09.863 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.830+0000 7f39a4a7a700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39a00b4350 con 0x7f39a006c9b0 2023-10-14T10:36:09.863 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.830+0000 7f8399ae7700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f839406c9b0 0x7f8394156970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:09.863 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.830+0000 7f8399ae7700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f83940c2840 0x7f83940b2db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:09.863 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.830+0000 7f8399ae7700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f83940c9c50 0x7f83940c3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:09.864 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.830+0000 7f8399ae7700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83940c7000 con 0x7f839406c9b0 2023-10-14T10:36:09.864 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.830+0000 7f8399ae7700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83940c3fa0 con 0x7f83940c2840 2023-10-14T10:36:09.864 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.830+0000 7f8399ae7700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8394156eb0 con 0x7f83940c9c50 2023-10-14T10:36:09.864 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.830+0000 7f83937fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f839406c9b0 0x7f8394156970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:09.864 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.830+0000 7f83937fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f839406c9b0 0x7f8394156970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43972/0 (socket says 172.21.15.67:43972) 2023-10-14T10:36:09.865 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.830+0000 7f83937fe700 1 -- 172.21.15.67:0/2741570795 learned_addr learned my addr 172.21.15.67:0/2741570795 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:09.865 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.829+0000 7f8ffc94f700 1 -- 172.21.15.67:0/3594213276 wait complete. 2023-10-14T10:36:09.865 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.830+0000 7f399e59c700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f39a006c9b0 0x7f39a00c9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:09.866 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.830+0000 7f8393fff700 1 --2- 172.21.15.67:0/2741570795 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f83940c9c50 0x7f83940c3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:09.867 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.830+0000 7f399e59c700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f39a006c9b0 0x7f39a00c9300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3301/0 says I am v2:172.21.15.67:60258/0 (socket says 172.21.15.67:60258) 2023-10-14T10:36:09.867 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.830+0000 7f399e59c700 1 -- 172.21.15.67:0/289332590 learned_addr learned my addr 172.21.15.67:0/289332590 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:09.867 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.830+0000 7f399dd9b700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f39a00ccfc0 0x7f39a0169ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:09.867 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.831+0000 7f399e59c700 1 -- 172.21.15.67:0/289332590 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f39a02123d0 msgr2=0x7f39a0212860 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-14T10:36:09.868 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.831+0000 7f399e59c700 1 --2- 172.21.15.67:0/289332590 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f39a02123d0 0x7f39a0212860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:09.868 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.831+0000 7f399e59c700 1 -- 172.21.15.67:0/289332590 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f39a00ccfc0 msgr2=0x7f39a0169ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:09.868 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.831+0000 7f399e59c700 1 --2- 172.21.15.67:0/289332590 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f39a00ccfc0 0x7f39a0169ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:09.868 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.831+0000 7f399e59c700 1 -- 172.21.15.67:0/289332590 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3998009a70 con 0x7f39a006c9b0 2023-10-14T10:36:09.869 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.831+0000 7f399e59c700 1 --2- 172.21.15.67:0/289332590 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f39a006c9b0 0x7f39a00c9300 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f39900061a0 tx=0x7f399000b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:09.869 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.830+0000 7f17fdffb700 1 -- 172.21.15.67:0/723757319 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 96653096 0 0) 0x7f17e80035b0 con 0x7f18000c9a40 2023-10-14T10:36:09.869 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.830+0000 7f17fdffb700 1 -- 172.21.15.67:0/723757319 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f18000c27d0 msgr2=0x7f18000ce000 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-14T10:36:09.869 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.830+0000 7f17fdffb700 1 --2- 172.21.15.67:0/723757319 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f18000c27d0 0x7f18000ce000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:09.870 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.830+0000 7f17fdffb700 1 -- 172.21.15.67:0/723757319 >> v2:172.21.15.72:3300/0 conn(0x7f18000b0e70 msgr2=0x7f18000b12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:09.870 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.830+0000 7f17fdffb700 1 --2- 172.21.15.67:0/723757319 >> v2:172.21.15.72:3300/0 conn(0x7f18000b0e70 0x7f18000b12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:09.870 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.830+0000 7f17fdffb700 1 -- 172.21.15.67:0/723757319 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18000cd560 con 0x7f18000c9a40 2023-10-14T10:36:09.870 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.830+0000 7f17fdffb700 1 -- 172.21.15.67:0/723757319 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3180043755 0 0) 0x7f17f80019f0 con 0x7f18000c9a40 2023-10-14T10:36:09.870 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.830+0000 7f17fdffb700 1 -- 172.21.15.67:0/723757319 >> v1:172.21.15.67:6789/0 conn(0x7f18000c9a40 legacy=0x7f1800070a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:09.871 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.830+0000 7f17fdffb700 1 --2- 172.21.15.67:0/723757319 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f17e8003a00 0x7f17e8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:09.871 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.830+0000 7f17fdffb700 1 --2- 172.21.15.67:0/723757319 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f17e8004850 0x7f17e8004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:09.871 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.830+0000 7f17fdffb700 1 --2- 172.21.15.67:0/723757319 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f17e8005210 0x7f17e800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:09.871 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.830+0000 7f17fdffb700 1 -- 172.21.15.67:0/723757319 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18000cdbe0 con 0x7f17e8003a00 2023-10-14T10:36:09.871 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.830+0000 7f17fdffb700 1 -- 172.21.15.67:0/723757319 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18000b1830 con 0x7f17e8004850 2023-10-14T10:36:09.872 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.830+0000 7f17fdffb700 1 -- 172.21.15.67:0/723757319 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17e800c060 con 0x7f17e8005210 2023-10-14T10:36:09.872 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.831+0000 7f17feffd700 1 --2- 172.21.15.67:0/723757319 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f17e8003a00 0x7f17e8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:09.872 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.831+0000 7f83937fe700 1 -- 172.21.15.67:0/2741570795 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f83940c9c50 msgr2=0x7f83940c3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:09.872 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.831+0000 7f83937fe700 1 --2- 172.21.15.67:0/2741570795 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f83940c9c50 0x7f83940c3670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:09.873 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.831+0000 7f83937fe700 1 -- 172.21.15.67:0/2741570795 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f83940c2840 msgr2=0x7f83940b2db0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-14T10:36:09.873 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.831+0000 7f83937fe700 1 --2- 172.21.15.67:0/2741570795 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f83940c2840 0x7f83940b2db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:09.873 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.831+0000 7f83937fe700 1 -- 172.21.15.67:0/2741570795 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83880094a0 con 0x7f839406c9b0 2023-10-14T10:36:09.874 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.831+0000 7f8ffc94f700 1 Processor -- start 2023-10-14T10:36:09.874 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.831+0000 7f83937fe700 1 --2- 172.21.15.67:0/2741570795 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f839406c9b0 0x7f8394156970 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8388004270 tx=0x7f8388004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:09.875 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.831+0000 7f2ca6869700 1 --2- 172.21.15.67:0/1142749056 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2ca0088120 0x7f2ca00a89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-14T10:36:09.875 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.831+0000 7f17feffd700 1 -- 172.21.15.67:0/723757319 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f17e8005210 msgr2=0x7f17e800ba10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-14T10:36:09.875 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.831+0000 7f17feffd700 1 --2- 172.21.15.67:0/723757319 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f17e8005210 0x7f17e800ba10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:09.876 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.831+0000 7f17feffd700 1 -- 172.21.15.67:0/723757319 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f17e8004850 msgr2=0x7f17e8004cd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-14T10:36:09.876 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.831+0000 7f17feffd700 1 --2- 172.21.15.67:0/723757319 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f17e8004850 0x7f17e8004cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:09.876 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.831+0000 7f17feffd700 1 -- 172.21.15.67:0/723757319 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18000cd560 con 0x7f17e8003a00 2023-10-14T10:36:09.876 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.831+0000 7f2ca6068700 1 --2- 172.21.15.67:0/1142749056 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2ca006d0c0 0x7f2ca00b46b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2c98015000 tx=0x7f2c98004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:09.877 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.831+0000 7f398effd700 1 -- 172.21.15.67:0/289332590 <== mon.2 v2:172.21.15.72:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f399000f8d0 con 0x7f39a006c9b0 2023-10-14T10:36:09.877 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.831+0000 7f837bfff700 1 -- 172.21.15.67:0/2741570795 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8388003ab0 con 0x7f839406c9b0 2023-10-14T10:36:09.878 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.831+0000 7f398effd700 1 -- 172.21.15.67:0/289332590 <== mon.2 v2:172.21.15.72:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f399000fa30 con 0x7f39a006c9b0 2023-10-14T10:36:09.878 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.831+0000 7f398effd700 1 -- 172.21.15.67:0/289332590 <== mon.2 v2:172.21.15.72:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f399000fdc0 con 0x7f39a006c9b0 2023-10-14T10:36:09.878 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.832+0000 7f2c96ffd700 1 -- 172.21.15.67:0/1142749056 <== mon.1 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c98004680 con 0x7f2ca006d0c0 2023-10-14T10:36:09.879 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.832+0000 7f39a4a7a700 1 -- 172.21.15.67:0/289332590 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39a00b4630 con 0x7f39a006c9b0 2023-10-14T10:36:09.879 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.832+0000 7f39a4a7a700 1 -- 172.21.15.67:0/289332590 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39a00c9990 con 0x7f39a006c9b0 2023-10-14T10:36:09.879 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.832+0000 7f39a4a7a700 1 -- 172.21.15.67:0/289332590 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f39a00ca260 con 0x7f39a006c9b0 2023-10-14T10:36:09.880 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.832+0000 7f17feffd700 1 --2- 172.21.15.67:0/723757319 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f17e8003a00 0x7f17e8003e20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f180006eab0 tx=0x7f17e8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4051ca4ef22beec3 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:09.880 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.832+0000 7f17fdffb700 1 -- 172.21.15.67:0/723757319 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17f0011620 con 0x7f17e8003a00 2023-10-14T10:36:09.880 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.832+0000 7f17fdffb700 1 -- 172.21.15.67:0/723757319 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17f0011780 con 0x7f17e8003a00 2023-10-14T10:36:09.880 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.832+0000 7f17fdffb700 1 -- 172.21.15.67:0/723757319 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17f0011b30 con 0x7f17e8003a00 2023-10-14T10:36:09.881 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.831+0000 7f837bfff700 1 -- 172.21.15.67:0/2741570795 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8388003c10 con 0x7f839406c9b0 2023-10-14T10:36:09.881 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.832+0000 7f837bfff700 1 -- 172.21.15.67:0/2741570795 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8388004570 con 0x7f839406c9b0 2023-10-14T10:36:09.882 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.832+0000 7f2c96ffd700 1 -- 172.21.15.67:0/1142749056 <== mon.1 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c98004800 con 0x7f2ca006d0c0 2023-10-14T10:36:09.882 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.832+0000 7f2c96ffd700 1 -- 172.21.15.67:0/1142749056 <== mon.1 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c98004970 con 0x7f2ca006d0c0 2023-10-14T10:36:09.882 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.833+0000 7f2ca82cc700 1 -- 172.21.15.67:0/1142749056 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ca00b3810 con 0x7f2ca006d0c0 2023-10-14T10:36:09.883 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.833+0000 7f2ca82cc700 1 -- 172.21.15.67:0/1142749056 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ca00b3dd0 con 0x7f2ca006d0c0 2023-10-14T10:36:09.883 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.834+0000 7f398effd700 1 -- 172.21.15.67:0/289332590 <== mon.2 v2:172.21.15.72:3301/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f399000a070 con 0x7f39a006c9b0 2023-10-14T10:36:09.883 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.834+0000 7f398effd700 1 --2- 172.21.15.67:0/289332590 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f398006af80 0x7f398006d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:09.883 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.835+0000 7f398effd700 1 -- 172.21.15.67:0/289332590 <== mon.2 v2:172.21.15.72:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3990089cf0 con 0x7f39a006c9b0 2023-10-14T10:36:09.884 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.835+0000 7f399dd9b700 1 --2- 172.21.15.67:0/289332590 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f398006af80 0x7f398006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:09.884 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.835+0000 7f398effd700 1 --2- 172.21.15.67:0/289332590 >> [v2:172.21.15.67:6800/1287255499,v1:172.21.15.67:6801/1287255499] conn(0x7f39800731a0 0x7f39800755c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:09.884 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.835+0000 7f398effd700 1 -- 172.21.15.67:0/289332590 --> [v2:172.21.15.67:6800/1287255499,v1:172.21.15.67:6801/1287255499] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3980075c70 con 0x7f39800731a0 2023-10-14T10:36:09.884 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.835+0000 7f398effd700 1 -- 172.21.15.67:0/289332590 <== mon.2 v2:172.21.15.72:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3990033ea0 con 0x7f39a006c9b0 2023-10-14T10:36:09.884 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.835+0000 7f399ed9d700 1 --2- 172.21.15.67:0/289332590 >> [v2:172.21.15.67:6800/1287255499,v1:172.21.15.67:6801/1287255499] conn(0x7f39800731a0 0x7f39800755c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:09.885 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.836+0000 7f399ed9d700 1 --2- 172.21.15.67:0/289332590 >> [v2:172.21.15.67:6800/1287255499,v1:172.21.15.67:6801/1287255499] conn(0x7f39800731a0 0x7f39800755c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:09.885 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.833+0000 7f1805303700 1 -- 172.21.15.67:0/723757319 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f17e8003a00 msgr2=0x7f17e8003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:09.885 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.833+0000 7f1805303700 1 --2- 172.21.15.67:0/723757319 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f17e8003a00 0x7f17e8003e20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f180006eab0 tx=0x7f17e8005880 comp rx=0 tx=0).stop 2023-10-14T10:36:09.885 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.834+0000 7f17ff7fe700 1 -- 172.21.15.67:0/723757319 reap_dead start 2023-10-14T10:36:09.886 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.834+0000 7f1805303700 1 -- 172.21.15.67:0/723757319 shutdown_connections 2023-10-14T10:36:09.886 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.834+0000 7f1805303700 1 -- 172.21.15.67:0/723757319 >> 172.21.15.67:0/723757319 conn(0x7f180006afb0 msgr2=0x7f18000cbe70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:09.886 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.834+0000 7f1805303700 1 -- 172.21.15.67:0/723757319 shutdown_connections 2023-10-14T10:36:09.886 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.834+0000 7f1805303700 1 -- 172.21.15.67:0/723757319 wait complete. 2023-10-14T10:36:09.887 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.833+0000 7f8399ae7700 1 -- 172.21.15.67:0/2741570795 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83940c6660 con 0x7f839406c9b0 2023-10-14T10:36:09.887 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.833+0000 7f8399ae7700 1 -- 172.21.15.67:0/2741570795 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83940c6c20 con 0x7f839406c9b0 2023-10-14T10:36:09.887 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.834+0000 7f8399ae7700 1 -- 172.21.15.67:0/2741570795 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f83940cd660 con 0x7f839406c9b0 2023-10-14T10:36:09.888 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.836+0000 7f837bfff700 1 -- 172.21.15.67:0/2741570795 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f838801a420 con 0x7f839406c9b0 2023-10-14T10:36:09.888 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.831+0000 7f8ffc94f700 1 -- start start 2023-10-14T10:36:09.888 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.833+0000 7f8ffc94f700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f8ff806d0c0 0x7f8ff82045a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:09.888 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.833+0000 7f8ffc94f700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ff808a700 0x7f8ff80b11d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:09.889 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.833+0000 7f8ffc94f700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8ff816c970 0x7f8ff816cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:09.889 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.833+0000 7f8ffc94f700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff80c7510 con 0x7f8ff808a700 2023-10-14T10:36:09.889 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.833+0000 7f8ffc94f700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff816d330 con 0x7f8ff816c970 2023-10-14T10:36:09.889 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.833+0000 7f8ffc94f700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff80c7760 con 0x7f8ff806d0c0 2023-10-14T10:36:09.890 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.833+0000 7f8ff659c700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f8ff806d0c0 0x7f8ff82045a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:09.890 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.833+0000 7f8ff659c700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f8ff806d0c0 0x7f8ff82045a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3301/0 says I am v2:172.21.15.67:60264/0 (socket says 172.21.15.67:60264) 2023-10-14T10:36:09.890 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.833+0000 7f8ff659c700 1 -- 172.21.15.67:0/3721919745 learned_addr learned my addr 172.21.15.67:0/3721919745 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:09.890 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.833+0000 7f8ff6d9d700 1 --2- 172.21.15.67:0/3721919745 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8ff816c970 0x7f8ff816cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:09.890 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.833+0000 7f8ff5d9b700 1 --2- 172.21.15.67:0/3721919745 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ff808a700 0x7f8ff80b11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:09.891 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.835+0000 7f8ff5d9b700 1 -- 172.21.15.67:0/3721919745 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f8ff806d0c0 msgr2=0x7f8ff82045a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:09.891 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.835+0000 7f8ff5d9b700 1 --2- 172.21.15.67:0/3721919745 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f8ff806d0c0 0x7f8ff82045a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:09.891 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.835+0000 7f8ff5d9b700 1 -- 172.21.15.67:0/3721919745 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8ff816c970 msgr2=0x7f8ff816cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:09.891 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.835+0000 7f8ff5d9b700 1 --2- 172.21.15.67:0/3721919745 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8ff816c970 0x7f8ff816cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:09.891 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.835+0000 7f8ff5d9b700 1 -- 172.21.15.67:0/3721919745 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ff0007910 con 0x7f8ff808a700 2023-10-14T10:36:09.892 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.835+0000 7f8ff5d9b700 1 --2- 172.21.15.67:0/3721919745 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ff808a700 0x7f8ff80b11d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8fe8000f60 tx=0x7f8fe800c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:09.892 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.835+0000 7f2c96ffd700 1 -- 172.21.15.67:0/1142749056 <== mon.1 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f2c98012430 con 0x7f2ca006d0c0 2023-10-14T10:36:09.892 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.835+0000 7f2c96ffd700 1 --2- 172.21.15.67:0/1142749056 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f2c8406af00 0x7f2c8406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:09.892 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.836+0000 7f2c96ffd700 1 -- 172.21.15.67:0/1142749056 <== mon.1 v2:172.21.15.72:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2c9808ae30 con 0x7f2ca006d0c0 2023-10-14T10:36:09.893 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.836+0000 7f2ca5867700 1 --2- 172.21.15.67:0/1142749056 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f2c8406af00 0x7f2c8406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:09.893 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.836+0000 7f2ca5867700 1 --2- 172.21.15.67:0/1142749056 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f2c8406af00 0x7f2c8406d3c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2ca00c6060 tx=0x7f2c9c0090e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:09.893 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.836+0000 7f1805303700 1 Processor -- start 2023-10-14T10:36:09.894 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.837+0000 7f398effd700 1 -- 172.21.15.67:0/289332590 <== osd.0 v2:172.21.15.67:6800/1287255499 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7f3980075c70 con 0x7f39800731a0 2023-10-14T10:36:09.894 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.837+0000 7f399dd9b700 1 --2- 172.21.15.67:0/289332590 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f398006af80 0x7f398006d440 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f39a00ae420 tx=0x7f3994008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:09.894 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.837+0000 7f1805303700 1 -- start start 2023-10-14T10:36:09.895 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.837+0000 7f837bfff700 1 --2- 172.21.15.67:0/2741570795 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f838006af50 0x7f838006d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:09.895 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.837+0000 7f837bfff700 1 -- 172.21.15.67:0/2741570795 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f838808ac30 con 0x7f839406c9b0 2023-10-14T10:36:09.896 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.837+0000 7f837bfff700 1 --2- 172.21.15.67:0/2741570795 >> [v2:172.21.15.72:6802/1891230785,v1:172.21.15.72:6804/1891230785] conn(0x7f83800730f0 0x7f8380075510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:09.896 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.837+0000 7f837bfff700 1 -- 172.21.15.67:0/2741570795 --> [v2:172.21.15.72:6802/1891230785,v1:172.21.15.72:6804/1891230785] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8380075bc0 con 0x7f83800730f0 2023-10-14T10:36:09.896 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.837+0000 7f8fe6ffd700 1 -- 172.21.15.67:0/3721919745 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe80052d0 con 0x7f8ff808a700 2023-10-14T10:36:09.896 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.837+0000 7f8fe6ffd700 1 -- 172.21.15.67:0/3721919745 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fe8005430 con 0x7f8ff808a700 2023-10-14T10:36:09.897 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.837+0000 7f8fe6ffd700 1 -- 172.21.15.67:0/3721919745 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe80038c0 con 0x7f8ff808a700 2023-10-14T10:36:09.897 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.838+0000 7f8392ffd700 1 --2- 172.21.15.67:0/2741570795 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f838006af50 0x7f838006d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:09.898 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.837+0000 7f8ffc94f700 1 -- 172.21.15.67:0/3721919745 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ff80c7a40 con 0x7f8ff808a700 2023-10-14T10:36:09.898 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.837+0000 7f8ffc94f700 1 -- 172.21.15.67:0/3721919745 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ff80c8000 con 0x7f8ff808a700 2023-10-14T10:36:09.898 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.838+0000 7f1805303700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f18000b02b0 0x7f18000b2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:09.899 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.838+0000 7f1805303700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f18000b0690 0x7f18000b3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:09.899 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.838+0000 7f1805303700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f180014b190 0x7f180014b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:09.899 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.838+0000 7f1805303700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18000b4dc0 con 0x7f180014b190 2023-10-14T10:36:09.899 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.838+0000 7f1805303700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f180014bb50 con 0x7f18000b0690 2023-10-14T10:36:09.900 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.838+0000 7f1805303700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18000b4520 con 0x7f18000b02b0 2023-10-14T10:36:09.900 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.838+0000 7f17feffd700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f18000b02b0 0x7f18000b2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:09.900 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.838+0000 7f17feffd700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f18000b02b0 0x7f18000b2ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3301/0 says I am v2:172.21.15.67:60280/0 (socket says 172.21.15.67:60280) 2023-10-14T10:36:09.900 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.838+0000 7f17feffd700 1 -- 172.21.15.67:0/2866748129 learned_addr learned my addr 172.21.15.67:0/2866748129 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:09.901 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.838+0000 7f8392ffd700 1 --2- 172.21.15.67:0/2741570795 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f838006af50 0x7f838006d410 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f838c0012e0 tx=0x7f838c0015b0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:09.901 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.838+0000 7f837bfff700 1 -- 172.21.15.67:0/2741570795 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8388055d50 con 0x7f839406c9b0 2023-10-14T10:36:09.901 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.838+0000 7f8393fff700 1 --2- 172.21.15.67:0/2741570795 >> [v2:172.21.15.72:6802/1891230785,v1:172.21.15.72:6804/1891230785] conn(0x7f83800730f0 0x7f8380075510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:09.902 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.838+0000 7f8ffc94f700 1 -- 172.21.15.67:0/3721919745 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8ff816e860 con 0x7f8ff808a700 2023-10-14T10:36:09.902 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.838+0000 7f17feffd700 1 -- 172.21.15.67:0/2866748129 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f18000b0690 msgr2=0x7f18000b3b80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-14T10:36:09.902 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.838+0000 7f17feffd700 1 --2- 172.21.15.67:0/2866748129 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f18000b0690 0x7f18000b3b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:09.903 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.838+0000 7f17ff7fe700 1 --2- 172.21.15.67:0/2866748129 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f180014b190 0x7f180014b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:09.903 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.838+0000 7f17feffd700 1 -- 172.21.15.67:0/2866748129 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f180014b190 msgr2=0x7f180014b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:09.903 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.838+0000 7f17feffd700 1 --2- 172.21.15.67:0/2866748129 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f180014b190 0x7f180014b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:09.903 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.838+0000 7f17feffd700 1 -- 172.21.15.67:0/2866748129 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18000b4800 con 0x7f18000b02b0 2023-10-14T10:36:09.903 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.839+0000 7f17ff7fe700 1 --2- 172.21.15.67:0/2866748129 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f180014b190 0x7f180014b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-14T10:36:09.904 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.839+0000 7f8393fff700 1 --2- 172.21.15.67:0/2741570795 >> [v2:172.21.15.72:6802/1891230785,v1:172.21.15.72:6804/1891230785] conn(0x7f83800730f0 0x7f8380075510 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:09.904 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.840+0000 7f837bfff700 1 -- 172.21.15.67:0/2741570795 <== osd.4 v2:172.21.15.72:6802/1891230785 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7f8380075bc0 con 0x7f83800730f0 2023-10-14T10:36:09.905 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.840+0000 7f8fe6ffd700 1 -- 172.21.15.67:0/3721919745 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f8fe8003c30 con 0x7f8ff808a700 2023-10-14T10:36:09.905 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.840+0000 7f8fe6ffd700 1 --2- 172.21.15.67:0/3721919745 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f8fd806afd0 0x7f8fd806d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:09.905 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.840+0000 7f8fe6ffd700 1 -- 172.21.15.67:0/3721919745 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8fe8088f10 con 0x7f8ff808a700 2023-10-14T10:36:09.906 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.839+0000 7f2ca82cc700 1 --2- 172.21.15.67:0/1142749056 >> [v2:172.21.15.72:6816/3767685934,v1:172.21.15.72:6817/3767685934] conn(0x7f2ca00ceae0 0x7f2ca00612f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:09.906 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.839+0000 7f2ca82cc700 1 -- 172.21.15.67:0/1142749056 --> [v2:172.21.15.72:6816/3767685934,v1:172.21.15.72:6817/3767685934] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2ca00cf250 con 0x7f2ca00ceae0 2023-10-14T10:36:09.906 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.840+0000 7f2ca6869700 1 --2- 172.21.15.67:0/1142749056 >> [v2:172.21.15.72:6816/3767685934,v1:172.21.15.72:6817/3767685934] conn(0x7f2ca00ceae0 0x7f2ca00612f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:09.906 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.840+0000 7f2ca6869700 1 --2- 172.21.15.67:0/1142749056 >> [v2:172.21.15.72:6816/3767685934,v1:172.21.15.72:6817/3767685934] conn(0x7f2ca00ceae0 0x7f2ca00612f0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:09.907 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.839+0000 7f17feffd700 1 --2- 172.21.15.67:0/2866748129 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f18000b02b0 0x7f18000b2ca0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f17f0011b50 tx=0x7f17f0004790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:09.907 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.839+0000 7f17ef7fe700 1 -- 172.21.15.67:0/2866748129 <== mon.2 v2:172.21.15.72:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17f001a830 con 0x7f18000b02b0 2023-10-14T10:36:09.907 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.841+0000 7f8ff659c700 1 --2- 172.21.15.67:0/3721919745 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f8fd806afd0 0x7f8fd806d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:09.908 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.839+0000 7f1805303700 1 -- 172.21.15.67:0/2866748129 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18000b4ae0 con 0x7f18000b02b0 2023-10-14T10:36:09.908 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.839+0000 7f1805303700 1 -- 172.21.15.67:0/2866748129 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18000b40c0 con 0x7f18000b02b0 2023-10-14T10:36:09.909 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.839+0000 7f17ef7fe700 1 -- 172.21.15.67:0/2866748129 <== mon.2 v2:172.21.15.72:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17f0011e90 con 0x7f18000b02b0 2023-10-14T10:36:09.909 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.839+0000 7f17ef7fe700 1 -- 172.21.15.67:0/2866748129 <== mon.2 v2:172.21.15.72:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17f001abf0 con 0x7f18000b02b0 2023-10-14T10:36:09.909 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.840+0000 7f1805303700 1 -- 172.21.15.67:0/2866748129 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f18000ca620 con 0x7f18000b02b0 2023-10-14T10:36:09.909 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.841+0000 7f8fe6ffd700 1 --2- 172.21.15.67:0/3721919745 >> [v2:172.21.15.72:6800/569686991,v1:172.21.15.72:6801/569686991] conn(0x7f8fd80731f0 0x7f8fd8075610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:09.910 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.841+0000 7f8fe6ffd700 1 -- 172.21.15.67:0/3721919745 --> [v2:172.21.15.72:6800/569686991,v1:172.21.15.72:6801/569686991] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8fd8075cc0 con 0x7f8fd80731f0 2023-10-14T10:36:09.910 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.841+0000 7f8fe6ffd700 1 -- 172.21.15.67:0/3721919745 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8fe80892c0 con 0x7f8ff808a700 2023-10-14T10:36:09.910 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.841+0000 7fc42cde9700 1 Processor -- start 2023-10-14T10:36:09.911 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.841+0000 7fc42cde9700 1 -- start start 2023-10-14T10:36:09.911 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.841+0000 7f17ef7fe700 1 -- 172.21.15.67:0/2866748129 <== mon.2 v2:172.21.15.72:3301/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f17f001ad50 con 0x7f18000b02b0 2023-10-14T10:36:09.912 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.842+0000 7f17ef7fe700 1 --2- 172.21.15.67:0/2866748129 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f17e406aed0 0x7f17e406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:09.912 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.842+0000 7f17ef7fe700 1 -- 172.21.15.67:0/2866748129 <== mon.2 v2:172.21.15.72:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f17f008a660 con 0x7f18000b02b0 2023-10-14T10:36:09.912 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.842+0000 7fc42cde9700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc4200e1070 0x7fc4200387b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:09.913 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.842+0000 7f17ef7fe700 1 --2- 172.21.15.67:0/2866748129 >> [v2:172.21.15.67:6817/1848001176,v1:172.21.15.67:6819/1848001176] conn(0x7f17e4073070 0x7f17e4075490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:09.913 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.842+0000 7f17ef7fe700 1 -- 172.21.15.67:0/2866748129 --> [v2:172.21.15.67:6817/1848001176,v1:172.21.15.67:6819/1848001176] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f17e4075b40 con 0x7f17e4073070 2023-10-14T10:36:09.913 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.842+0000 7f17ef7fe700 1 -- 172.21.15.67:0/2866748129 <== mon.2 v2:172.21.15.72:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f17f008cd20 con 0x7f18000b02b0 2023-10-14T10:36:09.913 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.842+0000 7f17ff7fe700 1 --2- 172.21.15.67:0/2866748129 >> [v2:172.21.15.67:6817/1848001176,v1:172.21.15.67:6819/1848001176] conn(0x7f17e4073070 0x7f17e4075490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:09.914 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.842+0000 7f8ff659c700 1 --2- 172.21.15.67:0/3721919745 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f8fd806afd0 0x7f8fd806d490 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8ff80b1ea0 tx=0x7f8ff0012000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:09.915 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.842+0000 7fc42cde9700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc420033f50 con 0x7fc420033720 2023-10-14T10:36:09.915 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.842+0000 7fc42cde9700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc42004f930 con 0x7fc42003a7c0 2023-10-14T10:36:09.915 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.842+0000 7fc42cde9700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc420033c70 con 0x7fc4200e1070 2023-10-14T10:36:09.915 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.842+0000 7fc426ffd700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fc420033720 0x7fc42003a0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:51780/0 (socket says 172.21.15.67:51780) 2023-10-14T10:36:09.915 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.842+0000 7fc426ffd700 1 -- 172.21.15.67:0/2423751001 learned_addr learned my addr 172.21.15.67:0/2423751001 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:36:09.918 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.843+0000 7f17ff7fe700 1 --2- 172.21.15.67:0/2866748129 >> [v2:172.21.15.67:6817/1848001176,v1:172.21.15.67:6819/1848001176] conn(0x7f17e4073070 0x7f17e4075490 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:09.919 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.843+0000 7fc4277fe700 1 --2- 172.21.15.67:0/2423751001 >> v2:172.21.15.67:3300/0 conn(0x7fc4200e1070 0x7fc4200387b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:09.919 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.843+0000 7fc4267fc700 1 -- 172.21.15.67:0/2423751001 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2468595261 0 0) 0x7fc420033f50 con 0x7fc420033720 2023-10-14T10:36:09.920 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.843+0000 7fc4267fc700 1 -- 172.21.15.67:0/2423751001 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4180079c0 con 0x7fc420033720 2023-10-14T10:36:09.920 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.843+0000 7fc4267fc700 1 -- 172.21.15.67:0/2423751001 <== mon.1 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2495889110 0 0) 0x7fc42004f930 con 0x7fc42003a7c0 2023-10-14T10:36:09.920 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.843+0000 7fc4267fc700 1 -- 172.21.15.67:0/2423751001 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc420033f50 con 0x7fc42003a7c0 2023-10-14T10:36:09.920 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.843+0000 7f8ff6d9d700 1 --2- 172.21.15.67:0/3721919745 >> [v2:172.21.15.72:6800/569686991,v1:172.21.15.72:6801/569686991] conn(0x7f8fd80731f0 0x7f8fd8075610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:09.921 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.843+0000 7fc4267fc700 1 -- 172.21.15.67:0/2423751001 <== mon.1 v1:172.21.15.72:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 363957481 0 0) 0x7fc420033f50 con 0x7fc42003a7c0 2023-10-14T10:36:09.921 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.844+0000 7fc4267fc700 1 -- 172.21.15.67:0/2423751001 >> v2:172.21.15.67:3300/0 conn(0x7fc4200e1070 msgr2=0x7fc4200387b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:09.921 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.844+0000 7fc4267fc700 1 --2- 172.21.15.67:0/2423751001 >> v2:172.21.15.67:3300/0 conn(0x7fc4200e1070 0x7fc4200387b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:09.921 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.844+0000 7fc4267fc700 1 -- 172.21.15.67:0/2423751001 >> v1:172.21.15.67:6789/0 conn(0x7fc420033720 legacy=0x7fc42003a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:09.922 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.844+0000 7fc4267fc700 1 -- 172.21.15.67:0/2423751001 --> v1:172.21.15.72:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc42004ec90 con 0x7fc42003a7c0 2023-10-14T10:36:09.922 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.844+0000 7fc4267fc700 1 -- 172.21.15.67:0/2423751001 <== mon.1 v1:172.21.15.72:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3180043755 0 0) 0x7fc42804f400 con 0x7fc42003a7c0 2023-10-14T10:36:09.922 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.844+0000 7fc4267fc700 1 -- 172.21.15.67:0/2423751001 >> v1:172.21.15.72:6789/0 conn(0x7fc42003a7c0 legacy=0x7fc420050110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:09.922 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.844+0000 7fc4267fc700 1 --2- 172.21.15.67:0/2423751001 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fc418007fa0 0x7fc4180083a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:09.922 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.844+0000 7fc4267fc700 1 --2- 172.21.15.67:0/2423751001 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc418008c60 0x7fc4180090e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:09.923 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.844+0000 7fc4267fc700 1 --2- 172.21.15.67:0/2423751001 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc4180074f0 0x7fc418009620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:09.923 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.844+0000 7fc4267fc700 1 -- 172.21.15.67:0/2423751001 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc420033c70 con 0x7fc418008c60 2023-10-14T10:36:09.923 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.844+0000 7fc4267fc700 1 -- 172.21.15.67:0/2423751001 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc418014720 con 0x7fc4180074f0 2023-10-14T10:36:09.923 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.844+0000 7fc4267fc700 1 -- 172.21.15.67:0/2423751001 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4180148e0 con 0x7fc418007fa0 2023-10-14T10:36:09.923 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.844+0000 7fc4277fe700 1 --2- 172.21.15.67:0/2423751001 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fc418007fa0 0x7fc4180083a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:09.924 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.844+0000 7fc427fff700 1 --2- 172.21.15.67:0/2423751001 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc4180074f0 0x7fc418009620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:09.924 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.844+0000 7f17fe7fc700 1 --2- 172.21.15.67:0/2866748129 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f17e406aed0 0x7f17e406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:09.924 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.844+0000 7f17ef7fe700 1 -- 172.21.15.67:0/2866748129 <== osd.3 v2:172.21.15.67:6817/1848001176 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7f17e4075b40 con 0x7f17e4073070 2023-10-14T10:36:09.925 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.844+0000 7f8ff6d9d700 1 --2- 172.21.15.67:0/3721919745 >> [v2:172.21.15.72:6800/569686991,v1:172.21.15.72:6801/569686991] conn(0x7f8fd80731f0 0x7f8fd8075610 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:09.925 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.844+0000 7f2c96ffd700 1 -- 172.21.15.67:0/1142749056 <== osd.6 v2:172.21.15.72:6816/3767685934 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7f2ca00cf250 con 0x7f2ca00ceae0 2023-10-14T10:36:09.925 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.844+0000 7fc4277fe700 1 -- 172.21.15.67:0/2423751001 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc4180074f0 msgr2=0x7fc418009620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:09.926 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.844+0000 7fc4277fe700 1 --2- 172.21.15.67:0/2423751001 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc4180074f0 0x7fc418009620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:09.926 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.844+0000 7fc4277fe700 1 -- 172.21.15.67:0/2423751001 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc418008c60 msgr2=0x7fc4180090e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:09.926 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.844+0000 7fc4277fe700 1 --2- 172.21.15.67:0/2423751001 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc418008c60 0x7fc4180090e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:09.927 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.844+0000 7f17fe7fc700 1 --2- 172.21.15.67:0/2866748129 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f17e406aed0 0x7f17e406d390 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f18000cf130 tx=0x7f17f40092c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:09.927 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.845+0000 7fc4277fe700 1 -- 172.21.15.67:0/2423751001 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc420033f50 con 0x7fc418007fa0 2023-10-14T10:36:09.927 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.845+0000 7fc427fff700 1 --2- 172.21.15.67:0/2423751001 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc4180074f0 0x7fc418009620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-14T10:36:09.928 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.845+0000 7fc427fff700 1 -- 172.21.15.67:0/2423751001 reap_dead start 2023-10-14T10:36:09.928 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.845+0000 7fc4277fe700 1 --2- 172.21.15.67:0/2423751001 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fc418007fa0 0x7fc4180083a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fc41c009b30 tx=0x7fc418009c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=813d24e2258e9f70 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:09.928 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.845+0000 7f8fe6ffd700 1 -- 172.21.15.67:0/3721919745 <== osd.5 v2:172.21.15.72:6800/569686991 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7f8fd8075cc0 con 0x7f8fd80731f0 2023-10-14T10:36:09.930 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.845+0000 7fc4267fc700 1 -- 172.21.15.67:0/2423751001 <== mon.2 v2:172.21.15.72:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc41c00e6c0 con 0x7fc418007fa0 2023-10-14T10:36:09.930 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.845+0000 7fc4267fc700 1 -- 172.21.15.67:0/2423751001 <== mon.2 v2:172.21.15.72:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc41c00e820 con 0x7fc418007fa0 2023-10-14T10:36:09.931 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.845+0000 7fc4267fc700 1 -- 172.21.15.67:0/2423751001 <== mon.2 v2:172.21.15.72:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc41c00eb90 con 0x7fc418007fa0 2023-10-14T10:36:09.931 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.846+0000 7fc42cde9700 1 -- 172.21.15.67:0/2423751001 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fc418007fa0 msgr2=0x7fc4180083a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:09.931 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.846+0000 7fc42cde9700 1 --2- 172.21.15.67:0/2423751001 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fc418007fa0 0x7fc4180083a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fc41c009b30 tx=0x7fc418009c90 comp rx=0 tx=0).stop 2023-10-14T10:36:09.931 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.853+0000 7fc42cde9700 1 -- 172.21.15.67:0/2423751001 shutdown_connections 2023-10-14T10:36:09.931 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.853+0000 7fc42cde9700 1 --2- 172.21.15.67:0/2423751001 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fc418007fa0 0x7fc4180083a0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:09.932 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.853+0000 7fc42cde9700 1 -- 172.21.15.67:0/2423751001 >> 172.21.15.67:0/2423751001 conn(0x7fc420054c40 msgr2=0x7fc420055050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:09.932 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.853+0000 7fc42cde9700 1 -- 172.21.15.67:0/2423751001 shutdown_connections 2023-10-14T10:36:09.932 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.853+0000 7fc42cde9700 1 -- 172.21.15.67:0/2423751001 wait complete. 2023-10-14T10:36:09.932 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.855+0000 7fc42cde9700 1 Processor -- start 2023-10-14T10:36:09.932 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.855+0000 7fc42cde9700 1 -- start start 2023-10-14T10:36:09.933 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.856+0000 7fc42cde9700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fc418007fa0 0x7fc42004b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:09.933 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.856+0000 7fc42cde9700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc420049fb0 0x7fc42017f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:09.933 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.856+0000 7fc42cde9700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc4201888f0 0x7fc420188d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:09.933 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.856+0000 7fc42cde9700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4200e3fa0 con 0x7fc4201888f0 2023-10-14T10:36:09.933 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.856+0000 7fc42cde9700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc42004a890 con 0x7fc420049fb0 2023-10-14T10:36:09.934 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.856+0000 7fc42cde9700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4201892b0 con 0x7fc418007fa0 2023-10-14T10:36:09.934 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.856+0000 7fc4277fe700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fc418007fa0 0x7fc42004b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:09.934 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.856+0000 7fc4277fe700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fc418007fa0 0x7fc42004b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3301/0 says I am v2:172.21.15.67:60308/0 (socket says 172.21.15.67:60308) 2023-10-14T10:36:09.934 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.856+0000 7fc4277fe700 1 -- 172.21.15.67:0/3215872597 learned_addr learned my addr 172.21.15.67:0/3215872597 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:09.935 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.856+0000 7fc427fff700 1 --2- 172.21.15.67:0/3215872597 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc4201888f0 0x7fc420188d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:09.935 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.856+0000 7fc426ffd700 1 --2- 172.21.15.67:0/3215872597 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc420049fb0 0x7fc42017f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:09.935 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.856+0000 7fc427fff700 1 -- 172.21.15.67:0/3215872597 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fc418007fa0 msgr2=0x7fc42004b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:09.935 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.856+0000 7fc427fff700 1 --2- 172.21.15.67:0/3215872597 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fc418007fa0 0x7fc42004b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:09.935 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.856+0000 7fc427fff700 1 -- 172.21.15.67:0/3215872597 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc420049fb0 msgr2=0x7fc42017f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:09.936 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.856+0000 7fc427fff700 1 --2- 172.21.15.67:0/3215872597 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc420049fb0 0x7fc42017f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:09.936 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.857+0000 7fc427fff700 1 -- 172.21.15.67:0/3215872597 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc420048ff0 con 0x7fc4201888f0 2023-10-14T10:36:09.936 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.857+0000 7fc426ffd700 1 --2- 172.21.15.67:0/3215872597 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc420049fb0 0x7fc42017f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-14T10:36:09.936 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.857+0000 7fc427fff700 1 --2- 172.21.15.67:0/3215872597 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc4201888f0 0x7fc420188d70 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc428050430 tx=0x7fc428067bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:09.937 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.857+0000 7fc413fff700 1 -- 172.21.15.67:0/3215872597 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc42808f070 con 0x7fc4201888f0 2023-10-14T10:36:09.937 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.857+0000 7fc413fff700 1 -- 172.21.15.67:0/3215872597 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc42806f450 con 0x7fc4201888f0 2023-10-14T10:36:09.937 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.857+0000 7fc413fff700 1 -- 172.21.15.67:0/3215872597 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc42806f820 con 0x7fc4201888f0 2023-10-14T10:36:09.937 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.857+0000 7fc42cde9700 1 -- 172.21.15.67:0/3215872597 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc42004aaa0 con 0x7fc4201888f0 2023-10-14T10:36:09.937 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.857+0000 7fc42cde9700 1 -- 172.21.15.67:0/3215872597 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc42004b060 con 0x7fc4201888f0 2023-10-14T10:36:09.937 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.861+0000 7fc413fff700 1 -- 172.21.15.67:0/3215872597 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fc42806f980 con 0x7fc4201888f0 2023-10-14T10:36:09.938 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.862+0000 7fc413fff700 1 --2- 172.21.15.67:0/3215872597 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fc40c06af50 0x7fc40c06d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:09.938 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.862+0000 7fc413fff700 1 -- 172.21.15.67:0/3215872597 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc428103390 con 0x7fc4201888f0 2023-10-14T10:36:09.940 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.862+0000 7f7461d1b700 1 Processor -- start 2023-10-14T10:36:09.940 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.862+0000 7f7461d1b700 1 -- start start 2023-10-14T10:36:09.940 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.863+0000 7f7461d1b700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f745c06d0c0 0x7f745c0ca450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:09.940 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.863+0000 7f7461d1b700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f745c0ca990 con 0x7f745c0ceae0 2023-10-14T10:36:09.940 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.863+0000 7f7461d1b700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f745c0cf250 con 0x7f745c16f060 2023-10-14T10:36:09.941 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.863+0000 7f7461d1b700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f745c16ebd0 con 0x7f745c06d0c0 2023-10-14T10:36:09.941 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.863+0000 7f745b7fe700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f745c0ceae0 0x7f745c08a490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:51804/0 (socket says 172.21.15.67:51804) 2023-10-14T10:36:09.941 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.863+0000 7f745b7fe700 1 -- 172.21.15.67:0/476299162 learned_addr learned my addr 172.21.15.67:0/476299162 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:36:09.941 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.863+0000 7f745bfff700 1 --2- 172.21.15.67:0/476299162 >> v2:172.21.15.67:3300/0 conn(0x7f745c06d0c0 0x7f745c0ca450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:09.941 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.864+0000 7f745a7fc700 1 -- 172.21.15.67:0/476299162 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2100495110 0 0) 0x7f745c0ca990 con 0x7f745c0ceae0 2023-10-14T10:36:09.942 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.864+0000 7f745a7fc700 1 -- 172.21.15.67:0/476299162 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74440057c0 con 0x7f745c0ceae0 2023-10-14T10:36:09.942 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.864+0000 7f745a7fc700 1 -- 172.21.15.67:0/476299162 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3941057989 0 0) 0x7f74440057c0 con 0x7f745c0ceae0 2023-10-14T10:36:09.942 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.864+0000 7f745a7fc700 1 -- 172.21.15.67:0/476299162 >> v2:172.21.15.67:3300/0 conn(0x7f745c06d0c0 msgr2=0x7f745c0ca450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:09.942 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.864+0000 7f745a7fc700 1 --2- 172.21.15.67:0/476299162 >> v2:172.21.15.67:3300/0 conn(0x7f745c06d0c0 0x7f745c0ca450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:09.942 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.864+0000 7f745a7fc700 1 -- 172.21.15.67:0/476299162 >> v1:172.21.15.72:6789/0 conn(0x7f745c16f060 legacy=0x7f745c06c9b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-14T10:36:09.943 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.864+0000 7f745a7fc700 1 -- 172.21.15.67:0/476299162 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f745c16e580 con 0x7f745c0ceae0 2023-10-14T10:36:09.943 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.864+0000 7f745a7fc700 1 -- 172.21.15.67:0/476299162 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3180043755 0 0) 0x7f7450001670 con 0x7f745c0ceae0 2023-10-14T10:36:09.943 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.864+0000 7f745bfff700 1 --2- 172.21.15.67:0/476299162 >> v2:172.21.15.67:3300/0 conn(0x7f745c06d0c0 0x7f745c0ca450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-14T10:36:09.943 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.864+0000 7f745a7fc700 1 -- 172.21.15.67:0/476299162 >> v1:172.21.15.67:6789/0 conn(0x7f745c0ceae0 legacy=0x7f745c08a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:09.944 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.864+0000 7f745a7fc700 1 --2- 172.21.15.67:0/476299162 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7444005c10 0x7f7444006010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:09.944 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.864+0000 7f745a7fc700 1 --2- 172.21.15.67:0/476299162 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7444006550 0x7f7444006a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:09.944 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.864+0000 7f745a7fc700 1 --2- 172.21.15.67:0/476299162 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f7444007180 0x7f7444014810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:09.944 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.864+0000 7f745a7fc700 1 -- 172.21.15.67:0/476299162 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f745c16ebd0 con 0x7f7444005c10 2023-10-14T10:36:09.944 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.864+0000 7f745a7fc700 1 -- 172.21.15.67:0/476299162 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7444007600 con 0x7f7444006550 2023-10-14T10:36:09.945 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.864+0000 7f745a7fc700 1 -- 172.21.15.67:0/476299162 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74440077c0 con 0x7f7444007180 2023-10-14T10:36:09.945 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.865+0000 7f745bfff700 1 --2- 172.21.15.67:0/476299162 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f7444007180 0x7f7444014810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:09.945 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.865+0000 7f745b7fe700 1 --2- 172.21.15.67:0/476299162 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7444005c10 0x7f7444006010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:09.945 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.865+0000 7f745bfff700 1 -- 172.21.15.67:0/476299162 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7444006550 msgr2=0x7f7444006a60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-14T10:36:09.945 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.865+0000 7f745bfff700 1 --2- 172.21.15.67:0/476299162 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7444006550 0x7f7444006a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:09.946 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.865+0000 7f745bfff700 1 -- 172.21.15.67:0/476299162 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7444005c10 msgr2=0x7f7444006010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:09.946 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.865+0000 7f745bfff700 1 --2- 172.21.15.67:0/476299162 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7444005c10 0x7f7444006010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:09.946 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.865+0000 7f745bfff700 1 -- 172.21.15.67:0/476299162 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f745c16e580 con 0x7f7444007180 2023-10-14T10:36:09.946 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.865+0000 7f745bfff700 1 -- 172.21.15.67:0/476299162 reap_dead start 2023-10-14T10:36:09.946 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.865+0000 7f745bfff700 1 --2- 172.21.15.67:0/476299162 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f7444007180 0x7f7444014810 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f745c0ce8a0 tx=0x7f7444007a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=add4bf239d512a42 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:09.947 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.865+0000 7f745a7fc700 1 -- 172.21.15.67:0/476299162 <== mon.2 v2:172.21.15.72:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f745400ed90 con 0x7f7444007180 2023-10-14T10:36:09.947 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.866+0000 7f745a7fc700 1 -- 172.21.15.67:0/476299162 <== mon.2 v2:172.21.15.72:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f745400f040 con 0x7f7444007180 2023-10-14T10:36:09.947 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.866+0000 7f745a7fc700 1 -- 172.21.15.67:0/476299162 <== mon.2 v2:172.21.15.72:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7454003c60 con 0x7f7444007180 2023-10-14T10:36:09.948 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.866+0000 7fc4277fe700 1 --2- 172.21.15.67:0/3215872597 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fc40c06af50 0x7fc40c06d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:09.948 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.866+0000 7fc4277fe700 1 --2- 172.21.15.67:0/3215872597 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fc40c06af50 0x7fc40c06d410 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc41c0090b0 tx=0x7fc41c01e040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:09.948 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.866+0000 7f7461d1b700 1 -- 172.21.15.67:0/476299162 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f7444007180 msgr2=0x7f7444014810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:09.949 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.866+0000 7f7461d1b700 1 --2- 172.21.15.67:0/476299162 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f7444007180 0x7f7444014810 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f745c0ce8a0 tx=0x7f7444007a90 comp rx=0 tx=0).stop 2023-10-14T10:36:09.949 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.867+0000 7f7461d1b700 1 -- 172.21.15.67:0/476299162 shutdown_connections 2023-10-14T10:36:09.949 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.867+0000 7f7461d1b700 1 --2- 172.21.15.67:0/476299162 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f7444007180 0x7f7444014810 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:09.949 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.867+0000 7f7461d1b700 1 -- 172.21.15.67:0/476299162 >> 172.21.15.67:0/476299162 conn(0x7f745c0cf3f0 msgr2=0x7f745c0cf800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:09.950 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.867+0000 7fc42cde9700 1 --2- 172.21.15.67:0/3215872597 >> [v2:172.21.15.67:6808/2827491791,v1:172.21.15.67:6809/2827491791] conn(0x7fc4080015e0 0x7fc408003aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:09.950 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.867+0000 7fc42cde9700 1 -- 172.21.15.67:0/3215872597 --> [v2:172.21.15.67:6808/2827491791,v1:172.21.15.67:6809/2827491791] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc408006bd0 con 0x7fc4080015e0 2023-10-14T10:36:09.951 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.868+0000 7fc426ffd700 1 --2- 172.21.15.67:0/3215872597 >> [v2:172.21.15.67:6808/2827491791,v1:172.21.15.67:6809/2827491791] conn(0x7fc4080015e0 0x7fc408003aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:09.951 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.868+0000 7fc426ffd700 1 --2- 172.21.15.67:0/3215872597 >> [v2:172.21.15.67:6808/2827491791,v1:172.21.15.67:6809/2827491791] conn(0x7fc4080015e0 0x7fc408003aa0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:09.951 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.869+0000 7fc413fff700 1 -- 172.21.15.67:0/3215872597 <== osd.2 v2:172.21.15.67:6808/2827491791 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7fc408006bd0 con 0x7fc4080015e0 2023-10-14T10:36:09.951 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.876+0000 7f8399ae7700 1 -- 172.21.15.67:0/2741570795 --> [v2:172.21.15.72:6802/1891230785,v1:172.21.15.72:6804/1891230785] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f83940ca710 con 0x7f83800730f0 2023-10-14T10:36:09.952 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.876+0000 7f39a4a7a700 1 -- 172.21.15.67:0/289332590 --> [v2:172.21.15.67:6800/1287255499,v1:172.21.15.67:6801/1287255499] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f39a00cefa0 con 0x7f39800731a0 2023-10-14T10:36:09.952 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.877+0000 7f398effd700 1 -- 172.21.15.67:0/289332590 <== osd.0 v2:172.21.15.67:6800/1287255499 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f399800ce70 con 0x7f39800731a0 2023-10-14T10:36:09.952 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.877+0000 7f9ac8929700 1 Processor -- start 2023-10-14T10:36:09.953 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.877+0000 7f837bfff700 1 -- 172.21.15.67:0/2741570795 <== osd.4 v2:172.21.15.72:6802/1891230785 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f83840094a0 con 0x7f83800730f0 2023-10-14T10:36:09.954 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.877+0000 7f39a4a7a700 1 -- 172.21.15.67:0/289332590 >> [v2:172.21.15.67:6800/1287255499,v1:172.21.15.67:6801/1287255499] conn(0x7f39800731a0 msgr2=0x7f39800755c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:09.954 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.877+0000 7f39a4a7a700 1 --2- 172.21.15.67:0/289332590 >> [v2:172.21.15.67:6800/1287255499,v1:172.21.15.67:6801/1287255499] conn(0x7f39800731a0 0x7f39800755c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:09.954 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.877+0000 7f39a4a7a700 1 -- 172.21.15.67:0/289332590 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f398006af80 msgr2=0x7f398006d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:09.954 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.877+0000 7f39a4a7a700 1 --2- 172.21.15.67:0/289332590 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f398006af80 0x7f398006d440 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f39a00ae420 tx=0x7f3994008040 comp rx=0 tx=0).stop 2023-10-14T10:36:09.955 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.877+0000 7f39a4a7a700 1 -- 172.21.15.67:0/289332590 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f39a006c9b0 msgr2=0x7f39a00c9300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:09.955 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.877+0000 7f39a4a7a700 1 --2- 172.21.15.67:0/289332590 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f39a006c9b0 0x7f39a00c9300 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f39900061a0 tx=0x7f399000b8a0 comp rx=0 tx=0).stop 2023-10-14T10:36:09.955 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.877+0000 7f399ed9d700 1 -- 172.21.15.67:0/289332590 reap_dead start 2023-10-14T10:36:09.955 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.878+0000 7f39a4a7a700 1 -- 172.21.15.67:0/289332590 shutdown_connections 2023-10-14T10:36:09.955 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.878+0000 7f39a4a7a700 1 -- 172.21.15.67:0/289332590 >> 172.21.15.67:0/289332590 conn(0x7f39a00cf3f0 msgr2=0x7f39a00aef70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:09.956 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.878+0000 7f1805303700 1 -- 172.21.15.67:0/2866748129 --> [v2:172.21.15.67:6817/1848001176,v1:172.21.15.67:6819/1848001176] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f18000cd800 con 0x7f17e4073070 2023-10-14T10:36:09.956 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.877+0000 7f9ac8929700 1 -- start start 2023-10-14T10:36:09.957 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.878+0000 7f17ef7fe700 1 -- 172.21.15.67:0/2866748129 <== osd.3 v2:172.21.15.67:6817/1848001176 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f17f8002050 con 0x7f17e4073070 2023-10-14T10:36:09.957 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.878+0000 7f39a4a7a700 1 -- 172.21.15.67:0/289332590 shutdown_connections 2023-10-14T10:36:09.957 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.878+0000 7f39a4a7a700 1 -- 172.21.15.67:0/289332590 wait complete. 2023-10-14T10:36:09.958 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.878+0000 7f8399ae7700 1 -- 172.21.15.67:0/2741570795 >> [v2:172.21.15.72:6802/1891230785,v1:172.21.15.72:6804/1891230785] conn(0x7f83800730f0 msgr2=0x7f8380075510 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:09.958 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.878+0000 7f8399ae7700 1 --2- 172.21.15.67:0/2741570795 >> [v2:172.21.15.72:6802/1891230785,v1:172.21.15.72:6804/1891230785] conn(0x7f83800730f0 0x7f8380075510 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:09.959 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.878+0000 7f9ac8929700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f9ac40c9fc0 0x7f9ac4157250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:09.959 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.878+0000 7f9ac8929700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9ac4159fa0 0x7f9ac406c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:09.959 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.878+0000 7f9ac8929700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ac415af20 con 0x7f9ac40cff10 2023-10-14T10:36:09.959 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.878+0000 7f9ac8929700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ac415a960 con 0x7f9ac4159fa0 2023-10-14T10:36:09.960 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.878+0000 7f9ac8929700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac415ac40 con 0x7f9ac40c9fc0 2023-10-14T10:36:09.960 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.879+0000 7f9ac2d9d700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9ac4159fa0 0x7f9ac406c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:09.960 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.879+0000 7f9ac2d9d700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9ac4159fa0 0x7f9ac406c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44048/0 (socket says 172.21.15.67:44048) 2023-10-14T10:36:09.960 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.879+0000 7f9ac2d9d700 1 -- 172.21.15.67:0/1139010823 learned_addr learned my addr 172.21.15.67:0/1139010823 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:09.961 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.879+0000 7f1805303700 1 -- 172.21.15.67:0/2866748129 >> [v2:172.21.15.67:6817/1848001176,v1:172.21.15.67:6819/1848001176] conn(0x7f17e4073070 msgr2=0x7f17e4075490 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:09.961 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.879+0000 7f1805303700 1 --2- 172.21.15.67:0/2866748129 >> [v2:172.21.15.67:6817/1848001176,v1:172.21.15.67:6819/1848001176] conn(0x7f17e4073070 0x7f17e4075490 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:09.961 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.879+0000 7f1805303700 1 -- 172.21.15.67:0/2866748129 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f17e406aed0 msgr2=0x7f17e406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:09.961 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.879+0000 7f1805303700 1 --2- 172.21.15.67:0/2866748129 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f17e406aed0 0x7f17e406d390 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f18000cf130 tx=0x7f17f40092c0 comp rx=0 tx=0).stop 2023-10-14T10:36:09.962 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.879+0000 7f1805303700 1 -- 172.21.15.67:0/2866748129 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f18000b02b0 msgr2=0x7f18000b2ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:09.962 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.879+0000 7f1805303700 1 --2- 172.21.15.67:0/2866748129 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f18000b02b0 0x7f18000b2ca0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f17f0011b50 tx=0x7f17f0004790 comp rx=0 tx=0).stop 2023-10-14T10:36:09.963 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.879+0000 7f9ac1d9b700 1 --2- 172.21.15.67:0/1139010823 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f9ac40c9fc0 0x7f9ac4157250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:09.963 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.879+0000 7f9ac2d9d700 1 -- 172.21.15.67:0/1139010823 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f9ac40c9fc0 msgr2=0x7f9ac4157250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:09.963 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.879+0000 7f9ac2d9d700 1 --2- 172.21.15.67:0/1139010823 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f9ac40c9fc0 0x7f9ac4157250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:09.963 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.879+0000 7f9ac2d9d700 1 -- 172.21.15.67:0/1139010823 >> v1:172.21.15.72:6789/0 conn(0x7f9ac40cff10 legacy=0x7f9ac415df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:09.963 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.879+0000 7f9ac2d9d700 1 -- 172.21.15.67:0/1139010823 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ac415d730 con 0x7f9ac4159fa0 2023-10-14T10:36:09.964 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.879+0000 7f9ac2d9d700 1 --2- 172.21.15.67:0/1139010823 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ac4159fa0 0x7f9ac406c9b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9ab40131e0 tx=0x7f9ab40134f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15d40f3753162496 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:09.964 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.880+0000 7f9ac159a700 1 -- 172.21.15.67:0/1139010823 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab4016070 con 0x7f9ac4159fa0 2023-10-14T10:36:09.964 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.880+0000 7f9ac159a700 1 -- 172.21.15.67:0/1139010823 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ab4013df0 con 0x7f9ac4159fa0 2023-10-14T10:36:09.964 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.880+0000 7f9ac159a700 1 -- 172.21.15.67:0/1139010823 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab4004250 con 0x7f9ac4159fa0 2023-10-14T10:36:09.965 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.880+0000 7f8399ae7700 1 -- 172.21.15.67:0/2741570795 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f838006af50 msgr2=0x7f838006d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:09.965 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.880+0000 7f8399ae7700 1 --2- 172.21.15.67:0/2741570795 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f838006af50 0x7f838006d410 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f838c0012e0 tx=0x7f838c0015b0 comp rx=0 tx=0).stop 2023-10-14T10:36:09.966 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.880+0000 7f8399ae7700 1 -- 172.21.15.67:0/2741570795 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f839406c9b0 msgr2=0x7f8394156970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:09.966 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.880+0000 7f8399ae7700 1 --2- 172.21.15.67:0/2741570795 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f839406c9b0 0x7f8394156970 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8388004270 tx=0x7f8388004750 comp rx=0 tx=0).stop 2023-10-14T10:36:09.966 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.880+0000 7f8393fff700 1 -- 172.21.15.67:0/2741570795 reap_dead start 2023-10-14T10:36:09.966 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.880+0000 7f8399ae7700 1 -- 172.21.15.67:0/2741570795 shutdown_connections 2023-10-14T10:36:09.966 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.880+0000 7f8399ae7700 1 -- 172.21.15.67:0/2741570795 >> 172.21.15.67:0/2741570795 conn(0x7f83940cf3f0 msgr2=0x7f83940b1430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:09.967 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.880+0000 7f8ffc94f700 1 -- 172.21.15.67:0/3721919745 --> [v2:172.21.15.72:6800/569686991,v1:172.21.15.72:6801/569686991] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8ff816e580 con 0x7f8fd80731f0 2023-10-14T10:36:09.967 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.880+0000 7f17ff7fe700 1 -- 172.21.15.67:0/2866748129 reap_dead start 2023-10-14T10:36:09.967 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.880+0000 7f9ac8929700 1 -- 172.21.15.67:0/1139010823 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ac4159fa0 msgr2=0x7f9ac406c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:09.968 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.880+0000 7f9ac8929700 1 --2- 172.21.15.67:0/1139010823 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ac4159fa0 0x7f9ac406c9b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9ab40131e0 tx=0x7f9ab40134f0 comp rx=0 tx=0).stop 2023-10-14T10:36:09.968 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.878+0000 7f7461d1b700 1 -- 172.21.15.67:0/476299162 shutdown_connections 2023-10-14T10:36:09.968 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.878+0000 7f7461d1b700 1 -- 172.21.15.67:0/476299162 wait complete. 2023-10-14T10:36:09.969 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.880+0000 7f7461d1b700 1 Processor -- start 2023-10-14T10:36:09.969 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.881+0000 7f7461d1b700 1 -- start start 2023-10-14T10:36:09.969 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.881+0000 7f8fe6ffd700 1 -- 172.21.15.67:0/3721919745 <== osd.5 v2:172.21.15.72:6800/569686991 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f8fec001440 con 0x7f8fd80731f0 2023-10-14T10:36:09.970 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.881+0000 7f9ac8929700 1 -- 172.21.15.67:0/1139010823 shutdown_connections 2023-10-14T10:36:09.970 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.881+0000 7f9ac8929700 1 --2- 172.21.15.67:0/1139010823 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ac4159fa0 0x7f9ac406c9b0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:09.970 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.881+0000 7f9ac8929700 1 --2- 172.21.15.67:0/1139010823 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f9ac40c9fc0 0x7f9ac4157250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:09.970 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.881+0000 7f9ac8929700 1 -- 172.21.15.67:0/1139010823 >> 172.21.15.67:0/1139010823 conn(0x7f9ac40cf3f0 msgr2=0x7f9ac40cf800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:09.971 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.881+0000 7f1805303700 1 -- 172.21.15.67:0/2866748129 shutdown_connections 2023-10-14T10:36:09.971 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.881+0000 7f1805303700 1 -- 172.21.15.67:0/2866748129 >> 172.21.15.67:0/2866748129 conn(0x7f180006afb0 msgr2=0x7f18000c4bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:09.971 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.881+0000 7f1805303700 1 -- 172.21.15.67:0/2866748129 shutdown_connections 2023-10-14T10:36:09.972 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.881+0000 7f2ca82cc700 1 -- 172.21.15.67:0/1142749056 --> [v2:172.21.15.72:6816/3767685934,v1:172.21.15.72:6817/3767685934] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2ca00b3fc0 con 0x7f2ca00ceae0 2023-10-14T10:36:09.972 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.881+0000 7f1805303700 1 -- 172.21.15.67:0/2866748129 wait complete. 2023-10-14T10:36:09.973 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.882+0000 7f7461d1b700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7444007180 0x7f745c0b3a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:09.973 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.882+0000 7f7461d1b700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f745c16c950 0x7f745c06def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:09.973 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.882+0000 7f7461d1b700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f745c06e430 0x7f745c0a84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:09.973 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.882+0000 7f7461d1b700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f745c06e8b0 con 0x7f7444007180 2023-10-14T10:36:09.974 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.882+0000 7f7461d1b700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f745c0b3fb0 con 0x7f745c16c950 2023-10-14T10:36:09.974 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.882+0000 7f7461d1b700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f745c0b4ac0 con 0x7f745c06e430 2023-10-14T10:36:09.974 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.882+0000 7f8ffc94f700 1 -- 172.21.15.67:0/3721919745 >> [v2:172.21.15.72:6800/569686991,v1:172.21.15.72:6801/569686991] conn(0x7f8fd80731f0 msgr2=0x7f8fd8075610 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:09.974 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.882+0000 7f8ffc94f700 1 --2- 172.21.15.67:0/3721919745 >> [v2:172.21.15.72:6800/569686991,v1:172.21.15.72:6801/569686991] conn(0x7f8fd80731f0 0x7f8fd8075610 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:09.975 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.882+0000 7f745bfff700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f745c06e430 0x7f745c0a84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:09.975 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.882+0000 7f745bfff700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f745c06e430 0x7f745c0a84a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3301/0 says I am v2:172.21.15.67:60338/0 (socket says 172.21.15.67:60338) 2023-10-14T10:36:09.975 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.882+0000 7f745bfff700 1 -- 172.21.15.67:0/2709544394 learned_addr learned my addr 172.21.15.67:0/2709544394 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:09.976 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.882+0000 7f745affd700 1 --2- 172.21.15.67:0/2709544394 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f745c16c950 0x7f745c06def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:09.976 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.882+0000 7f8ffc94f700 1 -- 172.21.15.67:0/3721919745 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f8fd806afd0 msgr2=0x7f8fd806d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:09.977 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.882+0000 7f8ffc94f700 1 --2- 172.21.15.67:0/3721919745 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f8fd806afd0 0x7f8fd806d490 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8ff80b1ea0 tx=0x7f8ff0012000 comp rx=0 tx=0).stop 2023-10-14T10:36:09.977 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.882+0000 7f8ffc94f700 1 -- 172.21.15.67:0/3721919745 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ff808a700 msgr2=0x7f8ff80b11d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:09.977 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.882+0000 7f8ffc94f700 1 --2- 172.21.15.67:0/3721919745 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ff808a700 0x7f8ff80b11d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8fe8000f60 tx=0x7f8fe800c6e0 comp rx=0 tx=0).stop 2023-10-14T10:36:09.977 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.882+0000 7f8ff6d9d700 1 -- 172.21.15.67:0/3721919745 reap_dead start 2023-10-14T10:36:09.978 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.882+0000 7f745bfff700 1 -- 172.21.15.67:0/2709544394 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f745c16c950 msgr2=0x7f745c06def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:09.978 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.882+0000 7f745bfff700 1 --2- 172.21.15.67:0/2709544394 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f745c16c950 0x7f745c06def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:09.978 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.882+0000 7f745bfff700 1 -- 172.21.15.67:0/2709544394 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7444007180 msgr2=0x7f745c0b3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:10.080 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.882+0000 7f745bfff700 1 --2- 172.21.15.67:0/2709544394 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7444007180 0x7f745c0b3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:10.080 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.882+0000 7f745bfff700 1 -- 172.21.15.67:0/2709544394 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f745c0a9e30 con 0x7f745c06e430 2023-10-14T10:36:10.081 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.883+0000 7f745bfff700 1 --2- 172.21.15.67:0/2709544394 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f745c06e430 0x7f745c0a84a0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f745c0cf150 tx=0x7f7454009750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:10.081 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.883+0000 7f744bfff700 1 -- 172.21.15.67:0/2709544394 <== mon.2 v2:172.21.15.72:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7454007000 con 0x7f745c06e430 2023-10-14T10:36:10.081 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.883+0000 7f744bfff700 1 -- 172.21.15.67:0/2709544394 <== mon.2 v2:172.21.15.72:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7454004020 con 0x7f745c06e430 2023-10-14T10:36:10.081 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.883+0000 7f7461d1b700 1 -- 172.21.15.67:0/2709544394 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f745c0aa110 con 0x7f745c06e430 2023-10-14T10:36:10.081 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.883+0000 7f744bfff700 1 -- 172.21.15.67:0/2709544394 <== mon.2 v2:172.21.15.72:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74540043d0 con 0x7f745c06e430 2023-10-14T10:36:10.082 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.883+0000 7f7461d1b700 1 -- 172.21.15.67:0/2709544394 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f745c0aa6d0 con 0x7f745c06e430 2023-10-14T10:36:10.082 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.884+0000 7f2c96ffd700 1 -- 172.21.15.67:0/1142749056 <== osd.6 v2:172.21.15.72:6816/3767685934 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f2ca00b3fc0 con 0x7f2ca00ceae0 2023-10-14T10:36:10.083 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.884+0000 7f8399ae7700 1 -- 172.21.15.67:0/2741570795 shutdown_connections 2023-10-14T10:36:10.083 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.884+0000 7f8399ae7700 1 -- 172.21.15.67:0/2741570795 wait complete. 2023-10-14T10:36:10.084 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.885+0000 7f744bfff700 1 -- 172.21.15.67:0/2709544394 <== mon.2 v2:172.21.15.72:3301/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f7454023070 con 0x7f745c06e430 2023-10-14T10:36:10.084 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.885+0000 7f7449ffb700 1 -- 172.21.15.67:0/2709544394 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f745c16e860 con 0x7f745c06e430 2023-10-14T10:36:10.085 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.885+0000 7f744bfff700 1 --2- 172.21.15.67:0/2709544394 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f744006af50 0x7f744006d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:10.085 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.885+0000 7f744bfff700 1 -- 172.21.15.67:0/2709544394 <== mon.2 v2:172.21.15.72:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f745408b330 con 0x7f745c06e430 2023-10-14T10:36:10.085 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.885+0000 7f745b7fe700 1 --2- 172.21.15.67:0/2709544394 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f744006af50 0x7f744006d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:10.085 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.885+0000 7f744bfff700 1 --2- 172.21.15.67:0/2709544394 >> [v2:172.21.15.67:6816/2435538787,v1:172.21.15.67:6818/2435538787] conn(0x7f74400730f0 0x7f7440075510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:10.086 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.885+0000 7f744bfff700 1 -- 172.21.15.67:0/2709544394 --> [v2:172.21.15.67:6816/2435538787,v1:172.21.15.67:6818/2435538787] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7440075bc0 con 0x7f74400730f0 2023-10-14T10:36:10.086 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.885+0000 7f745affd700 1 --2- 172.21.15.67:0/2709544394 >> [v2:172.21.15.67:6816/2435538787,v1:172.21.15.67:6818/2435538787] conn(0x7f74400730f0 0x7f7440075510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:10.086 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.886+0000 7f745b7fe700 1 --2- 172.21.15.67:0/2709544394 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f744006af50 0x7f744006d410 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7450001560 tx=0x7f7450000f50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:10.086 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.886+0000 7f745affd700 1 --2- 172.21.15.67:0/2709544394 >> [v2:172.21.15.67:6816/2435538787,v1:172.21.15.67:6818/2435538787] conn(0x7f74400730f0 0x7f7440075510 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:10.086 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.887+0000 7f744bfff700 1 -- 172.21.15.67:0/2709544394 <== osd.1 v2:172.21.15.67:6816/2435538787 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7f7440075bc0 con 0x7f74400730f0 2023-10-14T10:36:10.087 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.888+0000 7f8ffc94f700 1 -- 172.21.15.67:0/3721919745 shutdown_connections 2023-10-14T10:36:10.087 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.888+0000 7f8ffc94f700 1 -- 172.21.15.67:0/3721919745 >> 172.21.15.67:0/3721919745 conn(0x7f8ff80cf3f0 msgr2=0x7f8ff80afe00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:10.087 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.889+0000 7f744bfff700 1 -- 172.21.15.67:0/2709544394 <== mon.2 v2:172.21.15.72:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7454056670 con 0x7f745c06e430 2023-10-14T10:36:10.088 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.891+0000 7f2ca82cc700 1 -- 172.21.15.67:0/1142749056 >> [v2:172.21.15.72:6816/3767685934,v1:172.21.15.72:6817/3767685934] conn(0x7f2ca00ceae0 msgr2=0x7f2ca00612f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:10.088 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.891+0000 7f2ca82cc700 1 --2- 172.21.15.67:0/1142749056 >> [v2:172.21.15.72:6816/3767685934,v1:172.21.15.72:6817/3767685934] conn(0x7f2ca00ceae0 0x7f2ca00612f0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:10.089 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.891+0000 7f2ca82cc700 1 -- 172.21.15.67:0/1142749056 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f2c8406af00 msgr2=0x7f2c8406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:10.089 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.891+0000 7f2ca82cc700 1 --2- 172.21.15.67:0/1142749056 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f2c8406af00 0x7f2c8406d3c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2ca00c6060 tx=0x7f2c9c0090e0 comp rx=0 tx=0).stop 2023-10-14T10:36:10.089 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.891+0000 7f2ca82cc700 1 -- 172.21.15.67:0/1142749056 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2ca006d0c0 msgr2=0x7f2ca00b46b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:10.089 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.891+0000 7f2ca82cc700 1 --2- 172.21.15.67:0/1142749056 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2ca006d0c0 0x7f2ca00b46b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2c98015000 tx=0x7f2c98004ad0 comp rx=0 tx=0).stop 2023-10-14T10:36:10.089 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.891+0000 7f2ca6869700 1 -- 172.21.15.67:0/1142749056 reap_dead start 2023-10-14T10:36:10.090 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.892+0000 7f8ffc94f700 1 -- 172.21.15.67:0/3721919745 shutdown_connections 2023-10-14T10:36:10.090 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.894+0000 7f2ca82cc700 1 -- 172.21.15.67:0/1142749056 shutdown_connections 2023-10-14T10:36:10.091 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.894+0000 7f2ca82cc700 1 -- 172.21.15.67:0/1142749056 >> 172.21.15.67:0/1142749056 conn(0x7f2ca00cf3f0 msgr2=0x7f2ca0162fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:10.091 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.894+0000 7f2ca82cc700 1 -- 172.21.15.67:0/1142749056 shutdown_connections 2023-10-14T10:36:10.091 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.894+0000 7f2ca82cc700 1 -- 172.21.15.67:0/1142749056 wait complete. 2023-10-14T10:36:10.092 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.894+0000 7fc42cde9700 1 -- 172.21.15.67:0/3215872597 --> [v2:172.21.15.67:6808/2827491791,v1:172.21.15.67:6809/2827491791] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc408005cb0 con 0x7fc4080015e0 2023-10-14T10:36:10.092 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.895+0000 7fc413fff700 1 -- 172.21.15.67:0/3215872597 <== osd.2 v2:172.21.15.67:6808/2827491791 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fc408005cb0 con 0x7fc4080015e0 2023-10-14T10:36:10.092 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.896+0000 7fc42cde9700 1 -- 172.21.15.67:0/3215872597 >> [v2:172.21.15.67:6808/2827491791,v1:172.21.15.67:6809/2827491791] conn(0x7fc4080015e0 msgr2=0x7fc408003aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:10.093 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.896+0000 7fc42cde9700 1 --2- 172.21.15.67:0/3215872597 >> [v2:172.21.15.67:6808/2827491791,v1:172.21.15.67:6809/2827491791] conn(0x7fc4080015e0 0x7fc408003aa0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:10.093 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.897+0000 7f8ffc94f700 1 -- 172.21.15.67:0/3721919745 wait complete. 2023-10-14T10:36:10.093 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.898+0000 7fc42cde9700 1 -- 172.21.15.67:0/3215872597 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fc40c06af50 msgr2=0x7fc40c06d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:10.094 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.898+0000 7fc42cde9700 1 --2- 172.21.15.67:0/3215872597 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fc40c06af50 0x7fc40c06d410 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc41c0090b0 tx=0x7fc41c01e040 comp rx=0 tx=0).stop 2023-10-14T10:36:10.094 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.898+0000 7fc42cde9700 1 -- 172.21.15.67:0/3215872597 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc4201888f0 msgr2=0x7fc420188d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:10.094 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.898+0000 7fc42cde9700 1 --2- 172.21.15.67:0/3215872597 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc4201888f0 0x7fc420188d70 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc428050430 tx=0x7fc428067bb0 comp rx=0 tx=0).stop 2023-10-14T10:36:10.094 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.898+0000 7fc427fff700 1 -- 172.21.15.67:0/3215872597 reap_dead start 2023-10-14T10:36:10.095 INFO:teuthology.orchestra.run.smithi067.stdout:60129542150 2023-10-14T10:36:10.096 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-14T10:36:10.096 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.898+0000 7f9ac8929700 1 -- 172.21.15.67:0/1139010823 shutdown_connections 2023-10-14T10:36:10.097 INFO:teuthology.orchestra.run.smithi067.stdout:60129542149 2023-10-14T10:36:10.098 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-14T10:36:10.098 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.902+0000 7fc42cde9700 1 -- 172.21.15.67:0/3215872597 shutdown_connections 2023-10-14T10:36:10.099 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.902+0000 7fc42cde9700 1 -- 172.21.15.67:0/3215872597 >> 172.21.15.67:0/3215872597 conn(0x7fc420054c40 msgr2=0x7fc420038880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:10.099 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.903+0000 7fc42cde9700 1 -- 172.21.15.67:0/3215872597 shutdown_connections 2023-10-14T10:36:10.099 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.903+0000 7fc42cde9700 1 -- 172.21.15.67:0/3215872597 wait complete. 2023-10-14T10:36:10.101 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.903+0000 7f9ac8929700 1 -- 172.21.15.67:0/1139010823 wait complete. 2023-10-14T10:36:10.101 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.905+0000 7f9ac8929700 1 Processor -- start 2023-10-14T10:36:10.102 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.905+0000 7f9ac8929700 1 -- start start 2023-10-14T10:36:10.102 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.906+0000 7f9ac8929700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9ac40c9fc0 0x7f9ac4146900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:10.102 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.906+0000 7f9ac8929700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f9ac40cff10 0x7f9ac40715e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:10.102 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.906+0000 7f9ac8929700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ac4159fa0 0x7f9ac4071b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:10.102 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.906+0000 7f9ac8929700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac4154480 con 0x7f9ac4159fa0 2023-10-14T10:36:10.103 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.906+0000 7f9ac8929700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac41599c0 con 0x7f9ac40c9fc0 2023-10-14T10:36:10.103 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.906+0000 7f9ac8929700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac4158c00 con 0x7f9ac40cff10 2023-10-14T10:36:10.103 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.906+0000 7f9ac2d9d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ac4159fa0 0x7f9ac4071b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:10.103 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.907+0000 7f9ac2d9d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ac4159fa0 0x7f9ac4071b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44062/0 (socket says 172.21.15.67:44062) 2023-10-14T10:36:10.103 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.907+0000 7f9ac2d9d700 1 -- 172.21.15.67:0/4055383437 learned_addr learned my addr 172.21.15.67:0/4055383437 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:10.104 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.907+0000 7f9ac2d9d700 1 -- 172.21.15.67:0/4055383437 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f9ac40cff10 msgr2=0x7f9ac40715e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-14T10:36:10.104 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.907+0000 7f9ac2d9d700 1 --2- 172.21.15.67:0/4055383437 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f9ac40cff10 0x7f9ac40715e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:10.104 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.907+0000 7f9ac2d9d700 1 -- 172.21.15.67:0/4055383437 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9ac40c9fc0 msgr2=0x7f9ac4146900 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-14T10:36:10.104 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.907+0000 7f9ac2d9d700 1 --2- 172.21.15.67:0/4055383437 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9ac40c9fc0 0x7f9ac4146900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:10.104 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.907+0000 7f9ac2d9d700 1 -- 172.21.15.67:0/4055383437 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ab4011df0 con 0x7f9ac4159fa0 2023-10-14T10:36:10.105 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.907+0000 7f9ac2d9d700 1 --2- 172.21.15.67:0/4055383437 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ac4159fa0 0x7f9ac4071b20 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f9ab40138f0 tx=0x7f9ab4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:10.105 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.907+0000 7f9ab2ffd700 1 -- 172.21.15.67:0/4055383437 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab4010e20 con 0x7f9ac4159fa0 2023-10-14T10:36:10.105 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.907+0000 7f9ab2ffd700 1 -- 172.21.15.67:0/4055383437 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ab40112e0 con 0x7f9ac4159fa0 2023-10-14T10:36:10.105 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.907+0000 7f9ab2ffd700 1 -- 172.21.15.67:0/4055383437 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab4011690 con 0x7f9ac4159fa0 2023-10-14T10:36:10.105 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.907+0000 7f9ac8929700 1 -- 172.21.15.67:0/4055383437 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ac4158ee0 con 0x7f9ac4159fa0 2023-10-14T10:36:10.106 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.908+0000 7f9ac8929700 1 -- 172.21.15.67:0/4055383437 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ac41594a0 con 0x7f9ac4159fa0 2023-10-14T10:36:10.106 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.909+0000 7f9ab2ffd700 1 -- 172.21.15.67:0/4055383437 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f9ab4021070 con 0x7f9ac4159fa0 2023-10-14T10:36:10.106 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.910+0000 7f9ab2ffd700 1 --2- 172.21.15.67:0/4055383437 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f9aa406af50 0x7f9aa406d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:10.106 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.910+0000 7f9ab2ffd700 1 -- 172.21.15.67:0/4055383437 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9ab4092ad0 con 0x7f9ac4159fa0 2023-10-14T10:36:10.106 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.910+0000 7f9ac259c700 1 --2- 172.21.15.67:0/4055383437 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f9aa406af50 0x7f9aa406d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:10.107 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.911+0000 7f9ac259c700 1 --2- 172.21.15.67:0/4055383437 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f9aa406af50 0x7f9aa406d410 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9abc007fd0 tx=0x7f9abc007da0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:10.107 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.913+0000 7f9ac8929700 1 --2- 172.21.15.67:0/4055383437 >> [v2:172.21.15.72:6824/3567465926,v1:172.21.15.72:6825/3567465926] conn(0x7f9ac41522d0 0x7f9ac406e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:10.107 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.913+0000 7f9ac8929700 1 -- 172.21.15.67:0/4055383437 --> [v2:172.21.15.72:6824/3567465926,v1:172.21.15.72:6825/3567465926] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9ac404ebb0 con 0x7f9ac41522d0 2023-10-14T10:36:10.108 INFO:teuthology.orchestra.run.smithi067.stdout:60129542149 2023-10-14T10:36:10.108 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-10-14T10:36:10.109 INFO:teuthology.orchestra.run.smithi067.stdout:60129542151 2023-10-14T10:36:10.110 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-10-14T10:36:10.110 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.915+0000 7f7449ffb700 1 -- 172.21.15.67:0/2709544394 --> [v2:172.21.15.67:6816/2435538787,v1:172.21.15.67:6818/2435538787] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f745c0b4ac0 con 0x7f74400730f0 2023-10-14T10:36:10.110 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.915+0000 7f9ac1d9b700 1 --2- 172.21.15.67:0/4055383437 >> [v2:172.21.15.72:6824/3567465926,v1:172.21.15.72:6825/3567465926] conn(0x7f9ac41522d0 0x7f9ac406e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:10.111 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.915+0000 7f744bfff700 1 -- 172.21.15.67:0/2709544394 <== osd.1 v2:172.21.15.67:6816/2435538787 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f745c0b4ac0 con 0x7f74400730f0 2023-10-14T10:36:10.111 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.915+0000 7f9ac1d9b700 1 --2- 172.21.15.67:0/4055383437 >> [v2:172.21.15.72:6824/3567465926,v1:172.21.15.72:6825/3567465926] conn(0x7f9ac41522d0 0x7f9ac406e400 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:10.112 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.916+0000 7f7449ffb700 1 -- 172.21.15.67:0/2709544394 >> [v2:172.21.15.67:6816/2435538787,v1:172.21.15.67:6818/2435538787] conn(0x7f74400730f0 msgr2=0x7f7440075510 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:10.113 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.916+0000 7f7449ffb700 1 --2- 172.21.15.67:0/2709544394 >> [v2:172.21.15.67:6816/2435538787,v1:172.21.15.67:6818/2435538787] conn(0x7f74400730f0 0x7f7440075510 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:10.113 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.916+0000 7f7449ffb700 1 -- 172.21.15.67:0/2709544394 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f744006af50 msgr2=0x7f744006d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:10.113 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.916+0000 7f7449ffb700 1 --2- 172.21.15.67:0/2709544394 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f744006af50 0x7f744006d410 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7450001560 tx=0x7f7450000f50 comp rx=0 tx=0).stop 2023-10-14T10:36:10.113 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.916+0000 7f7449ffb700 1 -- 172.21.15.67:0/2709544394 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f745c06e430 msgr2=0x7f745c0a84a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:10.113 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.916+0000 7f7449ffb700 1 --2- 172.21.15.67:0/2709544394 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f745c06e430 0x7f745c0a84a0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f745c0cf150 tx=0x7f7454009750 comp rx=0 tx=0).stop 2023-10-14T10:36:10.114 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.916+0000 7f745bfff700 1 -- 172.21.15.67:0/2709544394 reap_dead start 2023-10-14T10:36:10.114 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.916+0000 7f7449ffb700 1 -- 172.21.15.67:0/2709544394 shutdown_connections 2023-10-14T10:36:10.114 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.916+0000 7f7449ffb700 1 -- 172.21.15.67:0/2709544394 >> 172.21.15.67:0/2709544394 conn(0x7f745c0cf3f0 msgr2=0x7f745c0af7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:10.114 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.916+0000 7f7449ffb700 1 -- 172.21.15.67:0/2709544394 shutdown_connections 2023-10-14T10:36:10.114 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.916+0000 7f7449ffb700 1 -- 172.21.15.67:0/2709544394 wait complete. 2023-10-14T10:36:10.115 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.916+0000 7f9ab2ffd700 1 -- 172.21.15.67:0/4055383437 <== osd.7 v2:172.21.15.72:6824/3567465926 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7f9ac404ebb0 con 0x7f9ac41522d0 2023-10-14T10:36:10.116 INFO:teuthology.orchestra.run.smithi067.stdout:60129542150 2023-10-14T10:36:10.116 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-10-14T10:36:10.117 INFO:teuthology.orchestra.run.smithi067.stdout:60129542150 2023-10-14T10:36:10.117 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-14T10:36:10.118 INFO:teuthology.orchestra.run.smithi067.stdout:60129542150 2023-10-14T10:36:10.119 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-14T10:36:10.119 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.940+0000 7f9ac8929700 1 -- 172.21.15.67:0/4055383437 --> [v2:172.21.15.72:6824/3567465926,v1:172.21.15.72:6825/3567465926] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9ac40cd260 con 0x7f9ac41522d0 2023-10-14T10:36:10.119 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.940+0000 7f9ab2ffd700 1 -- 172.21.15.67:0/4055383437 <== osd.7 v2:172.21.15.72:6824/3567465926 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f9ac40cd260 con 0x7f9ac41522d0 2023-10-14T10:36:10.121 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.941+0000 7f9ac8929700 1 -- 172.21.15.67:0/4055383437 >> [v2:172.21.15.72:6824/3567465926,v1:172.21.15.72:6825/3567465926] conn(0x7f9ac41522d0 msgr2=0x7f9ac406e400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:10.121 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.941+0000 7f9ac8929700 1 --2- 172.21.15.67:0/4055383437 >> [v2:172.21.15.72:6824/3567465926,v1:172.21.15.72:6825/3567465926] conn(0x7f9ac41522d0 0x7f9ac406e400 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:10.122 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.941+0000 7f9ac8929700 1 -- 172.21.15.67:0/4055383437 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f9aa406af50 msgr2=0x7f9aa406d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:10.122 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.941+0000 7f9ac8929700 1 --2- 172.21.15.67:0/4055383437 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f9aa406af50 0x7f9aa406d410 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9abc007fd0 tx=0x7f9abc007da0 comp rx=0 tx=0).stop 2023-10-14T10:36:10.122 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.941+0000 7f9ac8929700 1 -- 172.21.15.67:0/4055383437 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ac4159fa0 msgr2=0x7f9ac4071b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:10.122 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.941+0000 7f9ac8929700 1 --2- 172.21.15.67:0/4055383437 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ac4159fa0 0x7f9ac4071b20 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f9ab40138f0 tx=0x7f9ab4004750 comp rx=0 tx=0).stop 2023-10-14T10:36:10.122 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.941+0000 7f9ac2d9d700 1 -- 172.21.15.67:0/4055383437 reap_dead start 2023-10-14T10:36:10.123 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.941+0000 7f9ac8929700 1 -- 172.21.15.67:0/4055383437 shutdown_connections 2023-10-14T10:36:10.123 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.941+0000 7f9ac8929700 1 -- 172.21.15.67:0/4055383437 >> 172.21.15.67:0/4055383437 conn(0x7f9ac40cf3f0 msgr2=0x7f9ac4156ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:10.123 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.941+0000 7f9ac8929700 1 -- 172.21.15.67:0/4055383437 shutdown_connections 2023-10-14T10:36:10.123 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:09.941+0000 7f9ac8929700 1 -- 172.21.15.67:0/4055383437 wait complete. 2023-10-14T10:36:10.123 INFO:teuthology.orchestra.run.smithi067.stdout:60129542150 2023-10-14T10:36:10.124 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-10-14T10:36:10.359 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.357+0000 7f5e6e599700 1 Processor -- start 2023-10-14T10:36:10.359 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.357+0000 7f5e6e599700 1 -- start start 2023-10-14T10:36:10.360 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.358+0000 7f077678a700 1 Processor -- start 2023-10-14T10:36:10.361 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.359+0000 7f5e6e599700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f5e680ca2c0 0x7f5e6806ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:10.362 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.359+0000 7f5e6e599700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e68070e30 con 0x7f5e6806de70 2023-10-14T10:36:10.362 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.359+0000 7f5e6e599700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e68071110 con 0x7f5e68159a20 2023-10-14T10:36:10.362 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.359+0000 7f5e6e599700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e68159e00 con 0x7f5e680ca2c0 2023-10-14T10:36:10.362 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.359+0000 7f5e6cd96700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f5e6806de70 0x7f5e680c9bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:51836/0 (socket says 172.21.15.67:51836) 2023-10-14T10:36:10.363 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.359+0000 7f5e6cd96700 1 -- 172.21.15.67:0/3957774239 learned_addr learned my addr 172.21.15.67:0/3957774239 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:36:10.363 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.359+0000 7f5e5ffff700 1 -- 172.21.15.67:0/3957774239 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2238927023 0 0) 0x7f5e68070e30 con 0x7f5e6806de70 2023-10-14T10:36:10.363 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.359+0000 7f5e5ffff700 1 -- 172.21.15.67:0/3957774239 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e540035b0 con 0x7f5e6806de70 2023-10-14T10:36:10.363 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.359+0000 7fae6b069700 1 Processor -- start 2023-10-14T10:36:10.364 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.358+0000 7f20e6935700 1 Processor -- start 2023-10-14T10:36:10.364 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.358+0000 7f20e6935700 1 -- start start 2023-10-14T10:36:10.364 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.360+0000 7f20e6935700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f20e00ca150 0x7f20e00b1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:10.364 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.360+0000 7f20e6935700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20e00ce710 con 0x7f20e00b0e70 2023-10-14T10:36:10.365 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.360+0000 7f20e6935700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20e0165e40 con 0x7f20e00c27d0 2023-10-14T10:36:10.365 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.360+0000 7f20e6935700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20e00cdbe0 con 0x7f20e00ca150 2023-10-14T10:36:10.365 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.358+0000 7f077678a700 1 -- start start 2023-10-14T10:36:10.366 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.359+0000 7f077678a700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f0770166620 0x7f07700cde60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:10.366 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.359+0000 7f077678a700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f07700ce3a0 0x7f077006c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:10.366 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.359+0000 7f077678a700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07700ca9f0 con 0x7f07700c2840 2023-10-14T10:36:10.366 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.359+0000 7f077678a700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0770162490 con 0x7f0770166620 2023-10-14T10:36:10.366 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.359+0000 7f077678a700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0770166a80 con 0x7f07700ce3a0 2023-10-14T10:36:10.367 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.360+0000 7f20e4ed2700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f20e00ca150 0x7f20e00b1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:10.367 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.360+0000 7f20e4ed2700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f20e00ca150 0x7f20e00b1320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3301/0 says I am v2:172.21.15.67:60358/0 (socket says 172.21.15.67:60358) 2023-10-14T10:36:10.367 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.360+0000 7f20e4ed2700 1 -- 172.21.15.67:0/1199457072 learned_addr learned my addr 172.21.15.67:0/1199457072 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:10.368 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.360+0000 7f076ffff700 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f07700c2840 0x7f07700c9c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.67:60080/0 (socket says 172.21.15.67:60080) 2023-10-14T10:36:10.368 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.360+0000 7f076ffff700 1 -- 172.21.15.67:0/223145796 learned_addr learned my addr 172.21.15.67:0/223145796 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:36:10.368 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.360+0000 7f5e6dd98700 1 --2- 172.21.15.67:0/3957774239 >> v2:172.21.15.72:3300/0 conn(0x7f5e680ca2c0 0x7f5e6806ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:10.369 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.360+0000 7f5e5ffff700 1 -- 172.21.15.67:0/3957774239 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2863509472 0 0) 0x7f5e540035b0 con 0x7f5e6806de70 2023-10-14T10:36:10.369 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.360+0000 7f5e5ffff700 1 -- 172.21.15.67:0/3957774239 >> v2:172.21.15.72:3300/0 conn(0x7f5e680ca2c0 msgr2=0x7f5e6806ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:10.369 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.360+0000 7f5e5ffff700 1 --2- 172.21.15.67:0/3957774239 >> v2:172.21.15.72:3300/0 conn(0x7f5e680ca2c0 0x7f5e6806ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:10.369 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.360+0000 7f5e5ffff700 1 -- 172.21.15.67:0/3957774239 >> v1:172.21.15.72:6789/0 conn(0x7f5e68159a20 legacy=0x7f5e680ccd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:10.370 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.360+0000 7f5e5ffff700 1 -- 172.21.15.67:0/3957774239 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e680713f0 con 0x7f5e6806de70 2023-10-14T10:36:10.370 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.360+0000 7f5e5ffff700 1 -- 172.21.15.67:0/3957774239 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3180043755 0 0) 0x7f5e58004d40 con 0x7f5e6806de70 2023-10-14T10:36:10.370 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.360+0000 7f5e5ffff700 1 -- 172.21.15.67:0/3957774239 >> v1:172.21.15.67:6789/0 conn(0x7f5e6806de70 legacy=0x7f5e680c9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:10.371 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.360+0000 7f5e5ffff700 1 --2- 172.21.15.67:0/3957774239 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f5e54003a00 0x7f5e54003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:10.371 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.360+0000 7f5e5ffff700 1 --2- 172.21.15.67:0/3957774239 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5e54004850 0x7f5e54004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:10.371 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.360+0000 7f5e5ffff700 1 --2- 172.21.15.67:0/3957774239 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5e540052e0 0x7f5e5400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:10.371 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.360+0000 7f5e5ffff700 1 -- 172.21.15.67:0/3957774239 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e68159e00 con 0x7f5e54004850 2023-10-14T10:36:10.371 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.360+0000 7f5e5ffff700 1 -- 172.21.15.67:0/3957774239 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5400c230 con 0x7f5e540052e0 2023-10-14T10:36:10.372 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.360+0000 7f5e5ffff700 1 -- 172.21.15.67:0/3957774239 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5400c3f0 con 0x7f5e54003a00 2023-10-14T10:36:10.372 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.360+0000 7f0774d27700 1 --2- 172.21.15.67:0/223145796 >> v2:172.21.15.72:3300/0 conn(0x7f07700ce3a0 0x7f077006c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:10.372 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.360+0000 7f076effd700 1 -- 172.21.15.67:0/223145796 <== mon.1 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4190643681 0 0) 0x7f07700ca9f0 con 0x7f07700c2840 2023-10-14T10:36:10.372 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.360+0000 7f076effd700 1 -- 172.21.15.67:0/223145796 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07580035b0 con 0x7f07700c2840 2023-10-14T10:36:10.373 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.361+0000 7f076effd700 1 -- 172.21.15.67:0/223145796 <== mon.1 v1:172.21.15.72:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1370157746 0 0) 0x7f07580035b0 con 0x7f07700c2840 2023-10-14T10:36:10.373 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.361+0000 7f076effd700 1 -- 172.21.15.67:0/223145796 >> v2:172.21.15.72:3300/0 conn(0x7f07700ce3a0 msgr2=0x7f077006c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:10.373 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.361+0000 7f076effd700 1 --2- 172.21.15.67:0/223145796 >> v2:172.21.15.72:3300/0 conn(0x7f07700ce3a0 0x7f077006c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:10.373 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.361+0000 7f076f7fe700 1 --2- 172.21.15.67:0/223145796 >> v2:172.21.15.67:3300/0 conn(0x7f0770166620 0x7f07700cde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:10.373 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.361+0000 7f076effd700 1 -- 172.21.15.67:0/223145796 >> v2:172.21.15.67:3300/0 conn(0x7f0770166620 msgr2=0x7f07700cde60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:10.374 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.361+0000 7f076effd700 1 --2- 172.21.15.67:0/223145796 >> v2:172.21.15.67:3300/0 conn(0x7f0770166620 0x7f07700cde60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:10.374 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.361+0000 7f076effd700 1 -- 172.21.15.67:0/223145796 --> v1:172.21.15.72:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07700ca430 con 0x7f07700c2840 2023-10-14T10:36:10.374 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.361+0000 7f076effd700 1 -- 172.21.15.67:0/223145796 <== mon.1 v1:172.21.15.72:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3180043755 0 0) 0x7f07680019f0 con 0x7f07700c2840 2023-10-14T10:36:10.374 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.361+0000 7f076effd700 1 -- 172.21.15.67:0/223145796 >> v1:172.21.15.72:6789/0 conn(0x7f07700c2840 legacy=0x7f07700c9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:10.375 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.361+0000 7f076effd700 1 --2- 172.21.15.67:0/223145796 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0758003a30 0x7f0758003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:10.375 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.361+0000 7f076effd700 1 --2- 172.21.15.67:0/223145796 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0758004850 0x7f0758004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:10.375 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.361+0000 7f076effd700 1 --2- 172.21.15.67:0/223145796 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f0758005210 0x7f075800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:10.375 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.361+0000 7f076effd700 1 -- 172.21.15.67:0/223145796 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0770162490 con 0x7f0758003a30 2023-10-14T10:36:10.375 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.361+0000 7f076effd700 1 -- 172.21.15.67:0/223145796 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0770166a80 con 0x7f0758004850 2023-10-14T10:36:10.375 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.361+0000 7f076effd700 1 -- 172.21.15.67:0/223145796 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f075800c140 con 0x7f0758005210 2023-10-14T10:36:10.376 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.361+0000 7f076f7fe700 1 --2- 172.21.15.67:0/223145796 >> v2:172.21.15.67:3300/0 conn(0x7f0770166620 0x7f07700cde60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-14T10:36:10.376 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.361+0000 7f076f7fe700 1 --2- 172.21.15.67:0/223145796 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0758004850 0x7f0758004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:10.376 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.361+0000 7f0774d27700 1 --2- 172.21.15.67:0/223145796 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f0758005210 0x7f075800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:10.377 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.359+0000 7fae6b069700 1 -- start start 2023-10-14T10:36:10.377 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.360+0000 7fae6b069700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fae64157250 0x7fae6415df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:10.377 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.360+0000 7fae6b069700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fae640c9fc0 0x7fae64159fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:10.377 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.360+0000 7fae6b069700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae6415af20 con 0x7fae6415a4e0 2023-10-14T10:36:10.377 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.360+0000 7fae6b069700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae6415e470 con 0x7fae640c9fc0 2023-10-14T10:36:10.378 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.361+0000 7fae6b069700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae6415a960 con 0x7fae64157250 2023-10-14T10:36:10.378 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.361+0000 7fae63fff700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fae640c9fc0 0x7fae64159fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:10.378 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.361+0000 7fae69606700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fae6415a4e0 0x7fae6406c9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:51858/0 (socket says 172.21.15.67:51858) 2023-10-14T10:36:10.378 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.361+0000 7fae69606700 1 -- 172.21.15.67:0/2572801647 learned_addr learned my addr 172.21.15.67:0/2572801647 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:36:10.378 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.361+0000 7fae63fff700 1 -- 172.21.15.67:0/2572801647 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fae64157250 msgr2=0x7fae6415df30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-14T10:36:10.379 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.361+0000 7fae63fff700 1 --2- 172.21.15.67:0/2572801647 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fae64157250 0x7fae6415df30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:10.379 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.361+0000 7fae63fff700 1 -- 172.21.15.67:0/2572801647 >> v1:172.21.15.67:6789/0 conn(0x7fae6415a4e0 legacy=0x7fae6406c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:10.379 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.361+0000 7fae63fff700 1 -- 172.21.15.67:0/2572801647 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae6415ac40 con 0x7fae640c9fc0 2023-10-14T10:36:10.379 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.360+0000 7f20e4ed2700 1 -- 172.21.15.67:0/1199457072 >> v1:172.21.15.72:6789/0 conn(0x7f20e00c27d0 legacy=0x7f20e00ce000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:10.380 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.360+0000 7f20e4ed2700 1 -- 172.21.15.67:0/1199457072 >> v1:172.21.15.67:6789/0 conn(0x7f20e00b0e70 legacy=0x7f20e00c9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:10.380 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.360+0000 7f20e4ed2700 1 -- 172.21.15.67:0/1199457072 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20e00cd560 con 0x7f20e00ca150 2023-10-14T10:36:10.380 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.361+0000 7f20e4ed2700 1 --2- 172.21.15.67:0/1199457072 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f20e00ca150 0x7f20e00b1320 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f20d8009e80 tx=0x7f20d800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0343c4429f4002e server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:10.381 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.362+0000 7f076f7fe700 1 -- 172.21.15.67:0/223145796 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f0758005210 msgr2=0x7f075800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:10.381 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.362+0000 7f076f7fe700 1 --2- 172.21.15.67:0/223145796 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f0758005210 0x7f075800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:10.381 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.362+0000 7f076f7fe700 1 -- 172.21.15.67:0/223145796 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0758003a30 msgr2=0x7f0758003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:10.381 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.362+0000 7f076f7fe700 1 --2- 172.21.15.67:0/223145796 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0758003a30 0x7f0758003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:10.381 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.362+0000 7f076f7fe700 1 -- 172.21.15.67:0/223145796 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07700ca430 con 0x7f0758004850 2023-10-14T10:36:10.382 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.362+0000 7f5e6dd98700 1 --2- 172.21.15.67:0/3957774239 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5e540052e0 0x7f5e5400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:10.382 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.361+0000 7f20deffd700 1 -- 172.21.15.67:0/1199457072 <== mon.2 v2:172.21.15.72:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d800e070 con 0x7f20e00ca150 2023-10-14T10:36:10.382 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.361+0000 7f20deffd700 1 -- 172.21.15.67:0/1199457072 <== mon.2 v2:172.21.15.72:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20d8003a20 con 0x7f20e00ca150 2023-10-14T10:36:10.383 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.361+0000 7f20deffd700 1 -- 172.21.15.67:0/1199457072 <== mon.2 v2:172.21.15.72:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d8003dd0 con 0x7f20e00ca150 2023-10-14T10:36:10.383 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.362+0000 7f076f7fe700 1 --2- 172.21.15.67:0/223145796 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0758004850 0x7f0758004cd0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f076400ab20 tx=0x7f0758005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db0922d8fd7c8ee4 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:10.383 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.362+0000 7f5e6d597700 1 --2- 172.21.15.67:0/3957774239 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f5e54003a00 0x7f5e54003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:10.384 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.362+0000 7f5e6cd96700 1 --2- 172.21.15.67:0/3957774239 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5e54004850 0x7f5e54004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:10.384 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.362+0000 7f8d4cddd700 1 Processor -- start 2023-10-14T10:36:10.385 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.362+0000 7f076effd700 1 -- 172.21.15.67:0/223145796 <== mon.1 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f076400c880 con 0x7f0758004850 2023-10-14T10:36:10.385 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.362+0000 7f076effd700 1 -- 172.21.15.67:0/223145796 <== mon.1 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f076400c9e0 con 0x7f0758004850 2023-10-14T10:36:10.385 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.362+0000 7f076effd700 1 -- 172.21.15.67:0/223145796 <== mon.1 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f076400cd50 con 0x7f0758004850 2023-10-14T10:36:10.385 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.362+0000 7f0774d27700 1 -- 172.21.15.67:0/223145796 reap_dead start 2023-10-14T10:36:10.386 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.363+0000 7f077678a700 1 -- 172.21.15.67:0/223145796 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0758004850 msgr2=0x7f0758004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:10.386 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.363+0000 7f077678a700 1 --2- 172.21.15.67:0/223145796 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0758004850 0x7f0758004cd0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f076400ab20 tx=0x7f0758005880 comp rx=0 tx=0).stop 2023-10-14T10:36:10.386 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.362+0000 7f414c6e7700 1 Processor -- start 2023-10-14T10:36:10.387 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.362+0000 7fae63fff700 1 --2- 172.21.15.67:0/2572801647 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fae640c9fc0 0x7fae64159fa0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fae5c00bfa0 tx=0x7fae5c00d330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db0dd9536839fdd2 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:10.387 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.363+0000 7fae637fe700 1 -- 172.21.15.67:0/2572801647 <== mon.1 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae5c010070 con 0x7fae640c9fc0 2023-10-14T10:36:10.387 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.363+0000 7fae637fe700 1 -- 172.21.15.67:0/2572801647 <== mon.1 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae5c00b2e0 con 0x7fae640c9fc0 2023-10-14T10:36:10.387 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.363+0000 7fae637fe700 1 -- 172.21.15.67:0/2572801647 <== mon.1 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae5c00dcf0 con 0x7fae640c9fc0 2023-10-14T10:36:10.388 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.362+0000 7f20e6935700 1 -- 172.21.15.67:0/1199457072 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f20e00ca150 msgr2=0x7f20e00b1320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:10.388 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.362+0000 7f20e6935700 1 --2- 172.21.15.67:0/1199457072 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f20e00ca150 0x7f20e00b1320 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f20d8009e80 tx=0x7f20d800b330 comp rx=0 tx=0).stop 2023-10-14T10:36:10.388 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.362+0000 7f5e6d597700 1 -- 172.21.15.67:0/3957774239 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5e540052e0 msgr2=0x7f5e5400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:10.389 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.362+0000 7f5e6d597700 1 --2- 172.21.15.67:0/3957774239 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5e540052e0 0x7f5e5400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:10.389 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.362+0000 7f5e6d597700 1 -- 172.21.15.67:0/3957774239 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5e54004850 msgr2=0x7f5e54004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:10.389 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.362+0000 7f5e6d597700 1 --2- 172.21.15.67:0/3957774239 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5e54004850 0x7f5e54004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:10.389 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.362+0000 7f5e6d597700 1 -- 172.21.15.67:0/3957774239 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e680713f0 con 0x7f5e54003a00 2023-10-14T10:36:10.389 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.362+0000 7f5e6cd96700 1 --2- 172.21.15.67:0/3957774239 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5e54004850 0x7f5e54004cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-14T10:36:10.390 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.363+0000 7f5e6dd98700 1 -- 172.21.15.67:0/3957774239 reap_dead start 2023-10-14T10:36:10.390 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.363+0000 7f5e6d597700 1 --2- 172.21.15.67:0/3957774239 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f5e54003a00 0x7f5e54003e00 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f5e640016c0 tx=0x7f5e54005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea9bf8f2217d8ebd server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:10.390 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.364+0000 7fae6b069700 1 -- 172.21.15.67:0/2572801647 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fae640c9fc0 msgr2=0x7fae64159fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:10.391 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.364+0000 7fae6b069700 1 --2- 172.21.15.67:0/2572801647 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fae640c9fc0 0x7fae64159fa0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fae5c00bfa0 tx=0x7fae5c00d330 comp rx=0 tx=0).stop 2023-10-14T10:36:10.391 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.364+0000 7f077678a700 1 -- 172.21.15.67:0/223145796 shutdown_connections 2023-10-14T10:36:10.391 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.364+0000 7f077678a700 1 --2- 172.21.15.67:0/223145796 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0758004850 0x7f0758004cd0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:10.392 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.364+0000 7f077678a700 1 -- 172.21.15.67:0/223145796 >> 172.21.15.67:0/223145796 conn(0x7f07700cf3f0 msgr2=0x7f07700cf800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:10.392 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.364+0000 7f5e5ffff700 1 -- 172.21.15.67:0/3957774239 <== mon.2 v2:172.21.15.72:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e640052d0 con 0x7f5e54003a00 2023-10-14T10:36:10.392 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.364+0000 7f71055c8700 1 Processor -- start 2023-10-14T10:36:10.393 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.364+0000 7f71055c8700 1 -- start start 2023-10-14T10:36:10.393 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.364+0000 7f5e5ffff700 1 -- 172.21.15.67:0/3957774239 <== mon.2 v2:172.21.15.72:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e64005430 con 0x7f5e54003a00 2023-10-14T10:36:10.393 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.364+0000 7f5e5ffff700 1 -- 172.21.15.67:0/3957774239 <== mon.2 v2:172.21.15.72:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e640038c0 con 0x7f5e54003a00 2023-10-14T10:36:10.394 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.364+0000 7f077678a700 1 -- 172.21.15.67:0/223145796 shutdown_connections 2023-10-14T10:36:10.394 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.362+0000 7f8d4cddd700 1 -- start start 2023-10-14T10:36:10.395 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.364+0000 7f8d4cddd700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f8d480c27d0 0x7f8d480ce000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:10.395 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.364+0000 7f8d4cddd700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f8d480b0e70 0x7f8d480b1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:10.395 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.364+0000 7f8d4cddd700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f8d480c9a40 0x7f8d480c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:10.395 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.364+0000 7f8d4cddd700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d480cdbe0 con 0x7f8d480c27d0 2023-10-14T10:36:10.396 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.364+0000 7f8d4cddd700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d480b1860 con 0x7f8d480b0e70 2023-10-14T10:36:10.396 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.364+0000 7f8d4cddd700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d480ca410 con 0x7f8d480c9a40 2023-10-14T10:36:10.396 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.364+0000 7f8d45d9b700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f8d480b0e70 0x7f8d480b1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:10.396 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.364+0000 7f8d45d9b700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f8d480b0e70 0x7f8d480b1320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.67:38896/0 (socket says 172.21.15.67:38896) 2023-10-14T10:36:10.396 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.364+0000 7f8d45d9b700 1 -- 172.21.15.67:0/37524163 learned_addr learned my addr 172.21.15.67:0/37524163 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:10.397 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.364+0000 7f077678a700 1 -- 172.21.15.67:0/223145796 wait complete. 2023-10-14T10:36:10.397 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.364+0000 7f8d46d9d700 1 --2- 172.21.15.67:0/37524163 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f8d480c9a40 0x7f8d480c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:10.398 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.364+0000 7f20e6935700 1 -- 172.21.15.67:0/1199457072 shutdown_connections 2023-10-14T10:36:10.398 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.364+0000 7f20e6935700 1 --2- 172.21.15.67:0/1199457072 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f20e00ca150 0x7f20e00b1320 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:10.398 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.364+0000 7f20e6935700 1 -- 172.21.15.67:0/1199457072 >> 172.21.15.67:0/1199457072 conn(0x7f20e006afb0 msgr2=0x7f20e00cbe70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:10.399 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.365+0000 7fae6b069700 1 -- 172.21.15.67:0/2572801647 shutdown_connections 2023-10-14T10:36:10.399 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.365+0000 7fae6b069700 1 --2- 172.21.15.67:0/2572801647 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fae64157250 0x7fae6415df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:10.399 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.365+0000 7fae6b069700 1 --2- 172.21.15.67:0/2572801647 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fae640c9fc0 0x7fae64159fa0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:10.399 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.365+0000 7fae6b069700 1 -- 172.21.15.67:0/2572801647 >> 172.21.15.67:0/2572801647 conn(0x7fae640cf3f0 msgr2=0x7fae640cf800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:10.400 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.365+0000 7f20e6935700 1 -- 172.21.15.67:0/1199457072 shutdown_connections 2023-10-14T10:36:10.400 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.363+0000 7f414c6e7700 1 -- start start 2023-10-14T10:36:10.401 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.364+0000 7f414c6e7700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f414415dac0 0x7f41440ce770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:10.401 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.364+0000 7f414c6e7700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f414415e9d0 con 0x7f41440cecb0 2023-10-14T10:36:10.401 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.364+0000 7f414c6e7700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41440ce2e0 con 0x7f4144153ff0 2023-10-14T10:36:10.401 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.364+0000 7f414c6e7700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f414415e410 con 0x7f414415dac0 2023-10-14T10:36:10.402 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.364+0000 7f4149c82700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f414415dac0 0x7f41440ce770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:10.402 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.365+0000 7f4149c82700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f414415dac0 0x7f41440ce770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44110/0 (socket says 172.21.15.67:44110) 2023-10-14T10:36:10.402 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.365+0000 7f4149c82700 1 -- 172.21.15.67:0/2008625724 learned_addr learned my addr 172.21.15.67:0/2008625724 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:10.403 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.365+0000 7f8d46d9d700 1 -- 172.21.15.67:0/37524163 >> v2:172.21.15.72:3300/0 conn(0x7f8d480b0e70 msgr2=0x7f8d480b1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:10.403 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.365+0000 7f8d46d9d700 1 --2- 172.21.15.67:0/37524163 >> v2:172.21.15.72:3300/0 conn(0x7f8d480b0e70 0x7f8d480b1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:10.403 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.365+0000 7f8d46d9d700 1 -- 172.21.15.67:0/37524163 >> v2:172.21.15.67:3300/0 conn(0x7f8d480c27d0 msgr2=0x7f8d480ce000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:10.403 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.365+0000 7f8d46d9d700 1 --2- 172.21.15.67:0/37524163 >> v2:172.21.15.67:3300/0 conn(0x7f8d480c27d0 0x7f8d480ce000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:10.404 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.365+0000 7f8d46d9d700 1 -- 172.21.15.67:0/37524163 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d48165e40 con 0x7f8d480c9a40 2023-10-14T10:36:10.404 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.365+0000 7f20e6935700 1 -- 172.21.15.67:0/1199457072 wait complete. 2023-10-14T10:36:10.404 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.365+0000 7f71055c8700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f71000cdfd0 0x7f71000c9c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:10.405 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.365+0000 7f71055c8700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f710016edf0 0x7f710006e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:10.405 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.365+0000 7f71055c8700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71000ca5d0 con 0x7f7100072430 2023-10-14T10:36:10.405 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.365+0000 7f71055c8700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71000ceae0 con 0x7f71000cdfd0 2023-10-14T10:36:10.405 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.365+0000 7f71055c8700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71000cf280 con 0x7f710016edf0 2023-10-14T10:36:10.405 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.365+0000 7f5e6e599700 1 -- 172.21.15.67:0/3957774239 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f5e54003a00 msgr2=0x7f5e54003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:10.406 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.365+0000 7f5e6e599700 1 --2- 172.21.15.67:0/3957774239 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f5e54003a00 0x7f5e54003e00 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f5e640016c0 tx=0x7f5e54005880 comp rx=0 tx=0).stop 2023-10-14T10:36:10.406 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.365+0000 7fae6b069700 1 -- 172.21.15.67:0/2572801647 shutdown_connections 2023-10-14T10:36:10.406 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.366+0000 7f4149c82700 1 -- 172.21.15.67:0/2008625724 >> v1:172.21.15.72:6789/0 conn(0x7f4144153ff0 legacy=0x7f41441543d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-14T10:36:10.407 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.366+0000 7f4149c82700 1 -- 172.21.15.67:0/2008625724 >> v1:172.21.15.67:6789/0 conn(0x7f41440cecb0 legacy=0x7f41440b0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:10.407 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.366+0000 7f4149c82700 1 -- 172.21.15.67:0/2008625724 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f414415e6f0 con 0x7f414415dac0 2023-10-14T10:36:10.407 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.366+0000 7fae6b069700 1 -- 172.21.15.67:0/2572801647 wait complete. 2023-10-14T10:36:10.408 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.366+0000 7f4149c82700 1 --2- 172.21.15.67:0/2008625724 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f414415dac0 0x7f41440ce770 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f413c009d40 tx=0x7f413c00c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d08a5dc515fe5058 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:10.408 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.366+0000 7f70fe7fc700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f71000cdfd0 0x7f71000c9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:10.408 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.366+0000 7f70feffd700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f7100072430 0x7f710006def0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:51884/0 (socket says 172.21.15.67:51884) 2023-10-14T10:36:10.409 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.366+0000 7f70feffd700 1 -- 172.21.15.67:0/1510343540 learned_addr learned my addr 172.21.15.67:0/1510343540 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:36:10.409 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.366+0000 7f077678a700 1 Processor -- start 2023-10-14T10:36:10.409 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.366+0000 7f5e6e599700 1 -- 172.21.15.67:0/3957774239 shutdown_connections 2023-10-14T10:36:10.410 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.366+0000 7f5e6e599700 1 --2- 172.21.15.67:0/3957774239 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f5e54003a00 0x7f5e54003e00 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:10.410 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.366+0000 7f5e6e599700 1 -- 172.21.15.67:0/3957774239 >> 172.21.15.67:0/3957774239 conn(0x7f5e680cf1c0 msgr2=0x7f5e680cf5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:10.410 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.366+0000 7f70fdffb700 1 -- 172.21.15.67:0/1510343540 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2972615720 0 0) 0x7f71000ca5d0 con 0x7f7100072430 2023-10-14T10:36:10.411 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.366+0000 7f70fdffb700 1 -- 172.21.15.67:0/1510343540 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70e80057c0 con 0x7f7100072430 2023-10-14T10:36:10.411 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.366+0000 7f4149481700 1 -- 172.21.15.67:0/2008625724 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f413c009e90 con 0x7f414415dac0 2023-10-14T10:36:10.411 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.366+0000 7f4149481700 1 -- 172.21.15.67:0/2008625724 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f413c00ca90 con 0x7f414415dac0 2023-10-14T10:36:10.411 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.366+0000 7f4149481700 1 -- 172.21.15.67:0/2008625724 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f413c00ce40 con 0x7f414415dac0 2023-10-14T10:36:10.412 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.367+0000 7f70fdffb700 1 -- 172.21.15.67:0/1510343540 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 322822333 0 0) 0x7f70e80057c0 con 0x7f7100072430 2023-10-14T10:36:10.412 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.367+0000 7f70fdffb700 1 -- 172.21.15.67:0/1510343540 >> v2:172.21.15.72:3300/0 conn(0x7f710016edf0 msgr2=0x7f710006e600 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-14T10:36:10.412 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.367+0000 7f70fdffb700 1 --2- 172.21.15.67:0/1510343540 >> v2:172.21.15.72:3300/0 conn(0x7f710016edf0 0x7f710006e600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:10.413 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.367+0000 7f70fdffb700 1 -- 172.21.15.67:0/1510343540 >> v2:172.21.15.67:3300/0 conn(0x7f71000cdfd0 msgr2=0x7f71000c9c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:10.413 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.367+0000 7f70fdffb700 1 --2- 172.21.15.67:0/1510343540 >> v2:172.21.15.67:3300/0 conn(0x7f71000cdfd0 0x7f71000c9c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:10.413 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.367+0000 7f70fdffb700 1 -- 172.21.15.67:0/1510343540 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f710016f2a0 con 0x7f7100072430 2023-10-14T10:36:10.413 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.367+0000 7f70fdffb700 1 -- 172.21.15.67:0/1510343540 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3180043755 0 0) 0x7f70f0001670 con 0x7f7100072430 2023-10-14T10:36:10.413 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.367+0000 7f70fdffb700 1 -- 172.21.15.67:0/1510343540 >> v1:172.21.15.67:6789/0 conn(0x7f7100072430 legacy=0x7f710006def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:10.414 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.367+0000 7f20e6935700 1 Processor -- start 2023-10-14T10:36:10.414 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.367+0000 7f70fdffb700 1 --2- 172.21.15.67:0/1510343540 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f70e8005c10 0x7f70e8006030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:10.415 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.367+0000 7f70fdffb700 1 --2- 172.21.15.67:0/1510343540 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f70e8006a60 0x7f70e8006ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:10.415 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.367+0000 7f70fdffb700 1 --2- 172.21.15.67:0/1510343540 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f70e8007420 0x7f70e800dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:10.415 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.367+0000 7f70fdffb700 1 -- 172.21.15.67:0/1510343540 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71000ceae0 con 0x7f70e8007420 2023-10-14T10:36:10.415 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.367+0000 7f8d46d9d700 1 --2- 172.21.15.67:0/37524163 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f8d480c9a40 0x7f8d480c9ed0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f8d480ccba0 tx=0x7f8d38009b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6d64a1061da50f2 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:10.416 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.367+0000 7f70fdffb700 1 -- 172.21.15.67:0/1510343540 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71000cf280 con 0x7f70e8006a60 2023-10-14T10:36:10.416 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.367+0000 7f70fdffb700 1 -- 172.21.15.67:0/1510343540 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70e8003630 con 0x7f70e8005c10 2023-10-14T10:36:10.416 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.367+0000 7f5e6e599700 1 -- 172.21.15.67:0/3957774239 shutdown_connections 2023-10-14T10:36:10.417 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.366+0000 7f077678a700 1 -- start start 2023-10-14T10:36:10.417 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.367+0000 7f077678a700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0758004850 0x7f07700b0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:10.418 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.367+0000 7f077678a700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f07700b1fe0 0x7f07700b2440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:10.418 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.367+0000 7f077678a700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f07700b3d80 0x7f07700b4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:10.418 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.367+0000 7f077678a700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07701629f0 con 0x7f0758004850 2023-10-14T10:36:10.418 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.367+0000 7f077678a700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07700b2aa0 con 0x7f07700b1fe0 2023-10-14T10:36:10.418 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.367+0000 7f077678a700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07700b2d80 con 0x7f07700b3d80 2023-10-14T10:36:10.419 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.367+0000 7fae6b069700 1 Processor -- start 2023-10-14T10:36:10.419 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.367+0000 7f414c6e7700 1 -- 172.21.15.67:0/2008625724 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f414415dac0 msgr2=0x7f41440ce770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:10.419 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.367+0000 7f414c6e7700 1 --2- 172.21.15.67:0/2008625724 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f414415dac0 0x7f41440ce770 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f413c009d40 tx=0x7f413c00c440 comp rx=0 tx=0).stop 2023-10-14T10:36:10.420 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.368+0000 7f414c6e7700 1 -- 172.21.15.67:0/2008625724 shutdown_connections 2023-10-14T10:36:10.420 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.368+0000 7f414c6e7700 1 --2- 172.21.15.67:0/2008625724 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f414415dac0 0x7f41440ce770 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:10.420 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.368+0000 7f70feffd700 1 --2- 172.21.15.67:0/1510343540 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f70e8005c10 0x7f70e8006030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:10.421 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.368+0000 7f414c6e7700 1 -- 172.21.15.67:0/2008625724 >> 172.21.15.67:0/2008625724 conn(0x7f414406afb0 msgr2=0x7f41440cbe70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:10.421 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.368+0000 7fae6b069700 1 -- start start 2023-10-14T10:36:10.422 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.368+0000 7f076f7fe700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f07700b1fe0 0x7f07700b2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:10.422 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.368+0000 7f0774d27700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f07700b3d80 0x7f07700b4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:10.422 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.368+0000 7f076f7fe700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f07700b1fe0 0x7f07700b2440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.67:38916/0 (socket says 172.21.15.67:38916) 2023-10-14T10:36:10.422 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.368+0000 7f076f7fe700 1 -- 172.21.15.67:0/1179860214 learned_addr learned my addr 172.21.15.67:0/1179860214 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:10.422 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.368+0000 7f076f7fe700 1 -- 172.21.15.67:0/1179860214 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f07700b3d80 msgr2=0x7f07700b4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:10.423 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.368+0000 7f076f7fe700 1 --2- 172.21.15.67:0/1179860214 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f07700b3d80 0x7f07700b4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:10.423 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.368+0000 7f076f7fe700 1 -- 172.21.15.67:0/1179860214 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0758004850 msgr2=0x7f07700b0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:10.423 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.368+0000 7f076f7fe700 1 --2- 172.21.15.67:0/1179860214 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0758004850 0x7f07700b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:10.423 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.368+0000 7f076f7fe700 1 -- 172.21.15.67:0/1179860214 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07700b3060 con 0x7f07700b1fe0 2023-10-14T10:36:10.423 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.368+0000 7f076f7fe700 1 --2- 172.21.15.67:0/1179860214 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f07700b1fe0 0x7f07700b2440 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f075800c110 tx=0x7f07640047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:10.424 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.368+0000 7f70fe7fc700 1 --2- 172.21.15.67:0/1510343540 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f70e8006a60 0x7f70e8006ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:10.424 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.369+0000 7f414c6e7700 1 -- 172.21.15.67:0/2008625724 shutdown_connections 2023-10-14T10:36:10.425 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.368+0000 7f076cff9700 1 -- 172.21.15.67:0/1179860214 <== mon.1 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f076401f070 con 0x7f07700b1fe0 2023-10-14T10:36:10.425 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.369+0000 7f076cff9700 1 -- 172.21.15.67:0/1179860214 <== mon.1 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f076401aac0 con 0x7f07700b1fe0 2023-10-14T10:36:10.425 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.369+0000 7f076cff9700 1 -- 172.21.15.67:0/1179860214 <== mon.1 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f076401ae70 con 0x7f07700b1fe0 2023-10-14T10:36:10.426 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.368+0000 7f5e6e599700 1 -- 172.21.15.67:0/3957774239 wait complete. 2023-10-14T10:36:10.426 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.369+0000 7f5e6e599700 1 Processor -- start 2023-10-14T10:36:10.426 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.369+0000 7f077678a700 1 -- 172.21.15.67:0/1179860214 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0770162610 con 0x7f07700b1fe0 2023-10-14T10:36:10.427 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.369+0000 7f077678a700 1 -- 172.21.15.67:0/1179860214 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07700b0840 con 0x7f07700b1fe0 2023-10-14T10:36:10.427 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.369+0000 7f414c6e7700 1 -- 172.21.15.67:0/2008625724 wait complete. 2023-10-14T10:36:10.427 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.369+0000 7fae6b069700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fae640c9fc0 0x7fae64155db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:10.428 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.369+0000 7fae6b069700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fae64157250 0x7fae64071fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:10.428 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.369+0000 7fae6b069700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fae6415a4e0 0x7fae640724e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:10.428 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.369+0000 7fae6b069700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae640c50d0 con 0x7fae64157250 2023-10-14T10:36:10.428 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.369+0000 7fae6b069700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae64159500 con 0x7fae640c9fc0 2023-10-14T10:36:10.428 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.369+0000 7fae6b069700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae640c4730 con 0x7fae6415a4e0 2023-10-14T10:36:10.429 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.370+0000 7fae69606700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fae6415a4e0 0x7fae640724e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:10.429 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.370+0000 7fae69606700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fae6415a4e0 0x7fae640724e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3301/0 says I am v2:172.21.15.67:60424/0 (socket says 172.21.15.67:60424) 2023-10-14T10:36:10.429 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.370+0000 7fae69606700 1 -- 172.21.15.67:0/3055713902 learned_addr learned my addr 172.21.15.67:0/3055713902 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:10.429 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.370+0000 7fae68e05700 1 --2- 172.21.15.67:0/3055713902 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fae640c9fc0 0x7fae64155db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:10.430 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.370+0000 7f70fe7fc700 1 -- 172.21.15.67:0/1510343540 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f70e8005c10 msgr2=0x7f70e8006030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:10.430 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.370+0000 7f70fe7fc700 1 --2- 172.21.15.67:0/1510343540 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f70e8005c10 0x7f70e8006030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:10.430 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.370+0000 7f70fe7fc700 1 -- 172.21.15.67:0/1510343540 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f70e8007420 msgr2=0x7f70e800dc20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-14T10:36:10.430 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.370+0000 7f70fe7fc700 1 --2- 172.21.15.67:0/1510343540 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f70e8007420 0x7f70e800dc20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:10.431 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.370+0000 7f70fe7fc700 1 -- 172.21.15.67:0/1510343540 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f710016f2a0 con 0x7f70e8006a60 2023-10-14T10:36:10.431 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.370+0000 7f70fe7fc700 1 --2- 172.21.15.67:0/1510343540 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f70e8006a60 0x7f70e8006ee0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f70f4009b60 tx=0x7f70e8007a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10036ed962b7a973 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:10.431 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.367+0000 7f20e6935700 1 -- start start 2023-10-14T10:36:10.431 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.368+0000 7f20e6935700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f20e00b0e70 0x7f20e00c8730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:10.432 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.368+0000 7f20e6935700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20e00c27d0 0x7f20e01549e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:10.432 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.368+0000 7f20e6935700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f20e00ca150 0x7f20e01ebd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:10.432 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.369+0000 7f20e6935700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20e00b1860 con 0x7f20e00c27d0 2023-10-14T10:36:10.432 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.369+0000 7f20e6935700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20e0154f20 con 0x7f20e00ca150 2023-10-14T10:36:10.432 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.369+0000 7f20e6935700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20e00c7690 con 0x7f20e00b0e70 2023-10-14T10:36:10.432 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.369+0000 7f20dffff700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f20e00b0e70 0x7f20e00c8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:10.433 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.369+0000 7f20df7fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20e00c27d0 0x7f20e01549e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:10.433 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.369+0000 7f20df7fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20e00c27d0 0x7f20e01549e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44132/0 (socket says 172.21.15.67:44132) 2023-10-14T10:36:10.433 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.369+0000 7f20df7fe700 1 -- 172.21.15.67:0/1796285297 learned_addr learned my addr 172.21.15.67:0/1796285297 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:10.433 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.369+0000 7f20e4ed2700 1 --2- 172.21.15.67:0/1796285297 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f20e00ca150 0x7f20e01ebd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:10.433 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.370+0000 7f20e4ed2700 1 -- 172.21.15.67:0/1796285297 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f20e00b0e70 msgr2=0x7f20e00c8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:10.434 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.370+0000 7f20e4ed2700 1 --2- 172.21.15.67:0/1796285297 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f20e00b0e70 0x7f20e00c8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:10.434 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.370+0000 7f20e4ed2700 1 -- 172.21.15.67:0/1796285297 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20e00c27d0 msgr2=0x7f20e01549e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:10.434 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.370+0000 7f20e4ed2700 1 --2- 172.21.15.67:0/1796285297 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20e00c27d0 0x7f20e01549e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:10.434 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.370+0000 7f20e4ed2700 1 -- 172.21.15.67:0/1796285297 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20d8009b30 con 0x7f20e00ca150 2023-10-14T10:36:10.434 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.370+0000 7f20e4ed2700 1 --2- 172.21.15.67:0/1796285297 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f20e00ca150 0x7f20e01ebd80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f20d8014000 tx=0x7f20d8004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:10.435 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.369+0000 7f8d4559a700 1 -- 172.21.15.67:0/37524163 <== mon.2 v2:172.21.15.72:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d38011830 con 0x7f8d480c9a40 2023-10-14T10:36:10.435 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.369+0000 7f8d4559a700 1 -- 172.21.15.67:0/37524163 <== mon.2 v2:172.21.15.72:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d38011990 con 0x7f8d480c9a40 2023-10-14T10:36:10.435 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.369+0000 7f8d4559a700 1 -- 172.21.15.67:0/37524163 <== mon.2 v2:172.21.15.72:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d38011d00 con 0x7f8d480c9a40 2023-10-14T10:36:10.436 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.370+0000 7f5e6e599700 1 -- start start 2023-10-14T10:36:10.436 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.370+0000 7f70fdffb700 1 -- 172.21.15.67:0/1510343540 <== mon.1 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f400e6c0 con 0x7f70e8006a60 2023-10-14T10:36:10.437 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.371+0000 7f414c6e7700 1 Processor -- start 2023-10-14T10:36:10.437 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.370+0000 7fae68e05700 1 -- 172.21.15.67:0/3055713902 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fae6415a4e0 msgr2=0x7fae640724e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:10.437 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.370+0000 7fae68e05700 1 --2- 172.21.15.67:0/3055713902 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fae6415a4e0 0x7fae640724e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:10.438 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.370+0000 7fae68e05700 1 -- 172.21.15.67:0/3055713902 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fae64157250 msgr2=0x7fae64071fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:10.438 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.370+0000 7fae68e05700 1 --2- 172.21.15.67:0/3055713902 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fae64157250 0x7fae64071fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:10.438 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.370+0000 7fae68e05700 1 -- 172.21.15.67:0/3055713902 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae5c00bc80 con 0x7fae640c9fc0 2023-10-14T10:36:10.438 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.370+0000 7fae68e05700 1 --2- 172.21.15.67:0/3055713902 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fae640c9fc0 0x7fae64155db0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fae58006fd0 tx=0x7fae5800b730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:10.438 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.371+0000 7fae617fa700 1 -- 172.21.15.67:0/3055713902 <== mon.1 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae58011830 con 0x7fae640c9fc0 2023-10-14T10:36:10.439 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.371+0000 7fae617fa700 1 -- 172.21.15.67:0/3055713902 <== mon.1 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae58011990 con 0x7fae640c9fc0 2023-10-14T10:36:10.439 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.371+0000 7fae617fa700 1 -- 172.21.15.67:0/3055713902 <== mon.1 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae58011d40 con 0x7fae640c9fc0 2023-10-14T10:36:10.439 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.370+0000 7f70ff7fe700 1 -- 172.21.15.67:0/1510343540 reap_dead start 2023-10-14T10:36:10.440 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.370+0000 7f20dcff9700 1 -- 172.21.15.67:0/1796285297 <== mon.1 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d8004440 con 0x7f20e00ca150 2023-10-14T10:36:10.440 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.371+0000 7f20dcff9700 1 -- 172.21.15.67:0/1796285297 <== mon.1 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20d80045a0 con 0x7f20e00ca150 2023-10-14T10:36:10.440 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.371+0000 7f20dcff9700 1 -- 172.21.15.67:0/1796285297 <== mon.1 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d8004910 con 0x7f20e00ca150 2023-10-14T10:36:10.440 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.371+0000 7f8d4cddd700 1 -- 172.21.15.67:0/37524163 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f8d480c9a40 msgr2=0x7f8d480c9ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:10.441 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.371+0000 7f8d4cddd700 1 --2- 172.21.15.67:0/37524163 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f8d480c9a40 0x7f8d480c9ed0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f8d480ccba0 tx=0x7f8d38009b20 comp rx=0 tx=0).stop 2023-10-14T10:36:10.441 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.371+0000 7f5e6e599700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f5e54003a00 0x7f5e680c3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:10.441 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.371+0000 7f5e6e599700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5e68157010 0x7f5e681534c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:10.442 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.371+0000 7f5e6e599700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5e680c51f0 0x7f5e680c5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:10.442 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.371+0000 7f5e6e599700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e68157d70 con 0x7f5e68157010 2023-10-14T10:36:10.442 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.371+0000 7f5e6e599700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e681577b0 con 0x7f5e680c51f0 2023-10-14T10:36:10.442 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.371+0000 7f5e6e599700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e68157a90 con 0x7f5e54003a00 2023-10-14T10:36:10.442 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.372+0000 7f5e6cd96700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5e68157010 0x7f5e681534c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:10.443 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.372+0000 7f5e6cd96700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5e68157010 0x7f5e681534c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44146/0 (socket says 172.21.15.67:44146) 2023-10-14T10:36:10.443 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.372+0000 7f5e6cd96700 1 -- 172.21.15.67:0/3763767372 learned_addr learned my addr 172.21.15.67:0/3763767372 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:10.443 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.372+0000 7f5e6dd98700 1 --2- 172.21.15.67:0/3763767372 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5e680c51f0 0x7f5e680c5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:10.443 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.372+0000 7f5e6d597700 1 --2- 172.21.15.67:0/3763767372 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f5e54003a00 0x7f5e680c3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:10.443 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.372+0000 7f5e6cd96700 1 -- 172.21.15.67:0/3763767372 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f5e54003a00 msgr2=0x7f5e680c3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:10.444 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.372+0000 7f5e6cd96700 1 --2- 172.21.15.67:0/3763767372 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f5e54003a00 0x7f5e680c3900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:10.444 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.372+0000 7f5e6cd96700 1 -- 172.21.15.67:0/3763767372 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5e680c51f0 msgr2=0x7f5e680c5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:10.444 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.372+0000 7f5e6cd96700 1 --2- 172.21.15.67:0/3763767372 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5e680c51f0 0x7f5e680c5670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:10.444 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.372+0000 7f5e6cd96700 1 -- 172.21.15.67:0/3763767372 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e680c7480 con 0x7f5e68157010 2023-10-14T10:36:10.444 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.373+0000 7f5e6cd96700 1 --2- 172.21.15.67:0/3763767372 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5e68157010 0x7f5e681534c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5e58010df0 tx=0x7f5e5800e670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:10.445 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.373+0000 7f5e5dffb700 1 -- 172.21.15.67:0/3763767372 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e5801a600 con 0x7f5e68157010 2023-10-14T10:36:10.445 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.373+0000 7f5e5dffb700 1 -- 172.21.15.67:0/3763767372 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e5801a760 con 0x7f5e68157010 2023-10-14T10:36:10.445 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.373+0000 7f5e5dffb700 1 -- 172.21.15.67:0/3763767372 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e5801aad0 con 0x7f5e68157010 2023-10-14T10:36:10.445 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.373+0000 7f5e6e599700 1 -- 172.21.15.67:0/3763767372 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e680c7760 con 0x7f5e68157010 2023-10-14T10:36:10.445 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.374+0000 7f5e6e599700 1 -- 172.21.15.67:0/3763767372 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e68157490 con 0x7f5e68157010 2023-10-14T10:36:10.446 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.372+0000 7f076cff9700 1 -- 172.21.15.67:0/1179860214 <== mon.1 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f076402c400 con 0x7f07700b1fe0 2023-10-14T10:36:10.446 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.372+0000 7f076cff9700 1 --2- 172.21.15.67:0/1179860214 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f075c06af50 0x7f075c06d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:10.446 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.373+0000 7f076cff9700 1 -- 172.21.15.67:0/1179860214 <== mon.1 v2:172.21.15.72:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f076408aa10 con 0x7f07700b1fe0 2023-10-14T10:36:10.447 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.371+0000 7f414c6e7700 1 -- start start 2023-10-14T10:36:10.447 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.372+0000 7f414c6e7700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41440cecb0 0x7f41441f5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:10.447 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.372+0000 7f414c6e7700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f41440b3650 0x7f414415a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:10.447 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.372+0000 7f414c6e7700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4144156cd0 0x7f4144157150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:10.448 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.372+0000 7f414c6e7700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4144157690 con 0x7f41440cecb0 2023-10-14T10:36:10.448 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.372+0000 7f414c6e7700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f414415aa40 con 0x7f4144156cd0 2023-10-14T10:36:10.448 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.372+0000 7f414c6e7700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41440b3ad0 con 0x7f41440b3650 2023-10-14T10:36:10.448 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.372+0000 7f4149c82700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f41440b3650 0x7f414415a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:10.448 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.372+0000 7f4149c82700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f41440b3650 0x7f414415a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3301/0 says I am v2:172.21.15.67:60432/0 (socket says 172.21.15.67:60432) 2023-10-14T10:36:10.449 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.372+0000 7f4149c82700 1 -- 172.21.15.67:0/39024195 learned_addr learned my addr 172.21.15.67:0/39024195 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:10.449 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.372+0000 7f414ac84700 1 --2- 172.21.15.67:0/39024195 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4144156cd0 0x7f4144157150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:10.449 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.373+0000 7f4149c82700 1 -- 172.21.15.67:0/39024195 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4144156cd0 msgr2=0x7f4144157150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:10.449 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.373+0000 7f4149c82700 1 --2- 172.21.15.67:0/39024195 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4144156cd0 0x7f4144157150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:10.449 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.373+0000 7f4149c82700 1 -- 172.21.15.67:0/39024195 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41440cecb0 msgr2=0x7f41441f5ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:10.450 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.373+0000 7f4149c82700 1 --2- 172.21.15.67:0/39024195 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41440cecb0 0x7f41441f5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:10.450 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.373+0000 7f4149c82700 1 -- 172.21.15.67:0/39024195 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f413c00d040 con 0x7f41440b3650 2023-10-14T10:36:10.450 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.373+0000 7f4149c82700 1 --2- 172.21.15.67:0/39024195 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f41440b3650 0x7f414415a500 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f413c000f80 tx=0x7f413c004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:10.450 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.371+0000 7fae6b069700 1 -- 172.21.15.67:0/3055713902 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae640c4a10 con 0x7fae640c9fc0 2023-10-14T10:36:10.451 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.371+0000 7fae6b069700 1 -- 172.21.15.67:0/3055713902 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae641587a0 con 0x7fae640c9fc0 2023-10-14T10:36:10.451 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.373+0000 7fae6b069700 1 -- 172.21.15.67:0/3055713902 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae6406d080 con 0x7fae640c9fc0 2023-10-14T10:36:10.451 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.372+0000 7f70fdffb700 1 -- 172.21.15.67:0/1510343540 <== mon.1 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70f400e820 con 0x7f70e8006a60 2023-10-14T10:36:10.452 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.372+0000 7f70fdffb700 1 -- 172.21.15.67:0/1510343540 <== mon.1 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f400eb90 con 0x7f70e8006a60 2023-10-14T10:36:10.452 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.373+0000 7f71055c8700 1 -- 172.21.15.67:0/1510343540 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f70e8006a60 msgr2=0x7f70e8006ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:10.452 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.373+0000 7f71055c8700 1 --2- 172.21.15.67:0/1510343540 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f70e8006a60 0x7f70e8006ee0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f70f4009b60 tx=0x7f70e8007a90 comp rx=0 tx=0).stop 2023-10-14T10:36:10.452 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.373+0000 7f71055c8700 1 -- 172.21.15.67:0/1510343540 shutdown_connections 2023-10-14T10:36:10.452 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.373+0000 7f71055c8700 1 --2- 172.21.15.67:0/1510343540 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f70e8006a60 0x7f70e8006ee0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:10.453 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.373+0000 7f71055c8700 1 -- 172.21.15.67:0/1510343540 >> 172.21.15.67:0/1510343540 conn(0x7f71000cf3f0 msgr2=0x7f71000cf800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:10.453 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.373+0000 7f71055c8700 1 -- 172.21.15.67:0/1510343540 shutdown_connections 2023-10-14T10:36:10.453 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.373+0000 7f71055c8700 1 -- 172.21.15.67:0/1510343540 wait complete. 2023-10-14T10:36:10.453 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.372+0000 7f20e6935700 1 -- 172.21.15.67:0/1796285297 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20e00c7970 con 0x7f20e00ca150 2023-10-14T10:36:10.454 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.372+0000 7f20e6935700 1 -- 172.21.15.67:0/1796285297 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20e00c7f30 con 0x7f20e00ca150 2023-10-14T10:36:10.454 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.374+0000 7f20e6935700 1 -- 172.21.15.67:0/1796285297 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20e004ebb0 con 0x7f20e00ca150 2023-10-14T10:36:10.454 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.373+0000 7f8d4cddd700 1 -- 172.21.15.67:0/37524163 shutdown_connections 2023-10-14T10:36:10.454 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.373+0000 7f8d4cddd700 1 --2- 172.21.15.67:0/37524163 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f8d480c9a40 0x7f8d480c9ed0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:10.455 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.373+0000 7f8d4cddd700 1 --2- 172.21.15.67:0/37524163 >> v2:172.21.15.67:3300/0 conn(0x7f8d480c27d0 0x7f8d480ce000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:10.455 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.373+0000 7f8d4cddd700 1 --2- 172.21.15.67:0/37524163 >> v2:172.21.15.72:3300/0 conn(0x7f8d480b0e70 0x7f8d480b1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:10.455 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.373+0000 7f8d4cddd700 1 -- 172.21.15.67:0/37524163 >> 172.21.15.67:0/37524163 conn(0x7f8d4806afb0 msgr2=0x7f8d480cbe70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:10.455 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.373+0000 7f8d4cddd700 1 -- 172.21.15.67:0/37524163 shutdown_connections 2023-10-14T10:36:10.455 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.373+0000 7f8d4cddd700 1 -- 172.21.15.67:0/37524163 wait complete. 2023-10-14T10:36:10.456 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.375+0000 7f413affd700 1 -- 172.21.15.67:0/39024195 <== mon.2 v2:172.21.15.72:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f413c0099b0 con 0x7f41440b3650 2023-10-14T10:36:10.456 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.374+0000 7fae617fa700 1 -- 172.21.15.67:0/3055713902 <== mon.1 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fae5801a400 con 0x7fae640c9fc0 2023-10-14T10:36:10.457 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.375+0000 7fae617fa700 1 --2- 172.21.15.67:0/3055713902 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fae5006af00 0x7fae5006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:10.457 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.375+0000 7fae617fa700 1 -- 172.21.15.67:0/3055713902 <== mon.1 v2:172.21.15.72:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fae58089520 con 0x7fae640c9fc0 2023-10-14T10:36:10.457 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.375+0000 7f20dcff9700 1 -- 172.21.15.67:0/1796285297 <== mon.1 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f20d801e430 con 0x7f20e00ca150 2023-10-14T10:36:10.458 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.375+0000 7f8d4cddd700 1 Processor -- start 2023-10-14T10:36:10.458 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.375+0000 7f71055c8700 1 Processor -- start 2023-10-14T10:36:10.458 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.375+0000 7f8d4cddd700 1 -- start start 2023-10-14T10:36:10.459 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.375+0000 7f413affd700 1 -- 172.21.15.67:0/39024195 <== mon.2 v2:172.21.15.72:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f413c004240 con 0x7f41440b3650 2023-10-14T10:36:10.459 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.375+0000 7f413affd700 1 -- 172.21.15.67:0/39024195 <== mon.2 v2:172.21.15.72:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f413c0045d0 con 0x7f41440b3650 2023-10-14T10:36:10.460 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.374+0000 7f077678a700 1 -- 172.21.15.67:0/1179860214 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f077004ebb0 con 0x7f07700b1fe0 2023-10-14T10:36:10.460 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.374+0000 7f076ffff700 1 --2- 172.21.15.67:0/1179860214 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f075c06af50 0x7f075c06d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:10.460 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.375+0000 7f076ffff700 1 --2- 172.21.15.67:0/1179860214 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f075c06af50 0x7f075c06d410 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f07701538e0 tx=0x7f07680018e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:10.460 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.375+0000 7fae63fff700 1 --2- 172.21.15.67:0/3055713902 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fae5006af00 0x7fae5006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:10.461 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.375+0000 7f71055c8700 1 -- start start 2023-10-14T10:36:10.461 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.376+0000 7f414c6e7700 1 -- 172.21.15.67:0/39024195 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f414415ace0 con 0x7f41440b3650 2023-10-14T10:36:10.462 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.377+0000 7f414c6e7700 1 -- 172.21.15.67:0/39024195 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f414415b2a0 con 0x7f41440b3650 2023-10-14T10:36:10.462 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.377+0000 7fae63fff700 1 --2- 172.21.15.67:0/3055713902 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fae5006af00 0x7fae5006d3c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fae640ca3e0 tx=0x7fae5c012000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:10.462 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.377+0000 7f71055c8700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f70e8006a60 0x7f71000b3a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:10.463 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.377+0000 7f71055c8700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f71000c4650 0x7f71000c9060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:10.463 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.377+0000 7f71055c8700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f71000af800 0x7f71000afc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:10.463 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.377+0000 7f71055c8700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71000b01c0 con 0x7f70e8006a60 2023-10-14T10:36:10.463 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.377+0000 7f71055c8700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71000b4870 con 0x7f71000af800 2023-10-14T10:36:10.463 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.377+0000 7f71055c8700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71000c95a0 con 0x7f71000c4650 2023-10-14T10:36:10.464 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.377+0000 7f70feffd700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f70e8006a60 0x7f71000b3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:10.464 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.377+0000 7f70feffd700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f70e8006a60 0x7f71000b3a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44164/0 (socket says 172.21.15.67:44164) 2023-10-14T10:36:10.464 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.377+0000 7f70feffd700 1 -- 172.21.15.67:0/1311755677 learned_addr learned my addr 172.21.15.67:0/1311755677 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:10.465 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.376+0000 7f20dcff9700 1 --2- 172.21.15.67:0/1796285297 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f20cc06afd0 0x7f20cc06d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:10.465 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.376+0000 7f20dcff9700 1 -- 172.21.15.67:0/1796285297 <== mon.1 v2:172.21.15.72:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f20d808b0e0 con 0x7f20e00ca150 2023-10-14T10:36:10.465 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.376+0000 7f20dffff700 1 --2- 172.21.15.67:0/1796285297 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f20cc06afd0 0x7f20cc06d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:10.465 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.376+0000 7f20dffff700 1 --2- 172.21.15.67:0/1796285297 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f20cc06afd0 0x7f20cc06d490 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f20d4007fd0 tx=0x7f20d4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:10.466 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.376+0000 7f8d4cddd700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f8d480b0e70 0x7f8d481534a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:10.466 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.376+0000 7f8d4cddd700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d480c27d0 0x7f8d480b1f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:10.466 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.376+0000 7f8d4cddd700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8d480c9a40 0x7f8d480c4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:10.466 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.377+0000 7f8d4cddd700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d48154440 con 0x7f8d480c27d0 2023-10-14T10:36:10.466 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.377+0000 7f8d4cddd700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d48153ba0 con 0x7f8d480c9a40 2023-10-14T10:36:10.467 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.377+0000 7f8d4cddd700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d48153e80 con 0x7f8d480b0e70 2023-10-14T10:36:10.467 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.377+0000 7f8d45d9b700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d480c27d0 0x7f8d480b1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:10.467 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.377+0000 7f70fe7fc700 1 --2- 172.21.15.67:0/1311755677 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f71000c4650 0x7f71000c9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:10.467 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.377+0000 7f70ff7fe700 1 --2- 172.21.15.67:0/1311755677 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f71000af800 0x7f71000afc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:10.468 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.377+0000 7f8d45d9b700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d480c27d0 0x7f8d480b1f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44162/0 (socket says 172.21.15.67:44162) 2023-10-14T10:36:10.468 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.377+0000 7f8d45d9b700 1 -- 172.21.15.67:0/3163814754 learned_addr learned my addr 172.21.15.67:0/3163814754 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:10.468 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.377+0000 7f8d4659c700 1 --2- 172.21.15.67:0/3163814754 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f8d480b0e70 0x7f8d481534a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:10.469 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.375+0000 7f5e6e599700 1 -- 172.21.15.67:0/3763767372 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e480052f0 con 0x7f5e68157010 2023-10-14T10:36:10.469 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.377+0000 7f5e5dffb700 1 -- 172.21.15.67:0/3763767372 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f5e5800a070 con 0x7f5e68157010 2023-10-14T10:36:10.469 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.377+0000 7f5e5dffb700 1 --2- 172.21.15.67:0/3763767372 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f5e4c06af50 0x7f5e4c06d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:10.470 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.377+0000 7f8d46d9d700 1 --2- 172.21.15.67:0/3163814754 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8d480c9a40 0x7f8d480c4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:10.470 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.377+0000 7fd125a9a700 1 Processor -- start 2023-10-14T10:36:10.470 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.377+0000 7f5e5dffb700 1 -- 172.21.15.67:0/3763767372 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5e58006040 con 0x7f5e68157010 2023-10-14T10:36:10.471 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.377+0000 7f5e6d597700 1 --2- 172.21.15.67:0/3763767372 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f5e4c06af50 0x7f5e4c06d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:10.471 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.377+0000 7fd125a9a700 1 -- start start 2023-10-14T10:36:10.472 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.377+0000 7f70fe7fc700 1 -- 172.21.15.67:0/1311755677 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f71000af800 msgr2=0x7f71000afc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:10.472 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.377+0000 7f70fe7fc700 1 --2- 172.21.15.67:0/1311755677 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f71000af800 0x7f71000afc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:10.472 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.377+0000 7f70fe7fc700 1 -- 172.21.15.67:0/1311755677 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f70e8006a60 msgr2=0x7f71000b3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:10.472 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.377+0000 7f70fe7fc700 1 --2- 172.21.15.67:0/1311755677 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f70e8006a60 0x7f71000b3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:10.472 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.377+0000 7f70fe7fc700 1 -- 172.21.15.67:0/1311755677 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71000c4bc0 con 0x7f71000c4650 2023-10-14T10:36:10.473 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.378+0000 7f70fe7fc700 1 --2- 172.21.15.67:0/1311755677 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f71000c4650 0x7f71000c9060 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f70f4019000 tx=0x7f70f40047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:10.473 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.378+0000 7f8d4659c700 1 -- 172.21.15.67:0/3163814754 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8d480c9a40 msgr2=0x7f8d480c4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:10.473 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.378+0000 7f8d4659c700 1 --2- 172.21.15.67:0/3163814754 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8d480c9a40 0x7f8d480c4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:10.474 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.378+0000 7f8d4659c700 1 -- 172.21.15.67:0/3163814754 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d480c27d0 msgr2=0x7f8d480b1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:10.474 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.378+0000 7f8d4659c700 1 --2- 172.21.15.67:0/3163814754 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d480c27d0 0x7f8d480b1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:10.474 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.378+0000 7f8d4659c700 1 -- 172.21.15.67:0/3163814754 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d380094a0 con 0x7f8d480b0e70 2023-10-14T10:36:10.474 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.378+0000 7f70e77fe700 1 -- 172.21.15.67:0/1311755677 <== mon.2 v2:172.21.15.72:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f400e6c0 con 0x7f71000c4650 2023-10-14T10:36:10.475 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.378+0000 7f70e77fe700 1 -- 172.21.15.67:0/1311755677 <== mon.2 v2:172.21.15.72:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70f4013b60 con 0x7f71000c4650 2023-10-14T10:36:10.475 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.378+0000 7f414c6e7700 1 -- 172.21.15.67:0/39024195 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f414404ebb0 con 0x7f41440b3650 2023-10-14T10:36:10.476 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.378+0000 7f5e6d597700 1 --2- 172.21.15.67:0/3763767372 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f5e4c06af50 0x7f5e4c06d410 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5e640016c0 tx=0x7f5e64013000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:10.476 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.378+0000 7f70e77fe700 1 -- 172.21.15.67:0/1311755677 <== mon.2 v2:172.21.15.72:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f4004030 con 0x7f71000c4650 2023-10-14T10:36:10.477 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.378+0000 7f8d4659c700 1 --2- 172.21.15.67:0/3163814754 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f8d480b0e70 0x7f8d481534a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c00b450 tx=0x7f8d3c00b760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:10.477 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.378+0000 7f8d36ffd700 1 -- 172.21.15.67:0/3163814754 <== mon.2 v2:172.21.15.72:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d3c011830 con 0x7f8d480b0e70 2023-10-14T10:36:10.477 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.378+0000 7f8d36ffd700 1 -- 172.21.15.67:0/3163814754 <== mon.2 v2:172.21.15.72:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d3c011990 con 0x7f8d480b0e70 2023-10-14T10:36:10.477 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.378+0000 7f8d36ffd700 1 -- 172.21.15.67:0/3163814754 <== mon.2 v2:172.21.15.72:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d3c011d40 con 0x7f8d480b0e70 2023-10-14T10:36:10.477 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.378+0000 7f8d4cddd700 1 -- 172.21.15.67:0/3163814754 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d48154160 con 0x7f8d480b0e70 2023-10-14T10:36:10.478 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.378+0000 7f8d4cddd700 1 -- 172.21.15.67:0/3163814754 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d480b24b0 con 0x7f8d480b0e70 2023-10-14T10:36:10.479 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.379+0000 7fd125a9a700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd12015a280 0x7fd1200cbd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:10.479 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.379+0000 7fd125a9a700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fd12006d390 0x7fd12006d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:10.479 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.379+0000 7fd125a9a700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1200cf1d0 con 0x7fd1200ce5f0 2023-10-14T10:36:10.479 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.379+0000 7fd125a9a700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd12006dd80 con 0x7fd12015a280 2023-10-14T10:36:10.479 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.379+0000 7fd125a9a700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd120072200 con 0x7fd12006d390 2023-10-14T10:36:10.480 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.379+0000 7fd11f7fe700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd12015a280 0x7fd1200cbd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:10.480 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.379+0000 7fd11f7fe700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd12015a280 0x7fd1200cbd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44176/0 (socket says 172.21.15.67:44176) 2023-10-14T10:36:10.480 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.379+0000 7fd11f7fe700 1 -- 172.21.15.67:0/2079480014 learned_addr learned my addr 172.21.15.67:0/2079480014 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:10.480 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.379+0000 7fd11f7fe700 1 -- 172.21.15.67:0/2079480014 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fd12006d390 msgr2=0x7fd12006d840 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-14T10:36:10.480 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.379+0000 7fd11f7fe700 1 --2- 172.21.15.67:0/2079480014 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fd12006d390 0x7fd12006d840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:10.481 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.379+0000 7fd11f7fe700 1 -- 172.21.15.67:0/2079480014 >> v1:172.21.15.72:6789/0 conn(0x7fd1200ce5f0 legacy=0x7fd12006ab20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-14T10:36:10.481 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.379+0000 7fd11f7fe700 1 -- 172.21.15.67:0/2079480014 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1200cafb0 con 0x7fd12015a280 2023-10-14T10:36:10.481 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.379+0000 7f8d4cddd700 1 -- 172.21.15.67:0/3163814754 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d480c9e50 con 0x7f8d480b0e70 2023-10-14T10:36:10.482 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.380+0000 7f71055c8700 1 -- 172.21.15.67:0/1311755677 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71000c4ea0 con 0x7f71000c4650 2023-10-14T10:36:10.482 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.380+0000 7f71055c8700 1 -- 172.21.15.67:0/1311755677 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71000c5460 con 0x7f71000c4650 2023-10-14T10:36:10.482 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.380+0000 7fd11f7fe700 1 --2- 172.21.15.67:0/2079480014 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd12015a280 0x7fd1200cbd70 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd11000bb90 tx=0x7fd11000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4697e9e4249a96ea server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:10.483 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.380+0000 7f413affd700 1 -- 172.21.15.67:0/39024195 <== mon.2 v2:172.21.15.72:3301/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f413c004730 con 0x7f41440b3650 2023-10-14T10:36:10.483 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.380+0000 7f8d36ffd700 1 -- 172.21.15.67:0/3163814754 <== mon.2 v2:172.21.15.72:3301/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f8d3c01a400 con 0x7f8d480b0e70 2023-10-14T10:36:10.483 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.381+0000 7f413affd700 1 --2- 172.21.15.67:0/39024195 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f412806af50 0x7f412806d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:10.484 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.381+0000 7fae617fa700 1 -- 172.21.15.67:0/3055713902 <== mon.1 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fae580545d0 con 0x7fae640c9fc0 2023-10-14T10:36:10.484 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.381+0000 7f71055c8700 1 -- 172.21.15.67:0/1311755677 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f710004ebb0 con 0x7f71000c4650 2023-10-14T10:36:10.485 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.381+0000 7f8d36ffd700 1 --2- 172.21.15.67:0/3163814754 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f8d2806af50 0x7f8d2806d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:10.485 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.381+0000 7f413affd700 1 -- 172.21.15.67:0/39024195 <== mon.2 v2:172.21.15.72:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f413c08ae50 con 0x7f41440b3650 2023-10-14T10:36:10.485 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.381+0000 7f414a483700 1 --2- 172.21.15.67:0/39024195 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f412806af50 0x7f412806d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:10.486 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.381+0000 7f8d36ffd700 1 -- 172.21.15.67:0/3163814754 <== mon.2 v2:172.21.15.72:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8d3c089870 con 0x7f8d480b0e70 2023-10-14T10:36:10.486 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.381+0000 7f8d45d9b700 1 --2- 172.21.15.67:0/3163814754 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f8d2806af50 0x7f8d2806d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:10.486 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.381+0000 7f8d45d9b700 1 --2- 172.21.15.67:0/3163814754 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f8d2806af50 0x7f8d2806d410 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8d4816cbd0 tx=0x7f8d4000b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:10.487 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.381+0000 7f414a483700 1 --2- 172.21.15.67:0/39024195 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f412806af50 0x7f412806d410 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f41440b1da0 tx=0x7f4134006cd0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:10.487 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.384+0000 7fd11e7fc700 1 -- 172.21.15.67:0/2079480014 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd11000dd10 con 0x7fd12015a280 2023-10-14T10:36:10.488 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.384+0000 7fd11e7fc700 1 -- 172.21.15.67:0/2079480014 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd11000de70 con 0x7fd12015a280 2023-10-14T10:36:10.488 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.384+0000 7fd11e7fc700 1 -- 172.21.15.67:0/2079480014 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd110004230 con 0x7fd12015a280 2023-10-14T10:36:10.488 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.384+0000 7f5e5dffb700 1 -- 172.21.15.67:0/3763767372 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5e58055cb0 con 0x7f5e68157010 2023-10-14T10:36:10.489 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.384+0000 7fd125a9a700 1 -- 172.21.15.67:0/2079480014 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd12015a280 msgr2=0x7fd1200cbd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:10.489 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.384+0000 7fd125a9a700 1 --2- 172.21.15.67:0/2079480014 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd12015a280 0x7fd1200cbd70 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd11000bb90 tx=0x7fd11000bea0 comp rx=0 tx=0).stop 2023-10-14T10:36:10.489 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.384+0000 7fd125a9a700 1 -- 172.21.15.67:0/2079480014 shutdown_connections 2023-10-14T10:36:10.490 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.384+0000 7fd125a9a700 1 --2- 172.21.15.67:0/2079480014 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd12015a280 0x7fd1200cbd70 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:10.490 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.385+0000 7fd125a9a700 1 --2- 172.21.15.67:0/2079480014 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fd12006d390 0x7fd12006d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:10.490 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.385+0000 7fd125a9a700 1 -- 172.21.15.67:0/2079480014 >> 172.21.15.67:0/2079480014 conn(0x7fd1200cfc40 msgr2=0x7fd1200d0020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:10.490 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.385+0000 7fd125a9a700 1 -- 172.21.15.67:0/2079480014 shutdown_connections 2023-10-14T10:36:10.490 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.385+0000 7fd125a9a700 1 -- 172.21.15.67:0/2079480014 wait complete. 2023-10-14T10:36:10.491 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.385+0000 7f8d36ffd700 1 -- 172.21.15.67:0/3163814754 <== mon.2 v2:172.21.15.72:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8d3c054a90 con 0x7f8d480b0e70 2023-10-14T10:36:10.491 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.385+0000 7f20dcff9700 1 -- 172.21.15.67:0/1796285297 <== mon.1 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f20d8059d20 con 0x7f20e00ca150 2023-10-14T10:36:10.492 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.386+0000 7f70e77fe700 1 -- 172.21.15.67:0/1311755677 <== mon.2 v2:172.21.15.72:3301/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f70f4024070 con 0x7f71000c4650 2023-10-14T10:36:10.492 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.387+0000 7fd125a9a700 1 Processor -- start 2023-10-14T10:36:10.492 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.387+0000 7fd125a9a700 1 -- start start 2023-10-14T10:36:10.493 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.387+0000 7f70e77fe700 1 --2- 172.21.15.67:0/1311755677 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f70ec06af50 0x7f70ec06d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:10.493 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.387+0000 7f70feffd700 1 --2- 172.21.15.67:0/1311755677 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f70ec06af50 0x7f70ec06d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:10.493 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.388+0000 7f70feffd700 1 --2- 172.21.15.67:0/1311755677 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f70ec06af50 0x7f70ec06d410 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f70f0000ff0 tx=0x7f70f0007980 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:10.494 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.388+0000 7fd125a9a700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fd12006d390 0x7fd12015dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:10.495 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.388+0000 7fd125a9a700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd1200ce5f0 0x7fd1200aca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:10.495 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.388+0000 7fd125a9a700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd12015a280 0x7fd1200acf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:10.495 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.388+0000 7fd125a9a700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd120142400 con 0x7fd12015a280 2023-10-14T10:36:10.495 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.388+0000 7fd125a9a700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd120141800 con 0x7fd1200ce5f0 2023-10-14T10:36:10.496 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.388+0000 7fd125a9a700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd120141ad0 con 0x7fd12006d390 2023-10-14T10:36:10.496 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.388+0000 7fd11f7fe700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fd12006d390 0x7fd12015dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:10.496 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.389+0000 7fd11f7fe700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fd12006d390 0x7fd12015dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3301/0 says I am v2:172.21.15.67:60466/0 (socket says 172.21.15.67:60466) 2023-10-14T10:36:10.496 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.389+0000 7fd11f7fe700 1 -- 172.21.15.67:0/854972728 learned_addr learned my addr 172.21.15.67:0/854972728 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:10.496 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.389+0000 7fd11ffff700 1 --2- 172.21.15.67:0/854972728 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd12015a280 0x7fd1200acf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:10.497 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.389+0000 7fd11effd700 1 --2- 172.21.15.67:0/854972728 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd1200ce5f0 0x7fd1200aca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:10.497 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.389+0000 7fd11f7fe700 1 -- 172.21.15.67:0/854972728 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd1200ce5f0 msgr2=0x7fd1200aca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:10.497 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.389+0000 7fd11f7fe700 1 --2- 172.21.15.67:0/854972728 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd1200ce5f0 0x7fd1200aca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:10.497 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.389+0000 7fd11f7fe700 1 -- 172.21.15.67:0/854972728 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd12015a280 msgr2=0x7fd1200acf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:10.497 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.389+0000 7fd11f7fe700 1 --2- 172.21.15.67:0/854972728 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd12015a280 0x7fd1200acf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:10.498 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.389+0000 7fd11f7fe700 1 -- 172.21.15.67:0/854972728 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd11000b7f0 con 0x7fd12006d390 2023-10-14T10:36:10.498 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.389+0000 7fd11f7fe700 1 --2- 172.21.15.67:0/854972728 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fd12006d390 0x7fd12015dec0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd11000b340 tx=0x7fd110004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:10.498 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.390+0000 7fd107fff700 1 -- 172.21.15.67:0/854972728 <== mon.2 v2:172.21.15.72:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd11000dcc0 con 0x7fd12006d390 2023-10-14T10:36:10.498 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.390+0000 7fd125a9a700 1 -- 172.21.15.67:0/854972728 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd120141db0 con 0x7fd12006d390 2023-10-14T10:36:10.499 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.390+0000 7f076cff9700 1 -- 172.21.15.67:0/1179860214 <== mon.1 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0764015020 con 0x7f07700b1fe0 2023-10-14T10:36:10.499 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.390+0000 7f413affd700 1 -- 172.21.15.67:0/39024195 <== mon.2 v2:172.21.15.72:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f413c055ff0 con 0x7f41440b3650 2023-10-14T10:36:10.499 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.390+0000 7fd125a9a700 1 -- 172.21.15.67:0/854972728 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1200ad480 con 0x7fd12006d390 2023-10-14T10:36:10.500 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.390+0000 7f70e77fe700 1 -- 172.21.15.67:0/1311755677 <== mon.2 v2:172.21.15.72:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f70f405a5d0 con 0x7f71000c4650 2023-10-14T10:36:10.500 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.391+0000 7fd125a9a700 1 -- 172.21.15.67:0/854972728 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd120067d20 con 0x7fd12006d390 2023-10-14T10:36:10.501 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.394+0000 7fd107fff700 1 -- 172.21.15.67:0/854972728 <== mon.2 v2:172.21.15.72:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd110010470 con 0x7fd12006d390 2023-10-14T10:36:10.501 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.394+0000 7fd107fff700 1 -- 172.21.15.67:0/854972728 <== mon.2 v2:172.21.15.72:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1100107e0 con 0x7fd12006d390 2023-10-14T10:36:10.501 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.394+0000 7f70e77fe700 1 -- 172.21.15.67:0/1311755677 <== mon.2 v2:172.21.15.72:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f70f4056210 con 0x7f71000c4650 2023-10-14T10:36:10.501 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.399+0000 7fd107fff700 1 -- 172.21.15.67:0/854972728 <== mon.2 v2:172.21.15.72:3301/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd110023440 con 0x7fd12006d390 2023-10-14T10:36:10.502 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.399+0000 7fd107fff700 1 --2- 172.21.15.67:0/854972728 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fd10c06af00 0x7fd10c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:10.502 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.399+0000 7fd107fff700 1 -- 172.21.15.67:0/854972728 <== mon.2 v2:172.21.15.72:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd11008b360 con 0x7fd12006d390 2023-10-14T10:36:10.502 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.400+0000 7fd107fff700 1 -- 172.21.15.67:0/854972728 <== mon.2 v2:172.21.15.72:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd110056520 con 0x7fd12006d390 2023-10-14T10:36:10.502 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.401+0000 7fd11effd700 1 --2- 172.21.15.67:0/854972728 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fd10c06af00 0x7fd10c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:10.503 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.401+0000 7fd11effd700 1 --2- 172.21.15.67:0/854972728 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fd10c06af00 0x7fd10c06d3c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd1201586d0 tx=0x7fd114013040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:10.800 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.797+0000 7f5e6e599700 1 -- 172.21.15.67:0/3763767372 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f5e48005870 con 0x7f5e68157010 2023-10-14T10:36:10.803 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.803+0000 7f5e5dffb700 1 -- 172.21.15.67:0/3763767372 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5e5801f070 con 0x7f5e68157010 2023-10-14T10:36:10.805 INFO:teuthology.orchestra.run.smithi067.stdout:60129542148 2023-10-14T10:36:10.805 INFO:teuthology.orchestra.run.smithi067.stdout:60129542149 2023-10-14T10:36:10.805 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.804+0000 7f077678a700 1 -- 172.21.15.67:0/1179860214 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f07700b5370 con 0x7f07700b1fe0 2023-10-14T10:36:10.806 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.803+0000 7fae6b069700 1 -- 172.21.15.67:0/3055713902 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fae6404ebb0 con 0x7fae640c9fc0 2023-10-14T10:36:10.806 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.804+0000 7fae617fa700 1 -- 172.21.15.67:0/3055713902 <== mon.1 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fae58015020 con 0x7fae640c9fc0 2023-10-14T10:36:10.806 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.804+0000 7f076cff9700 1 -- 172.21.15.67:0/1179860214 <== mon.1 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0764055c30 con 0x7f07700b1fe0 2023-10-14T10:36:10.807 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.807+0000 7f71055c8700 1 -- 172.21.15.67:0/1311755677 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f71000b5370 con 0x7f71000c4650 2023-10-14T10:36:10.808 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.807+0000 7f20e6935700 1 -- 172.21.15.67:0/1796285297 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f20e00b5370 con 0x7f20e00ca150 2023-10-14T10:36:10.808 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.807+0000 7f8d4cddd700 1 -- 172.21.15.67:0/3163814754 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f8d48153fc0 con 0x7f8d480b0e70 2023-10-14T10:36:10.809 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.807+0000 7fae6b069700 1 -- 172.21.15.67:0/3055713902 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fae5006af00 msgr2=0x7fae5006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:10.809 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.807+0000 7fae6b069700 1 --2- 172.21.15.67:0/3055713902 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fae5006af00 0x7fae5006d3c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fae640ca3e0 tx=0x7fae5c012000 comp rx=0 tx=0).stop 2023-10-14T10:36:10.809 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.807+0000 7fae6b069700 1 -- 172.21.15.67:0/3055713902 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fae640c9fc0 msgr2=0x7fae64155db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:10.809 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.807+0000 7fae6b069700 1 --2- 172.21.15.67:0/3055713902 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fae640c9fc0 0x7fae64155db0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fae58006fd0 tx=0x7fae5800b730 comp rx=0 tx=0).stop 2023-10-14T10:36:10.810 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.807+0000 7f5e6e599700 1 -- 172.21.15.67:0/3763767372 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f5e4c06af50 msgr2=0x7f5e4c06d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:10.810 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.807+0000 7f5e6e599700 1 --2- 172.21.15.67:0/3763767372 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f5e4c06af50 0x7f5e4c06d410 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5e640016c0 tx=0x7f5e64013000 comp rx=0 tx=0).stop 2023-10-14T10:36:10.810 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.807+0000 7f5e6e599700 1 -- 172.21.15.67:0/3763767372 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5e68157010 msgr2=0x7f5e681534c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:10.811 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.807+0000 7f5e6e599700 1 --2- 172.21.15.67:0/3763767372 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5e68157010 0x7f5e681534c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5e58010df0 tx=0x7f5e5800e670 comp rx=0 tx=0).stop 2023-10-14T10:36:10.811 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.807+0000 7f5e6e599700 1 -- 172.21.15.67:0/3763767372 shutdown_connections 2023-10-14T10:36:10.811 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.807+0000 7f5e6e599700 1 --2- 172.21.15.67:0/3763767372 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f5e4c06af50 0x7f5e4c06d410 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:10.811 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.807+0000 7f5e6e599700 1 --2- 172.21.15.67:0/3763767372 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5e680c51f0 0x7f5e680c5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:10.811 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.807+0000 7f5e6e599700 1 --2- 172.21.15.67:0/3763767372 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f5e54003a00 0x7f5e680c3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:10.812 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.807+0000 7f5e6e599700 1 --2- 172.21.15.67:0/3763767372 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5e68157010 0x7f5e681534c0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:10.812 INFO:teuthology.orchestra.run.smithi067.stdout:60129542149 2023-10-14T10:36:10.812 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.807+0000 7f5e6e599700 1 -- 172.21.15.67:0/3763767372 >> 172.21.15.67:0/3763767372 conn(0x7f5e680cf1c0 msgr2=0x7f5e680af8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:10.813 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.807+0000 7fae6b069700 1 -- 172.21.15.67:0/3055713902 shutdown_connections 2023-10-14T10:36:10.813 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.807+0000 7fae6b069700 1 --2- 172.21.15.67:0/3055713902 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fae5006af00 0x7fae5006d3c0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:10.813 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.807+0000 7fae6b069700 1 --2- 172.21.15.67:0/3055713902 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fae6415a4e0 0x7fae640724e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:10.813 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.807+0000 7fae6b069700 1 --2- 172.21.15.67:0/3055713902 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fae640c9fc0 0x7fae64155db0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:10.814 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.807+0000 7fae6b069700 1 --2- 172.21.15.67:0/3055713902 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fae64157250 0x7fae64071fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:10.814 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.807+0000 7fae6b069700 1 -- 172.21.15.67:0/3055713902 >> 172.21.15.67:0/3055713902 conn(0x7fae640cf3f0 msgr2=0x7fae640ceb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:10.814 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.807+0000 7f5e6e599700 1 -- 172.21.15.67:0/3763767372 shutdown_connections 2023-10-14T10:36:10.815 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.807+0000 7f5e6e599700 1 -- 172.21.15.67:0/3763767372 wait complete. 2023-10-14T10:36:10.815 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.808+0000 7fae6b069700 1 -- 172.21.15.67:0/3055713902 shutdown_connections 2023-10-14T10:36:10.815 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.808+0000 7fae6b069700 1 -- 172.21.15.67:0/3055713902 wait complete. 2023-10-14T10:36:10.816 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.808+0000 7f70e77fe700 1 -- 172.21.15.67:0/1311755677 <== mon.2 v2:172.21.15.72:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f70f4017090 con 0x7f71000c4650 2023-10-14T10:36:10.816 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.809+0000 7f8d36ffd700 1 -- 172.21.15.67:0/3163814754 <== mon.2 v2:172.21.15.72:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8d48153fc0 con 0x7f8d480b0e70 2023-10-14T10:36:10.817 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.809+0000 7f20dcff9700 1 -- 172.21.15.67:0/1796285297 <== mon.1 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f20d80598b0 con 0x7f20e00ca150 2023-10-14T10:36:10.817 INFO:teuthology.orchestra.run.smithi067.stdout:60129542149 2023-10-14T10:36:10.817 INFO:teuthology.orchestra.run.smithi067.stdout:60129542149 2023-10-14T10:36:10.818 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.812+0000 7f077678a700 1 -- 172.21.15.67:0/1179860214 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f075c06af50 msgr2=0x7f075c06d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:10.818 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.812+0000 7f077678a700 1 --2- 172.21.15.67:0/1179860214 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f075c06af50 0x7f075c06d410 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f07701538e0 tx=0x7f07680018e0 comp rx=0 tx=0).stop 2023-10-14T10:36:10.818 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.812+0000 7f077678a700 1 -- 172.21.15.67:0/1179860214 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f07700b1fe0 msgr2=0x7f07700b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:10.819 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.812+0000 7f077678a700 1 --2- 172.21.15.67:0/1179860214 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f07700b1fe0 0x7f07700b2440 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f075800c110 tx=0x7f07640047c0 comp rx=0 tx=0).stop 2023-10-14T10:36:10.819 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.812+0000 7f077678a700 1 -- 172.21.15.67:0/1179860214 shutdown_connections 2023-10-14T10:36:10.819 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.812+0000 7f077678a700 1 --2- 172.21.15.67:0/1179860214 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f075c06af50 0x7f075c06d410 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:10.819 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.812+0000 7f077678a700 1 --2- 172.21.15.67:0/1179860214 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f07700b3d80 0x7f07700b4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:10.820 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.812+0000 7f077678a700 1 --2- 172.21.15.67:0/1179860214 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0758004850 0x7f07700b0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:10.820 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.812+0000 7f077678a700 1 --2- 172.21.15.67:0/1179860214 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f07700b1fe0 0x7f07700b2440 secure :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f075800c110 tx=0x7f07640047c0 comp rx=0 tx=0).stop 2023-10-14T10:36:10.820 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.812+0000 7f077678a700 1 -- 172.21.15.67:0/1179860214 >> 172.21.15.67:0/1179860214 conn(0x7f07700cf3f0 msgr2=0x7f07700ae710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:10.820 INFO:teuthology.orchestra.run.smithi067.stdout:60129542149 2023-10-14T10:36:10.821 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.812+0000 7f077678a700 1 -- 172.21.15.67:0/1179860214 shutdown_connections 2023-10-14T10:36:10.821 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.812+0000 7f71055c8700 1 -- 172.21.15.67:0/1311755677 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f70ec06af50 msgr2=0x7f70ec06d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:10.821 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.812+0000 7f71055c8700 1 --2- 172.21.15.67:0/1311755677 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f70ec06af50 0x7f70ec06d410 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f70f0000ff0 tx=0x7f70f0007980 comp rx=0 tx=0).stop 2023-10-14T10:36:10.822 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.812+0000 7f71055c8700 1 -- 172.21.15.67:0/1311755677 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f71000c4650 msgr2=0x7f71000c9060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:10.822 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.812+0000 7f71055c8700 1 --2- 172.21.15.67:0/1311755677 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f71000c4650 0x7f71000c9060 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f70f4019000 tx=0x7f70f40047c0 comp rx=0 tx=0).stop 2023-10-14T10:36:10.822 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.812+0000 7f077678a700 1 -- 172.21.15.67:0/1179860214 wait complete. 2023-10-14T10:36:10.823 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.813+0000 7f20e6935700 1 -- 172.21.15.67:0/1796285297 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f20cc06afd0 msgr2=0x7f20cc06d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:10.823 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.813+0000 7f20e6935700 1 --2- 172.21.15.67:0/1796285297 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f20cc06afd0 0x7f20cc06d490 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f20d4007fd0 tx=0x7f20d4009040 comp rx=0 tx=0).stop 2023-10-14T10:36:10.823 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.813+0000 7f20e6935700 1 -- 172.21.15.67:0/1796285297 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f20e00ca150 msgr2=0x7f20e01ebd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:10.823 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.813+0000 7f20e6935700 1 --2- 172.21.15.67:0/1796285297 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f20e00ca150 0x7f20e01ebd80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f20d8014000 tx=0x7f20d8004ad0 comp rx=0 tx=0).stop 2023-10-14T10:36:10.824 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.813+0000 7f20e6935700 1 -- 172.21.15.67:0/1796285297 shutdown_connections 2023-10-14T10:36:10.824 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.813+0000 7f20e6935700 1 --2- 172.21.15.67:0/1796285297 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f20cc06afd0 0x7f20cc06d490 secure :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f20d4007fd0 tx=0x7f20d4009040 comp rx=0 tx=0).stop 2023-10-14T10:36:10.824 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.813+0000 7f20e6935700 1 --2- 172.21.15.67:0/1796285297 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f20e00ca150 0x7f20e01ebd80 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:10.824 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.813+0000 7f20e6935700 1 --2- 172.21.15.67:0/1796285297 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f20e00b0e70 0x7f20e00c8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:10.825 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.813+0000 7f20e6935700 1 --2- 172.21.15.67:0/1796285297 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20e00c27d0 0x7f20e01549e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:10.825 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.813+0000 7f20e6935700 1 -- 172.21.15.67:0/1796285297 >> 172.21.15.67:0/1796285297 conn(0x7f20e006afb0 msgr2=0x7f20e00af150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:10.825 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.814+0000 7f71055c8700 1 -- 172.21.15.67:0/1311755677 shutdown_connections 2023-10-14T10:36:10.825 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.814+0000 7f71055c8700 1 --2- 172.21.15.67:0/1311755677 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f70ec06af50 0x7f70ec06d410 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:10.826 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.814+0000 7f71055c8700 1 --2- 172.21.15.67:0/1311755677 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f71000af800 0x7f71000afc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:10.826 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.814+0000 7f71055c8700 1 --2- 172.21.15.67:0/1311755677 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f70e8006a60 0x7f71000b3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:10.826 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.814+0000 7f71055c8700 1 --2- 172.21.15.67:0/1311755677 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f71000c4650 0x7f71000c9060 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:10.826 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.814+0000 7f71055c8700 1 -- 172.21.15.67:0/1311755677 >> 172.21.15.67:0/1311755677 conn(0x7f71000cf3f0 msgr2=0x7f710006e660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:10.827 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.815+0000 7f71055c8700 1 -- 172.21.15.67:0/1311755677 shutdown_connections 2023-10-14T10:36:10.827 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.816+0000 7f20e6935700 1 -- 172.21.15.67:0/1796285297 shutdown_connections 2023-10-14T10:36:10.828 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.817+0000 7f8d4cddd700 1 -- 172.21.15.67:0/3163814754 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f8d2806af50 msgr2=0x7f8d2806d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:10.828 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.817+0000 7f8d4cddd700 1 --2- 172.21.15.67:0/3163814754 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f8d2806af50 0x7f8d2806d410 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8d4816cbd0 tx=0x7f8d4000b040 comp rx=0 tx=0).stop 2023-10-14T10:36:10.828 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.817+0000 7f8d4cddd700 1 -- 172.21.15.67:0/3163814754 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f8d480b0e70 msgr2=0x7f8d481534a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:10.829 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.817+0000 7f8d4cddd700 1 --2- 172.21.15.67:0/3163814754 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f8d480b0e70 0x7f8d481534a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c00b450 tx=0x7f8d3c00b760 comp rx=0 tx=0).stop 2023-10-14T10:36:10.829 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.817+0000 7f8d4cddd700 1 -- 172.21.15.67:0/3163814754 shutdown_connections 2023-10-14T10:36:10.829 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.817+0000 7f8d4cddd700 1 --2- 172.21.15.67:0/3163814754 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f8d2806af50 0x7f8d2806d410 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:10.829 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.817+0000 7f8d4cddd700 1 --2- 172.21.15.67:0/3163814754 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8d480c9a40 0x7f8d480c4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:10.829 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.817+0000 7f8d4cddd700 1 --2- 172.21.15.67:0/3163814754 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f8d480b0e70 0x7f8d481534a0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:10.830 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.817+0000 7f8d4cddd700 1 --2- 172.21.15.67:0/3163814754 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d480c27d0 0x7f8d480b1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:10.830 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.817+0000 7f8d4cddd700 1 -- 172.21.15.67:0/3163814754 >> 172.21.15.67:0/3163814754 conn(0x7f8d4806afb0 msgr2=0x7f8d480cf380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:10.830 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.819+0000 7f8d4cddd700 1 -- 172.21.15.67:0/3163814754 shutdown_connections 2023-10-14T10:36:10.830 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.819+0000 7f8d4cddd700 1 -- 172.21.15.67:0/3163814754 wait complete. 2023-10-14T10:36:10.831 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.819+0000 7f414c6e7700 1 -- 172.21.15.67:0/39024195 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f41440cdba0 con 0x7f41440b3650 2023-10-14T10:36:10.831 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.819+0000 7f413affd700 1 -- 172.21.15.67:0/39024195 <== mon.2 v2:172.21.15.72:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f413c059610 con 0x7f41440b3650 2023-10-14T10:36:10.831 INFO:teuthology.orchestra.run.smithi067.stdout:60129542148 2023-10-14T10:36:10.832 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.823+0000 7f71055c8700 1 -- 172.21.15.67:0/1311755677 wait complete. 2023-10-14T10:36:10.832 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.823+0000 7f414c6e7700 1 -- 172.21.15.67:0/39024195 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f412806af50 msgr2=0x7f412806d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:10.832 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.823+0000 7f414c6e7700 1 --2- 172.21.15.67:0/39024195 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f412806af50 0x7f412806d410 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f41440b1da0 tx=0x7f4134006cd0 comp rx=0 tx=0).stop 2023-10-14T10:36:10.833 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.823+0000 7f414c6e7700 1 -- 172.21.15.67:0/39024195 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f41440b3650 msgr2=0x7f414415a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:10.833 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.823+0000 7f414c6e7700 1 --2- 172.21.15.67:0/39024195 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f41440b3650 0x7f414415a500 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f413c000f80 tx=0x7f413c004ad0 comp rx=0 tx=0).stop 2023-10-14T10:36:10.833 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.823+0000 7f20e6935700 1 -- 172.21.15.67:0/1796285297 wait complete. 2023-10-14T10:36:10.834 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.823+0000 7f414c6e7700 1 -- 172.21.15.67:0/39024195 shutdown_connections 2023-10-14T10:36:10.834 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.823+0000 7f414c6e7700 1 --2- 172.21.15.67:0/39024195 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f412806af50 0x7f412806d410 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:10.834 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.823+0000 7f414c6e7700 1 --2- 172.21.15.67:0/39024195 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4144156cd0 0x7f4144157150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:10.834 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.823+0000 7f414c6e7700 1 --2- 172.21.15.67:0/39024195 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41440cecb0 0x7f41441f5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:10.835 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.823+0000 7f414c6e7700 1 --2- 172.21.15.67:0/39024195 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f41440b3650 0x7f414415a500 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:10.835 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.823+0000 7f414c6e7700 1 -- 172.21.15.67:0/39024195 >> 172.21.15.67:0/39024195 conn(0x7f414406afb0 msgr2=0x7f41441543d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:10.835 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.825+0000 7f414c6e7700 1 -- 172.21.15.67:0/39024195 shutdown_connections 2023-10-14T10:36:10.835 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.825+0000 7f414c6e7700 1 -- 172.21.15.67:0/39024195 wait complete. 2023-10-14T10:36:10.836 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.824+0000 7fd125a9a700 1 -- 172.21.15.67:0/854972728 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fd120141c10 con 0x7fd12006d390 2023-10-14T10:36:10.837 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542149 for osd.5 2023-10-14T10:36:10.837 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.827+0000 7fd107fff700 1 -- 172.21.15.67:0/854972728 <== mon.2 v2:172.21.15.72:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd120141c10 con 0x7fd12006d390 2023-10-14T10:36:10.838 INFO:teuthology.orchestra.run.smithi067.stdout:60129542150 2023-10-14T10:36:10.839 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542148 for osd.0 2023-10-14T10:36:10.839 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.830+0000 7fd125a9a700 1 -- 172.21.15.67:0/854972728 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fd10c06af00 msgr2=0x7fd10c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:10.840 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.830+0000 7fd125a9a700 1 --2- 172.21.15.67:0/854972728 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fd10c06af00 0x7fd10c06d3c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd1201586d0 tx=0x7fd114013040 comp rx=0 tx=0).stop 2023-10-14T10:36:10.840 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.830+0000 7fd125a9a700 1 -- 172.21.15.67:0/854972728 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fd12006d390 msgr2=0x7fd12015dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:10.840 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.830+0000 7fd125a9a700 1 --2- 172.21.15.67:0/854972728 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fd12006d390 0x7fd12015dec0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd11000b340 tx=0x7fd110004750 comp rx=0 tx=0).stop 2023-10-14T10:36:10.840 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.833+0000 7fd125a9a700 1 -- 172.21.15.67:0/854972728 shutdown_connections 2023-10-14T10:36:10.840 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.833+0000 7fd125a9a700 1 --2- 172.21.15.67:0/854972728 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fd10c06af00 0x7fd10c06d3c0 secure :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd1201586d0 tx=0x7fd114013040 comp rx=0 tx=0).stop 2023-10-14T10:36:10.841 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.833+0000 7fd125a9a700 1 --2- 172.21.15.67:0/854972728 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd12015a280 0x7fd1200acf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:10.841 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.833+0000 7fd125a9a700 1 --2- 172.21.15.67:0/854972728 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fd12006d390 0x7fd12015dec0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:10.841 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.833+0000 7fd125a9a700 1 --2- 172.21.15.67:0/854972728 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd1200ce5f0 0x7fd1200aca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:10.841 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.833+0000 7fd125a9a700 1 -- 172.21.15.67:0/854972728 >> 172.21.15.67:0/854972728 conn(0x7fd1200cfc40 msgr2=0x7fd120071760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:10.841 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.834+0000 7fd125a9a700 1 -- 172.21.15.67:0/854972728 shutdown_connections 2023-10-14T10:36:10.842 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:10.834+0000 7fd125a9a700 1 -- 172.21.15.67:0/854972728 wait complete. 2023-10-14T10:36:10.842 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542149 for osd.3 2023-10-14T10:36:10.844 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542149 for osd.2 2023-10-14T10:36:10.846 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542149 for osd.1 2023-10-14T10:36:10.850 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542149 for osd.7 2023-10-14T10:36:10.853 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542148 for osd.4 2023-10-14T10:36:10.857 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542150 for osd.6 2023-10-14T10:36:11.837 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-10-14T10:36:11.840 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-14T10:36:11.844 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-14T10:36:11.844 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-14T10:36:11.848 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-14T10:36:11.850 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-10-14T10:36:11.854 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-10-14T10:36:11.858 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-10-14T10:36:12.060 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.059+0000 7f39af8d4700 1 Processor -- start 2023-10-14T10:36:12.061 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.059+0000 7f39af8d4700 1 -- start start 2023-10-14T10:36:12.061 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.060+0000 7f1934830700 1 Processor -- start 2023-10-14T10:36:12.062 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.060+0000 7f39af8d4700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f39a8072a80 0x7f39a815dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:12.062 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.060+0000 7f39af8d4700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39a815d560 con 0x7f39a80726a0 2023-10-14T10:36:12.062 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.060+0000 7f39af8d4700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39a815db20 con 0x7f39a80b1580 2023-10-14T10:36:12.063 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.060+0000 7f39af8d4700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39a815e4f0 con 0x7f39a8072a80 2023-10-14T10:36:12.063 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.061+0000 7f39ad670700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f39a80726a0 0x7f39a80b0e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:51960/0 (socket says 172.21.15.67:51960) 2023-10-14T10:36:12.063 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.061+0000 7f39ad670700 1 -- 172.21.15.67:0/253006205 learned_addr learned my addr 172.21.15.67:0/253006205 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:36:12.063 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.061+0000 7f399ffff700 1 -- 172.21.15.67:0/253006205 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3715372677 0 0) 0x7f39a815d560 con 0x7f39a80726a0 2023-10-14T10:36:12.063 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.061+0000 7f399ffff700 1 -- 172.21.15.67:0/253006205 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39940035b0 con 0x7f39a80726a0 2023-10-14T10:36:12.064 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.060+0000 7f1934830700 1 -- start start 2023-10-14T10:36:12.064 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.062+0000 7f1934830700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f1930157250 0x7f193015df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:12.064 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.062+0000 7f1934830700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f193015af20 con 0x7f19300c9fc0 2023-10-14T10:36:12.065 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.062+0000 7f1934830700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f193015e470 con 0x7f1930157630 2023-10-14T10:36:12.065 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.062+0000 7f1934830700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f193015a6b0 con 0x7f1930157250 2023-10-14T10:36:12.065 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.062+0000 7f399ffff700 1 -- 172.21.15.67:0/253006205 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1649003875 0 0) 0x7f39940035b0 con 0x7f39a80726a0 2023-10-14T10:36:12.065 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.062+0000 7f399ffff700 1 -- 172.21.15.67:0/253006205 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f39a8072a80 msgr2=0x7f39a815dfb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-14T10:36:12.066 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.062+0000 7f399ffff700 1 --2- 172.21.15.67:0/253006205 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f39a8072a80 0x7f39a815dfb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:12.066 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.062+0000 7f399ffff700 1 -- 172.21.15.67:0/253006205 >> v1:172.21.15.72:6789/0 conn(0x7f39a80b1580 legacy=0x7f39a80c9a40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-14T10:36:12.066 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.062+0000 7f399ffff700 1 -- 172.21.15.67:0/253006205 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39a815aa70 con 0x7f39a80726a0 2023-10-14T10:36:12.066 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.062+0000 7f399ffff700 1 -- 172.21.15.67:0/253006205 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3180043755 0 0) 0x7f39a00019f0 con 0x7f39a80726a0 2023-10-14T10:36:12.066 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.062+0000 7f399ffff700 1 -- 172.21.15.67:0/253006205 >> v1:172.21.15.67:6789/0 conn(0x7f39a80726a0 legacy=0x7f39a80b0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:12.067 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.062+0000 7f399ffff700 1 --2- 172.21.15.67:0/253006205 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3994003a30 0x7f3994003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:12.067 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.062+0000 7f399ffff700 1 --2- 172.21.15.67:0/253006205 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3994004850 0x7f3994004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:12.067 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.062+0000 7f399ffff700 1 --2- 172.21.15.67:0/253006205 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f3994005410 0x7f399400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:12.067 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.062+0000 7f399ffff700 1 -- 172.21.15.67:0/253006205 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39a815e4f0 con 0x7f3994003a30 2023-10-14T10:36:12.067 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.062+0000 7f399ffff700 1 -- 172.21.15.67:0/253006205 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f399400c2a0 con 0x7f3994004850 2023-10-14T10:36:12.068 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.062+0000 7f399ffff700 1 -- 172.21.15.67:0/253006205 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f399400c460 con 0x7f3994005410 2023-10-14T10:36:12.068 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.062+0000 7f192e59c700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f1930157250 0x7f193015df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:12.068 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.062+0000 7f192e59c700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f1930157250 0x7f193015df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.67:38986/0 (socket says 172.21.15.67:38986) 2023-10-14T10:36:12.068 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.062+0000 7f192e59c700 1 -- 172.21.15.67:0/775610162 learned_addr learned my addr 172.21.15.67:0/775610162 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:12.069 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.062+0000 7f39ade71700 1 --2- 172.21.15.67:0/253006205 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f3994005410 0x7f399400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:12.069 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.063+0000 7f39ad670700 1 --2- 172.21.15.67:0/253006205 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3994003a30 0x7f3994003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:12.070 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.063+0000 7f192e59c700 1 -- 172.21.15.67:0/775610162 >> v1:172.21.15.72:6789/0 conn(0x7f1930157630 legacy=0x7f193006c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:12.070 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.063+0000 7f192e59c700 1 -- 172.21.15.67:0/775610162 >> v1:172.21.15.67:6789/0 conn(0x7f19300c9fc0 legacy=0x7f1930159fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:12.070 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.063+0000 7f192e59c700 1 -- 172.21.15.67:0/775610162 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f193015a960 con 0x7f1930157250 2023-10-14T10:36:12.071 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.063+0000 7f39ace6f700 1 --2- 172.21.15.67:0/253006205 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3994004850 0x7f3994004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:12.071 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.063+0000 7f39ade71700 1 -- 172.21.15.67:0/253006205 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3994004850 msgr2=0x7f3994004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:12.071 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.063+0000 7f39ade71700 1 --2- 172.21.15.67:0/253006205 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3994004850 0x7f3994004cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:12.072 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.063+0000 7f39ade71700 1 -- 172.21.15.67:0/253006205 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3994003a30 msgr2=0x7f3994003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:12.072 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.063+0000 7f39ade71700 1 --2- 172.21.15.67:0/253006205 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3994003a30 0x7f3994003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:12.072 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.063+0000 7f39ade71700 1 -- 172.21.15.67:0/253006205 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39a815aa70 con 0x7f3994005410 2023-10-14T10:36:12.072 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.063+0000 7f39ade71700 1 --2- 172.21.15.67:0/253006205 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f3994005410 0x7f399400ba10 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f39a4009850 tx=0x7f3994005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53ae682d6fbac216 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:12.073 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.064+0000 7f399ffff700 1 -- 172.21.15.67:0/253006205 <== mon.2 v2:172.21.15.72:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a40092f0 con 0x7f3994005410 2023-10-14T10:36:12.073 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.064+0000 7f399ffff700 1 -- 172.21.15.67:0/253006205 <== mon.2 v2:172.21.15.72:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39a4011620 con 0x7f3994005410 2023-10-14T10:36:12.073 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.064+0000 7f399ffff700 1 -- 172.21.15.67:0/253006205 <== mon.2 v2:172.21.15.72:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a40119b0 con 0x7f3994005410 2023-10-14T10:36:12.073 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.064+0000 7f192e59c700 1 --2- 172.21.15.67:0/775610162 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1930157250 0x7f193015df30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f192800d370 tx=0x7f192800d680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=625f14ae9712f317 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:12.074 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.065+0000 7f39af8d4700 1 -- 172.21.15.67:0/253006205 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f3994005410 msgr2=0x7f399400ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:12.074 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.065+0000 7f39af8d4700 1 --2- 172.21.15.67:0/253006205 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f3994005410 0x7f399400ba10 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f39a4009850 tx=0x7f3994005880 comp rx=0 tx=0).stop 2023-10-14T10:36:12.075 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.065+0000 7f39ade71700 1 -- 172.21.15.67:0/253006205 reap_dead start 2023-10-14T10:36:12.075 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.065+0000 7f192d59a700 1 -- 172.21.15.67:0/775610162 <== mon.1 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1928010070 con 0x7f1930157250 2023-10-14T10:36:12.075 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.065+0000 7f192d59a700 1 -- 172.21.15.67:0/775610162 <== mon.1 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f192800dc90 con 0x7f1930157250 2023-10-14T10:36:12.075 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.065+0000 7f192d59a700 1 -- 172.21.15.67:0/775610162 <== mon.1 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1928007b30 con 0x7f1930157250 2023-10-14T10:36:12.076 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.065+0000 7f39af8d4700 1 -- 172.21.15.67:0/253006205 shutdown_connections 2023-10-14T10:36:12.076 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.065+0000 7f39af8d4700 1 -- 172.21.15.67:0/253006205 >> 172.21.15.67:0/253006205 conn(0x7f39a806afb0 msgr2=0x7f39a80cbe70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:12.076 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.065+0000 7f39af8d4700 1 -- 172.21.15.67:0/253006205 shutdown_connections 2023-10-14T10:36:12.077 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.066+0000 7f1934830700 1 -- 172.21.15.67:0/775610162 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1930157250 msgr2=0x7f193015df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:12.077 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.066+0000 7f1934830700 1 --2- 172.21.15.67:0/775610162 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1930157250 0x7f193015df30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f192800d370 tx=0x7f192800d680 comp rx=0 tx=0).stop 2023-10-14T10:36:12.077 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.066+0000 7f1934830700 1 -- 172.21.15.67:0/775610162 shutdown_connections 2023-10-14T10:36:12.077 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.066+0000 7f1934830700 1 --2- 172.21.15.67:0/775610162 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1930157250 0x7f193015df30 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:12.077 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.066+0000 7f1934830700 1 -- 172.21.15.67:0/775610162 >> 172.21.15.67:0/775610162 conn(0x7f19300cf3f0 msgr2=0x7f19300cf800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:12.079 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.065+0000 7f39af8d4700 1 -- 172.21.15.67:0/253006205 wait complete. 2023-10-14T10:36:12.079 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.067+0000 7f39af8d4700 1 Processor -- start 2023-10-14T10:36:12.079 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.067+0000 7f39af8d4700 1 -- start start 2023-10-14T10:36:12.079 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.069+0000 7f39af8d4700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3994005410 0x7f39a814d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:12.080 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.069+0000 7f39af8d4700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f39a80aca30 0x7f39a80aceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:12.080 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.069+0000 7f39af8d4700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f39a80ad3f0 0x7f39a80c9520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:12.080 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.069+0000 7f39af8d4700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39a80a91a0 con 0x7f39a80ad3f0 2023-10-14T10:36:12.080 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.069+0000 7f39af8d4700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39a80a85d0 con 0x7f3994005410 2023-10-14T10:36:12.080 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.069+0000 7f39af8d4700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39a80a8870 con 0x7f39a80aca30 2023-10-14T10:36:12.081 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.069+0000 7f39ad670700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3994005410 0x7f39a814d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:12.081 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.069+0000 7f39ad670700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3994005410 0x7f39a814d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.67:38998/0 (socket says 172.21.15.67:38998) 2023-10-14T10:36:12.081 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.069+0000 7f39ad670700 1 -- 172.21.15.67:0/3346455091 learned_addr learned my addr 172.21.15.67:0/3346455091 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:12.081 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.069+0000 7f39ad670700 1 -- 172.21.15.67:0/3346455091 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f39a80aca30 msgr2=0x7f39a80aceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:12.081 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.069+0000 7f39ad670700 1 --2- 172.21.15.67:0/3346455091 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f39a80aca30 0x7f39a80aceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:12.082 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.069+0000 7f39ad670700 1 -- 172.21.15.67:0/3346455091 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f39a80ad3f0 msgr2=0x7f39a80c9520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:12.082 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.069+0000 7f39ad670700 1 --2- 172.21.15.67:0/3346455091 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f39a80ad3f0 0x7f39a80c9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:12.082 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.069+0000 7f39ad670700 1 -- 172.21.15.67:0/3346455091 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39a80a8b50 con 0x7f3994005410 2023-10-14T10:36:12.083 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.070+0000 7f39ad670700 1 --2- 172.21.15.67:0/3346455091 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3994005410 0x7f39a814d800 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f39a000c3d0 tx=0x7f39a000c790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:12.083 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.071+0000 7f399dffb700 1 -- 172.21.15.67:0/3346455091 <== mon.1 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a0015630 con 0x7f3994005410 2023-10-14T10:36:12.083 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.071+0000 7f399dffb700 1 -- 172.21.15.67:0/3346455091 <== mon.1 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39a000e050 con 0x7f3994005410 2023-10-14T10:36:12.083 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.071+0000 7f399dffb700 1 -- 172.21.15.67:0/3346455091 <== mon.1 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a00159d0 con 0x7f3994005410 2023-10-14T10:36:12.084 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.071+0000 7f39af8d4700 1 -- 172.21.15.67:0/3346455091 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39a80a8e30 con 0x7f3994005410 2023-10-14T10:36:12.084 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.071+0000 7f39af8d4700 1 -- 172.21.15.67:0/3346455091 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39a80ad870 con 0x7f3994005410 2023-10-14T10:36:12.085 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.072+0000 7f1934830700 1 -- 172.21.15.67:0/775610162 shutdown_connections 2023-10-14T10:36:12.085 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.072+0000 7f1934830700 1 -- 172.21.15.67:0/775610162 wait complete. 2023-10-14T10:36:12.085 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.074+0000 7f1934830700 1 Processor -- start 2023-10-14T10:36:12.085 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.074+0000 7f1934830700 1 -- start start 2023-10-14T10:36:12.085 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.075+0000 7f1934830700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f19300c9fc0 0x7f193006cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:12.086 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.075+0000 7f1934830700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1930157250 0x7f1930071f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:12.086 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.075+0000 7f1934830700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1930157630 0x7f19300ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:12.086 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.075+0000 7f1934830700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19301f7090 con 0x7f1930157250 2023-10-14T10:36:12.086 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.075+0000 7f1934830700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19300c3050 con 0x7f1930157630 2023-10-14T10:36:12.087 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.075+0000 7f1934830700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19300c35d0 con 0x7f19300c9fc0 2023-10-14T10:36:12.087 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.076+0000 7f192dd9b700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1930157250 0x7f1930071f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:12.087 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.076+0000 7f192dd9b700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1930157250 0x7f1930071f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44216/0 (socket says 172.21.15.67:44216) 2023-10-14T10:36:12.087 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.076+0000 7f192dd9b700 1 -- 172.21.15.67:0/94689802 learned_addr learned my addr 172.21.15.67:0/94689802 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:12.087 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.076+0000 7f192ed9d700 1 --2- 172.21.15.67:0/94689802 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1930157630 0x7f19300ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:12.088 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.075+0000 7f399dffb700 1 -- 172.21.15.67:0/3346455091 <== mon.1 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f39a0014420 con 0x7f3994005410 2023-10-14T10:36:12.088 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.076+0000 7f399dffb700 1 --2- 172.21.15.67:0/3346455091 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f398c06afd0 0x7f398c06d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:12.088 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.076+0000 7f399dffb700 1 -- 172.21.15.67:0/3346455091 <== mon.1 v2:172.21.15.72:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f39a0089f60 con 0x7f3994005410 2023-10-14T10:36:12.089 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.076+0000 7f192ed9d700 1 -- 172.21.15.67:0/94689802 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f19300c9fc0 msgr2=0x7f193006cf50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-14T10:36:12.089 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.076+0000 7f192ed9d700 1 --2- 172.21.15.67:0/94689802 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f19300c9fc0 0x7f193006cf50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:12.089 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.076+0000 7f192ed9d700 1 -- 172.21.15.67:0/94689802 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1930157250 msgr2=0x7f1930071f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:12.089 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.076+0000 7f192ed9d700 1 --2- 172.21.15.67:0/94689802 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1930157250 0x7f1930071f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:12.090 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.076+0000 7f192ed9d700 1 -- 172.21.15.67:0/94689802 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f192800d050 con 0x7f1930157630 2023-10-14T10:36:12.090 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.076+0000 7f192dd9b700 1 --2- 172.21.15.67:0/94689802 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1930157250 0x7f1930071f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-14T10:36:12.091 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.077+0000 7f39ace6f700 1 --2- 172.21.15.67:0/3346455091 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f398c06afd0 0x7f398c06d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:12.091 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.077+0000 7f39ace6f700 1 --2- 172.21.15.67:0/3346455091 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f398c06afd0 0x7f398c06d490 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f39a80719f0 tx=0x7f3998008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:12.091 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.078+0000 7f39af8d4700 1 -- 172.21.15.67:0/3346455091 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39880052f0 con 0x7f3994005410 2023-10-14T10:36:12.092 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.078+0000 7f192ed9d700 1 --2- 172.21.15.67:0/94689802 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1930157630 0x7f19300ac8b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f192400c410 tx=0x7f192400c720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:12.092 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.083+0000 7f191effd700 1 -- 172.21.15.67:0/94689802 <== mon.1 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1924012620 con 0x7f1930157630 2023-10-14T10:36:12.092 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.084+0000 7f191effd700 1 -- 172.21.15.67:0/94689802 <== mon.1 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1924012780 con 0x7f1930157630 2023-10-14T10:36:12.093 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.084+0000 7f1934830700 1 -- 172.21.15.67:0/94689802 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19301f6db0 con 0x7f1930157630 2023-10-14T10:36:12.093 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.084+0000 7f1934830700 1 -- 172.21.15.67:0/94689802 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19301f63c0 con 0x7f1930157630 2023-10-14T10:36:12.093 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.084+0000 7f191effd700 1 -- 172.21.15.67:0/94689802 <== mon.1 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1924012b30 con 0x7f1930157630 2023-10-14T10:36:12.093 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.086+0000 7f191effd700 1 -- 172.21.15.67:0/94689802 <== mon.1 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f1924012c90 con 0x7f1930157630 2023-10-14T10:36:12.094 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.086+0000 7f1934830700 1 -- 172.21.15.67:0/94689802 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19300af830 con 0x7f1930157630 2023-10-14T10:36:12.094 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.086+0000 7f191effd700 1 --2- 172.21.15.67:0/94689802 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f191006afd0 0x7f191006d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:12.094 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.086+0000 7f191effd700 1 -- 172.21.15.67:0/94689802 <== mon.1 v2:172.21.15.72:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1924088f00 con 0x7f1930157630 2023-10-14T10:36:12.094 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.086+0000 7f192e59c700 1 --2- 172.21.15.67:0/94689802 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f191006afd0 0x7f191006d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:12.094 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.083+0000 7f399dffb700 1 -- 172.21.15.67:0/3346455091 <== mon.1 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f39a00b6ca0 con 0x7f3994005410 2023-10-14T10:36:12.095 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.087+0000 7f192e59c700 1 --2- 172.21.15.67:0/94689802 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f191006afd0 0x7f191006d490 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f192800d6c0 tx=0x7f1928012000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:12.096 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.091+0000 7f191effd700 1 -- 172.21.15.67:0/94689802 <== mon.1 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f19300af830 con 0x7f1930157630 2023-10-14T10:36:12.097 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.096+0000 7ff43c2fd700 1 Processor -- start 2023-10-14T10:36:12.097 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.096+0000 7ff43c2fd700 1 -- start start 2023-10-14T10:36:12.098 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.098+0000 7ff43c2fd700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7ff434070c10 0x7ff43415f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:12.101 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.098+0000 7ff43c2fd700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff4340714a0 0x7ff43415c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:12.101 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.098+0000 7ff43c2fd700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4340b5150 con 0x7ff434070ff0 2023-10-14T10:36:12.102 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.098+0000 7ff43c2fd700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff434160260 con 0x7ff4340714a0 2023-10-14T10:36:12.102 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.098+0000 7ff43c2fd700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff43415f9c0 con 0x7ff434070c10 2023-10-14T10:36:12.102 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.098+0000 7ff43a89a700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff4340714a0 0x7ff43415c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:12.103 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.098+0000 7ff43a89a700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff4340714a0 0x7ff43415c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44226/0 (socket says 172.21.15.67:44226) 2023-10-14T10:36:12.103 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.098+0000 7ff43a89a700 1 -- 172.21.15.67:0/1874944010 learned_addr learned my addr 172.21.15.67:0/1874944010 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:12.103 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.099+0000 7ff43a89a700 1 -- 172.21.15.67:0/1874944010 >> v2:172.21.15.72:3300/0 conn(0x7ff434070c10 msgr2=0x7ff43415f300 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-14T10:36:12.103 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.099+0000 7ff43a89a700 1 --2- 172.21.15.67:0/1874944010 >> v2:172.21.15.72:3300/0 conn(0x7ff434070c10 0x7ff43415f300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:12.103 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.099+0000 7ff43a89a700 1 -- 172.21.15.67:0/1874944010 >> v1:172.21.15.67:6789/0 conn(0x7ff434070ff0 legacy=0x7ff43415ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:12.104 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.099+0000 7ff43a89a700 1 -- 172.21.15.67:0/1874944010 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff43415fca0 con 0x7ff4340714a0 2023-10-14T10:36:12.104 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.099+0000 7ff43a89a700 1 --2- 172.21.15.67:0/1874944010 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff4340714a0 0x7ff43415c3b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff43406d590 tx=0x7ff42c014ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0bd47d8b27df4af server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:12.104 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.099+0000 7ff439097700 1 -- 172.21.15.67:0/1874944010 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff42c003c20 con 0x7ff4340714a0 2023-10-14T10:36:12.104 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.099+0000 7ff439097700 1 -- 172.21.15.67:0/1874944010 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff42c003d80 con 0x7ff4340714a0 2023-10-14T10:36:12.104 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.099+0000 7ff439097700 1 -- 172.21.15.67:0/1874944010 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff42c013c30 con 0x7ff4340714a0 2023-10-14T10:36:12.105 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.100+0000 7ff43c2fd700 1 -- 172.21.15.67:0/1874944010 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff4340714a0 msgr2=0x7ff43415c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:12.105 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.100+0000 7ff43c2fd700 1 --2- 172.21.15.67:0/1874944010 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff4340714a0 0x7ff43415c3b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff43406d590 tx=0x7ff42c014ce0 comp rx=0 tx=0).stop 2023-10-14T10:36:12.105 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.103+0000 7ff43c2fd700 1 -- 172.21.15.67:0/1874944010 shutdown_connections 2023-10-14T10:36:12.106 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.103+0000 7ff43c2fd700 1 --2- 172.21.15.67:0/1874944010 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff4340714a0 0x7ff43415c3b0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:12.106 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.103+0000 7ff43c2fd700 1 --2- 172.21.15.67:0/1874944010 >> v2:172.21.15.72:3300/0 conn(0x7ff434070c10 0x7ff43415f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:12.106 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.103+0000 7ff43c2fd700 1 -- 172.21.15.67:0/1874944010 >> 172.21.15.67:0/1874944010 conn(0x7ff4340cf3f0 msgr2=0x7ff4340cf800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:12.106 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.103+0000 7ff43c2fd700 1 -- 172.21.15.67:0/1874944010 shutdown_connections 2023-10-14T10:36:12.107 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.103+0000 7ff43c2fd700 1 -- 172.21.15.67:0/1874944010 wait complete. 2023-10-14T10:36:12.107 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.105+0000 7ff43c2fd700 1 Processor -- start 2023-10-14T10:36:12.108 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.105+0000 7ff43c2fd700 1 -- start start 2023-10-14T10:36:12.109 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.106+0000 7ff43c2fd700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff434070c10 0x7ff4340b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:12.109 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.106+0000 7ff43c2fd700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff434070ff0 0x7ff4340b3420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:12.109 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.106+0000 7ff43c2fd700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7ff4340714a0 0x7ff4341506a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:12.109 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.106+0000 7ff43c2fd700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff43415aac0 con 0x7ff434070c10 2023-10-14T10:36:12.110 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.106+0000 7ff43c2fd700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4340c9aa0 con 0x7ff434070ff0 2023-10-14T10:36:12.110 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.106+0000 7ff43c2fd700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4340c9d80 con 0x7ff4340714a0 2023-10-14T10:36:12.110 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.106+0000 7ff43a099700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff434070c10 0x7ff4340b18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:12.110 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.106+0000 7ff43a099700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff434070c10 0x7ff4340b18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44228/0 (socket says 172.21.15.67:44228) 2023-10-14T10:36:12.110 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.107+0000 7ff43a099700 1 -- 172.21.15.67:0/954900309 learned_addr learned my addr 172.21.15.67:0/954900309 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:12.111 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.107+0000 7ff439898700 1 --2- 172.21.15.67:0/954900309 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff434070ff0 0x7ff4340b3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:12.111 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.107+0000 7ff43a89a700 1 --2- 172.21.15.67:0/954900309 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7ff4340714a0 0x7ff4341506a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:12.111 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.107+0000 7ff439898700 1 -- 172.21.15.67:0/954900309 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7ff4340714a0 msgr2=0x7ff4341506a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:12.111 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.107+0000 7ff439898700 1 --2- 172.21.15.67:0/954900309 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7ff4340714a0 0x7ff4341506a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:12.111 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.107+0000 7ff439898700 1 -- 172.21.15.67:0/954900309 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff434070c10 msgr2=0x7ff4340b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:12.112 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.107+0000 7ff439898700 1 --2- 172.21.15.67:0/954900309 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff434070c10 0x7ff4340b18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:12.112 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.107+0000 7ff439898700 1 -- 172.21.15.67:0/954900309 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff42c014660 con 0x7ff434070ff0 2023-10-14T10:36:12.112 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.107+0000 7ff43a099700 1 --2- 172.21.15.67:0/954900309 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff434070c10 0x7ff4340b18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-14T10:36:12.112 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.107+0000 7ff439898700 1 --2- 172.21.15.67:0/954900309 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff434070ff0 0x7ff4340b3420 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff42400c750 tx=0x7ff42400ca60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:12.112 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.107+0000 7ff42affd700 1 -- 172.21.15.67:0/954900309 <== mon.1 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4240054e0 con 0x7ff434070ff0 2023-10-14T10:36:12.113 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.107+0000 7ff42affd700 1 -- 172.21.15.67:0/954900309 <== mon.1 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff424003680 con 0x7ff434070ff0 2023-10-14T10:36:12.113 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.107+0000 7ff43c2fd700 1 -- 172.21.15.67:0/954900309 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff43415ada0 con 0x7ff434070ff0 2023-10-14T10:36:12.113 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.107+0000 7ff42affd700 1 -- 172.21.15.67:0/954900309 <== mon.1 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff424003a30 con 0x7ff434070ff0 2023-10-14T10:36:12.113 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.108+0000 7ff43c2fd700 1 -- 172.21.15.67:0/954900309 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4340b3960 con 0x7ff434070ff0 2023-10-14T10:36:12.114 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.109+0000 7ff43c2fd700 1 -- 172.21.15.67:0/954900309 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff43404ebb0 con 0x7ff434070ff0 2023-10-14T10:36:12.114 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.109+0000 7ff42affd700 1 -- 172.21.15.67:0/954900309 <== mon.1 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff424003b90 con 0x7ff434070ff0 2023-10-14T10:36:12.115 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.110+0000 7ff42affd700 1 --2- 172.21.15.67:0/954900309 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7ff42007c090 0x7ff42007e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:12.115 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.110+0000 7ff42affd700 1 -- 172.21.15.67:0/954900309 <== mon.1 v2:172.21.15.72:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff4240889a0 con 0x7ff434070ff0 2023-10-14T10:36:12.115 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.110+0000 7ff43a099700 1 --2- 172.21.15.67:0/954900309 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7ff42007c090 0x7ff42007e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:12.115 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.111+0000 7ff43a099700 1 --2- 172.21.15.67:0/954900309 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7ff42007c090 0x7ff42007e550 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff434151ca0 tx=0x7ff430009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:12.116 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.115+0000 7ff42affd700 1 -- 172.21.15.67:0/954900309 <== mon.1 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff424053a50 con 0x7ff434070ff0 2023-10-14T10:36:12.120 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.119+0000 7f65c777d700 1 Processor -- start 2023-10-14T10:36:12.120 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.119+0000 7f65c777d700 1 -- start start 2023-10-14T10:36:12.122 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.121+0000 7f65c777d700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f65c015da70 0x7f65c00ca660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:12.123 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.121+0000 7f65c777d700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f65c006d0c0 0x7f65c00709a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:12.123 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.121+0000 7f65c777d700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65c00ce360 con 0x7f65c00ccfc0 2023-10-14T10:36:12.123 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.121+0000 7f65c777d700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65c00cd790 con 0x7f65c006d0c0 2023-10-14T10:36:12.123 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.121+0000 7f65c777d700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65c00cf040 con 0x7f65c015da70 2023-10-14T10:36:12.124 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.121+0000 7f65c4d18700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f65c00ccfc0 0x7f65c006c9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:52006/0 (socket says 172.21.15.67:52006) 2023-10-14T10:36:12.124 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.121+0000 7f65c4d18700 1 -- 172.21.15.67:0/1166499033 learned_addr learned my addr 172.21.15.67:0/1166499033 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:36:12.124 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.121+0000 7fab496fb700 1 Processor -- start 2023-10-14T10:36:12.125 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.121+0000 7fab496fb700 1 -- start start 2023-10-14T10:36:12.125 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.121+0000 7f3b1c04a700 1 Processor -- start 2023-10-14T10:36:12.126 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.122+0000 7f65b7fff700 1 -- 172.21.15.67:0/1166499033 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1606241143 0 0) 0x7f65c00ce360 con 0x7f65c00ccfc0 2023-10-14T10:36:12.126 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.122+0000 7f65b7fff700 1 -- 172.21.15.67:0/1166499033 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65ac0035b0 con 0x7f65c00ccfc0 2023-10-14T10:36:12.126 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.122+0000 7f65c5d1a700 1 --2- 172.21.15.67:0/1166499033 >> v2:172.21.15.72:3300/0 conn(0x7f65c006d0c0 0x7f65c00709a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:12.126 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.122+0000 7f65b7fff700 1 -- 172.21.15.67:0/1166499033 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2673001570 0 0) 0x7f65ac0035b0 con 0x7f65c00ccfc0 2023-10-14T10:36:12.127 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.123+0000 7f65b7fff700 1 -- 172.21.15.67:0/1166499033 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f65c015da70 msgr2=0x7f65c00ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:12.127 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.123+0000 7f65b7fff700 1 --2- 172.21.15.67:0/1166499033 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f65c015da70 0x7f65c00ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:12.127 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.123+0000 7f65b7fff700 1 -- 172.21.15.67:0/1166499033 >> v2:172.21.15.72:3300/0 conn(0x7f65c006d0c0 msgr2=0x7f65c00709a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:12.127 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.123+0000 7f65b7fff700 1 --2- 172.21.15.67:0/1166499033 >> v2:172.21.15.72:3300/0 conn(0x7f65c006d0c0 0x7f65c00709a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:12.127 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.123+0000 7f65b7fff700 1 -- 172.21.15.67:0/1166499033 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65c00cda30 con 0x7f65c00ccfc0 2023-10-14T10:36:12.128 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.123+0000 7f65b7fff700 1 -- 172.21.15.67:0/1166499033 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3180043755 0 0) 0x7f65b0001670 con 0x7f65c00ccfc0 2023-10-14T10:36:12.128 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.123+0000 7f65b7fff700 1 -- 172.21.15.67:0/1166499033 >> v1:172.21.15.67:6789/0 conn(0x7f65c00ccfc0 legacy=0x7f65c006c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:12.128 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.123+0000 7f65b7fff700 1 --2- 172.21.15.67:0/1166499033 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f65c015da70 0x7f65ac003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:12.129 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.123+0000 7f65b7fff700 1 --2- 172.21.15.67:0/1166499033 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f65ac003fd0 0x7f65ac004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:12.129 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.123+0000 7f65b7fff700 1 --2- 172.21.15.67:0/1166499033 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65ac004d90 0x7f65ac005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:12.129 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.123+0000 7f65b7fff700 1 -- 172.21.15.67:0/1166499033 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65c00cd790 con 0x7f65ac004d90 2023-10-14T10:36:12.129 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.123+0000 7f65b7fff700 1 -- 172.21.15.67:0/1166499033 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65c00cf040 con 0x7f65ac003fd0 2023-10-14T10:36:12.129 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.123+0000 7f65b7fff700 1 -- 172.21.15.67:0/1166499033 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65ac00bba0 con 0x7f65c015da70 2023-10-14T10:36:12.130 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.123+0000 7fab496fb700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fab440cc710 0x7fab440cdda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:12.130 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.123+0000 7fab496fb700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab440b1160 con 0x7fab44161160 2023-10-14T10:36:12.131 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.123+0000 7fab496fb700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab440b13b0 con 0x7fab44160d80 2023-10-14T10:36:12.131 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.123+0000 7fab496fb700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab440b1690 con 0x7fab440cc710 2023-10-14T10:36:12.131 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.123+0000 7fab42ffd700 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7fab44160d80 0x7fab440cc000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.67:60250/0 (socket says 172.21.15.67:60250) 2023-10-14T10:36:12.131 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.123+0000 7fab42ffd700 1 -- 172.21.15.67:0/626474598 learned_addr learned my addr 172.21.15.67:0/626474598 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:36:12.132 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.123+0000 7f65c4d18700 1 --2- 172.21.15.67:0/1166499033 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f65ac003fd0 0x7f65ac004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:12.132 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.122+0000 7f3b1c04a700 1 -- start start 2023-10-14T10:36:12.133 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.123+0000 7f3b1c04a700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f3b14157630 0x7f3b1406c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:12.133 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.123+0000 7fab427fc700 1 --2- 172.21.15.67:0/626474598 >> v2:172.21.15.72:3300/0 conn(0x7fab440cc710 0x7fab440cdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:12.133 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.123+0000 7f3b1c04a700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b1415af20 con 0x7f3b140c9fc0 2023-10-14T10:36:12.134 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.123+0000 7f3b1c04a700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b1415a6b0 con 0x7f3b14157250 2023-10-14T10:36:12.134 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.123+0000 7f3b1c04a700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b1415a960 con 0x7f3b14157630 2023-10-14T10:36:12.134 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.124+0000 7f3b1a5e7700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f3b14157630 0x7f3b1406c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:12.134 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.124+0000 7f65c5d1a700 1 --2- 172.21.15.67:0/1166499033 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65ac004d90 0x7f65ac005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:12.135 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.124+0000 7f3b1a5e7700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f3b14157630 0x7f3b1406c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.67:39042/0 (socket says 172.21.15.67:39042) 2023-10-14T10:36:12.135 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.124+0000 7f3b1a5e7700 1 -- 172.21.15.67:0/3472095057 learned_addr learned my addr 172.21.15.67:0/3472095057 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:12.136 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.124+0000 7f65c4d18700 1 -- 172.21.15.67:0/1166499033 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f65c015da70 msgr2=0x7f65ac003a00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-14T10:36:12.136 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.124+0000 7f65c4d18700 1 --2- 172.21.15.67:0/1166499033 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f65c015da70 0x7f65ac003a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:12.136 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.124+0000 7f65c4d18700 1 -- 172.21.15.67:0/1166499033 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65ac004d90 msgr2=0x7f65ac005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:12.136 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.124+0000 7f65c4d18700 1 --2- 172.21.15.67:0/1166499033 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65ac004d90 0x7f65ac005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:12.137 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.124+0000 7f65c4d18700 1 -- 172.21.15.67:0/1166499033 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65ac0035b0 con 0x7f65ac003fd0 2023-10-14T10:36:12.137 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.124+0000 7f3b1a5e7700 1 -- 172.21.15.67:0/3472095057 >> v1:172.21.15.72:6789/0 conn(0x7f3b14157250 legacy=0x7f3b1415df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:12.137 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.124+0000 7f3b1a5e7700 1 -- 172.21.15.67:0/3472095057 >> v1:172.21.15.67:6789/0 conn(0x7f3b140c9fc0 legacy=0x7f3b14159fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:12.137 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.124+0000 7f3b1a5e7700 1 -- 172.21.15.67:0/3472095057 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b1415ac40 con 0x7f3b14157630 2023-10-14T10:36:12.138 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.124+0000 7fab41ffb700 1 -- 172.21.15.67:0/626474598 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3561782727 0 0) 0x7fab440b1160 con 0x7fab44161160 2023-10-14T10:36:12.138 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.124+0000 7fab41ffb700 1 -- 172.21.15.67:0/626474598 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab2c0035b0 con 0x7fab44161160 2023-10-14T10:36:12.138 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.124+0000 7f3b1a5e7700 1 --2- 172.21.15.67:0/3472095057 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3b14157630 0x7f3b1406c9b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c009e50 tx=0x7f3b0c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70bee9c548d15f7c server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:12.139 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.125+0000 7fab41ffb700 1 -- 172.21.15.67:0/626474598 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 736584042 0 0) 0x7fab2c0035b0 con 0x7fab44161160 2023-10-14T10:36:12.139 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.125+0000 7fab41ffb700 1 -- 172.21.15.67:0/626474598 >> v2:172.21.15.72:3300/0 conn(0x7fab440cc710 msgr2=0x7fab440cdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:12.139 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.125+0000 7fab41ffb700 1 --2- 172.21.15.67:0/626474598 >> v2:172.21.15.72:3300/0 conn(0x7fab440cc710 0x7fab440cdda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:12.139 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.125+0000 7fab41ffb700 1 -- 172.21.15.67:0/626474598 >> v1:172.21.15.72:6789/0 conn(0x7fab44160d80 legacy=0x7fab440cc000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:12.140 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.125+0000 7fab41ffb700 1 -- 172.21.15.67:0/626474598 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab440b1970 con 0x7fab44161160 2023-10-14T10:36:12.140 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.125+0000 7fab41ffb700 1 -- 172.21.15.67:0/626474598 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3180043755 0 0) 0x7fab3c0019f0 con 0x7fab44161160 2023-10-14T10:36:12.140 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.125+0000 7fab41ffb700 1 -- 172.21.15.67:0/626474598 >> v1:172.21.15.67:6789/0 conn(0x7fab44161160 legacy=0x7fab440c9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:12.140 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.125+0000 7fab41ffb700 1 --2- 172.21.15.67:0/626474598 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab2c003a00 0x7fab2c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:12.140 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.125+0000 7fab41ffb700 1 --2- 172.21.15.67:0/626474598 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fab2c004850 0x7fab2c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:12.141 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.125+0000 7fab41ffb700 1 --2- 172.21.15.67:0/626474598 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fab2c005410 0x7fab2c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:12.141 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.125+0000 7fab41ffb700 1 -- 172.21.15.67:0/626474598 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab440b1690 con 0x7fab2c003a00 2023-10-14T10:36:12.141 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.125+0000 7fab41ffb700 1 -- 172.21.15.67:0/626474598 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab2c00c2d0 con 0x7fab2c004850 2023-10-14T10:36:12.141 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.125+0000 7fab41ffb700 1 -- 172.21.15.67:0/626474598 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab2c00c490 con 0x7fab2c005410 2023-10-14T10:36:12.142 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.125+0000 7f3b18de4700 1 -- 172.21.15.67:0/3472095057 <== mon.1 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b0c003c20 con 0x7f3b14157630 2023-10-14T10:36:12.142 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.125+0000 7f3b18de4700 1 -- 172.21.15.67:0/3472095057 <== mon.1 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b0c003d80 con 0x7f3b14157630 2023-10-14T10:36:12.142 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.125+0000 7f3b18de4700 1 -- 172.21.15.67:0/3472095057 <== mon.1 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b0c00bb80 con 0x7f3b14157630 2023-10-14T10:36:12.143 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.125+0000 7f65c4d18700 1 --2- 172.21.15.67:0/1166499033 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f65ac003fd0 0x7f65ac004850 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f65b000c160 tx=0x7f65ac005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e3dd6d662d144c5 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:12.143 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.125+0000 7fab437fe700 1 --2- 172.21.15.67:0/626474598 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fab2c005410 0x7fab2c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:12.143 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.126+0000 7f65b7fff700 1 -- 172.21.15.67:0/1166499033 <== mon.1 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65b0007b70 con 0x7f65ac003fd0 2023-10-14T10:36:12.144 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.126+0000 7f65b7fff700 1 -- 172.21.15.67:0/1166499033 <== mon.1 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65b000cd00 con 0x7f65ac003fd0 2023-10-14T10:36:12.144 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.126+0000 7f65b7fff700 1 -- 172.21.15.67:0/1166499033 <== mon.1 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65b0005ba0 con 0x7f65ac003fd0 2023-10-14T10:36:12.144 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.126+0000 7fab437fe700 1 -- 172.21.15.67:0/626474598 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fab2c004850 msgr2=0x7fab2c004cd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-14T10:36:12.144 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.126+0000 7fab437fe700 1 --2- 172.21.15.67:0/626474598 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fab2c004850 0x7fab2c004cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:12.145 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.126+0000 7fab437fe700 1 -- 172.21.15.67:0/626474598 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab2c003a00 msgr2=0x7fab2c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:12.145 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.126+0000 7fab437fe700 1 --2- 172.21.15.67:0/626474598 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab2c003a00 0x7fab2c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:12.145 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.126+0000 7fab437fe700 1 -- 172.21.15.67:0/626474598 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab440b1970 con 0x7fab2c005410 2023-10-14T10:36:12.145 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.126+0000 7fab437fe700 1 -- 172.21.15.67:0/626474598 reap_dead start 2023-10-14T10:36:12.146 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.124+0000 7f27fc4f8700 1 Processor -- start 2023-10-14T10:36:12.146 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.124+0000 7f27fc4f8700 1 -- start start 2023-10-14T10:36:12.146 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.126+0000 7f27fc4f8700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f27f40c2840 0x7f27f4166620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:12.147 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.126+0000 7f27fc4f8700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f27f40c9c50 0x7f27f40cde60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:12.147 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.126+0000 7f27fc4f8700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27f40ca9f0 con 0x7f27f40ce3a0 2023-10-14T10:36:12.147 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.126+0000 7f27fc4f8700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27f4162490 con 0x7f27f40c2840 2023-10-14T10:36:12.147 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.126+0000 7f27fc4f8700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27f40ca430 con 0x7f27f40c9c50 2023-10-14T10:36:12.148 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.126+0000 7f3b1c04a700 1 -- 172.21.15.67:0/3472095057 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3b14157630 msgr2=0x7f3b1406c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:12.148 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.126+0000 7f3b1c04a700 1 --2- 172.21.15.67:0/3472095057 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3b14157630 0x7f3b1406c9b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c009e50 tx=0x7f3b0c00b330 comp rx=0 tx=0).stop 2023-10-14T10:36:12.148 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.126+0000 7fab437fe700 1 --2- 172.21.15.67:0/626474598 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fab2c005410 0x7fab2c00ba10 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fab44161560 tx=0x7fab2c005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8cfbfb0d325094ea server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:12.149 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.127+0000 7f3b1c04a700 1 -- 172.21.15.67:0/3472095057 shutdown_connections 2023-10-14T10:36:12.149 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.127+0000 7f3b1c04a700 1 --2- 172.21.15.67:0/3472095057 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3b14157630 0x7f3b1406c9b0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:12.149 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.127+0000 7f3b1c04a700 1 -- 172.21.15.67:0/3472095057 >> 172.21.15.67:0/3472095057 conn(0x7f3b140cf3f0 msgr2=0x7f3b140cf800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:12.150 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.127+0000 7fab41ffb700 1 -- 172.21.15.67:0/626474598 <== mon.2 v2:172.21.15.72:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab3c012620 con 0x7fab2c005410 2023-10-14T10:36:12.150 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.127+0000 7fab41ffb700 1 -- 172.21.15.67:0/626474598 <== mon.2 v2:172.21.15.72:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab3c0127c0 con 0x7fab2c005410 2023-10-14T10:36:12.150 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.127+0000 7fab41ffb700 1 -- 172.21.15.67:0/626474598 <== mon.2 v2:172.21.15.72:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab3c012b70 con 0x7fab2c005410 2023-10-14T10:36:12.151 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.126+0000 7f27faa95700 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f27f40ce3a0 0x7f27f406c9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.67:60272/0 (socket says 172.21.15.67:60272) 2023-10-14T10:36:12.151 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.126+0000 7f27faa95700 1 -- 172.21.15.67:0/2567837334 learned_addr learned my addr 172.21.15.67:0/2567837334 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:36:12.151 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.126+0000 7f27fa294700 1 --2- 172.21.15.67:0/2567837334 >> v2:172.21.15.67:3300/0 conn(0x7f27f40c2840 0x7f27f4166620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:12.151 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.127+0000 7f27fa294700 1 -- 172.21.15.67:0/2567837334 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f27f40c9c50 msgr2=0x7f27f40cde60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-14T10:36:12.152 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.127+0000 7f27fa294700 1 --2- 172.21.15.67:0/2567837334 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f27f40c9c50 0x7f27f40cde60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:12.152 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.127+0000 7f27fa294700 1 -- 172.21.15.67:0/2567837334 >> v1:172.21.15.72:6789/0 conn(0x7f27f40ce3a0 legacy=0x7f27f406c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:12.152 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.127+0000 7f27fa294700 1 -- 172.21.15.67:0/2567837334 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27f40ca710 con 0x7f27f40c2840 2023-10-14T10:36:12.152 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.127+0000 7f27fa294700 1 --2- 172.21.15.67:0/2567837334 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f27f40c2840 0x7f27f4166620 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f27f0009870 tx=0x7f27f0009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57ba82dccce24b78 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:12.152 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.127+0000 7f3b1c04a700 1 -- 172.21.15.67:0/3472095057 shutdown_connections 2023-10-14T10:36:12.153 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.127+0000 7f65c777d700 1 -- 172.21.15.67:0/1166499033 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f65ac003fd0 msgr2=0x7f65ac004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:12.153 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.127+0000 7f65c777d700 1 --2- 172.21.15.67:0/1166499033 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f65ac003fd0 0x7f65ac004850 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f65b000c160 tx=0x7f65ac005880 comp rx=0 tx=0).stop 2023-10-14T10:36:12.154 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.127+0000 7f27f9292700 1 -- 172.21.15.67:0/2567837334 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f0004230 con 0x7f27f40c2840 2023-10-14T10:36:12.154 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.127+0000 7f27f9292700 1 -- 172.21.15.67:0/2567837334 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27f0004390 con 0x7f27f40c2840 2023-10-14T10:36:12.154 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.127+0000 7f27f9292700 1 -- 172.21.15.67:0/2567837334 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f00036a0 con 0x7f27f40c2840 2023-10-14T10:36:12.155 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.128+0000 7f65c5d1a700 1 -- 172.21.15.67:0/1166499033 reap_dead start 2023-10-14T10:36:12.155 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.128+0000 7f65c777d700 1 -- 172.21.15.67:0/1166499033 shutdown_connections 2023-10-14T10:36:12.155 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.128+0000 7f65c777d700 1 -- 172.21.15.67:0/1166499033 >> 172.21.15.67:0/1166499033 conn(0x7f65c00cf3f0 msgr2=0x7f65c00cf800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:12.155 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.127+0000 7f3b1c04a700 1 -- 172.21.15.67:0/3472095057 wait complete. 2023-10-14T10:36:12.156 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.128+0000 7fab496fb700 1 -- 172.21.15.67:0/626474598 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fab2c005410 msgr2=0x7fab2c00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:12.156 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.128+0000 7fab496fb700 1 --2- 172.21.15.67:0/626474598 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fab2c005410 0x7fab2c00ba10 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fab44161560 tx=0x7fab2c005880 comp rx=0 tx=0).stop 2023-10-14T10:36:12.156 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.128+0000 7fab496fb700 1 -- 172.21.15.67:0/626474598 shutdown_connections 2023-10-14T10:36:12.157 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.128+0000 7fab496fb700 1 --2- 172.21.15.67:0/626474598 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fab2c005410 0x7fab2c00ba10 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:12.157 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.128+0000 7fab496fb700 1 -- 172.21.15.67:0/626474598 >> 172.21.15.67:0/626474598 conn(0x7fab4406a840 msgr2=0x7fab4406ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:12.157 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.128+0000 7f27fc4f8700 1 -- 172.21.15.67:0/2567837334 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f27f40c2840 msgr2=0x7f27f4166620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:12.157 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.128+0000 7f27fc4f8700 1 --2- 172.21.15.67:0/2567837334 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f27f40c2840 0x7f27f4166620 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f27f0009870 tx=0x7f27f0009b80 comp rx=0 tx=0).stop 2023-10-14T10:36:12.158 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.129+0000 7f65c777d700 1 -- 172.21.15.67:0/1166499033 shutdown_connections 2023-10-14T10:36:12.158 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.129+0000 7f27fc4f8700 1 -- 172.21.15.67:0/2567837334 shutdown_connections 2023-10-14T10:36:12.158 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.129+0000 7f27fc4f8700 1 --2- 172.21.15.67:0/2567837334 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f27f40c2840 0x7f27f4166620 secure :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f27f0009870 tx=0x7f27f0009b80 comp rx=0 tx=0).stop 2023-10-14T10:36:12.159 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.129+0000 7f27fc4f8700 1 --2- 172.21.15.67:0/2567837334 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f27f40c9c50 0x7f27f40cde60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:12.159 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.129+0000 7f27fc4f8700 1 -- 172.21.15.67:0/2567837334 >> 172.21.15.67:0/2567837334 conn(0x7f27f40cf3f0 msgr2=0x7f27f40cf800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:12.159 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.129+0000 7f65c777d700 1 -- 172.21.15.67:0/1166499033 wait complete. 2023-10-14T10:36:12.160 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.129+0000 7f27fc4f8700 1 -- 172.21.15.67:0/2567837334 shutdown_connections 2023-10-14T10:36:12.160 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.129+0000 7f3b1c04a700 1 Processor -- start 2023-10-14T10:36:12.160 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.129+0000 7f3b1c04a700 1 -- start start 2023-10-14T10:36:12.161 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.130+0000 7f3b1c04a700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3b14157630 0x7f3b140c6d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:12.161 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.130+0000 7f3b1c04a700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f3b14154920 0x7f3b140c8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:12.161 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.131+0000 7f3b1c04a700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3b140c90f0 0x7f3b14153cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:12.161 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.131+0000 7f3b1c04a700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b14158730 con 0x7f3b14157630 2023-10-14T10:36:12.162 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.131+0000 7f3b1c04a700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b14157af0 con 0x7f3b140c90f0 2023-10-14T10:36:12.162 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.131+0000 7f3b1c04a700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b14157d90 con 0x7f3b14154920 2023-10-14T10:36:12.162 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.129+0000 7f27fc4f8700 1 -- 172.21.15.67:0/2567837334 wait complete. 2023-10-14T10:36:12.162 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.131+0000 7f27fc4f8700 1 Processor -- start 2023-10-14T10:36:12.163 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.131+0000 7f65c777d700 1 Processor -- start 2023-10-14T10:36:12.163 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.132+0000 7f65c777d700 1 -- start start 2023-10-14T10:36:12.163 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.131+0000 7f27fc4f8700 1 -- start start 2023-10-14T10:36:12.164 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.132+0000 7f27fc4f8700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f27f40c9c50 0x7f27f40c1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:12.164 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.132+0000 7f27fc4f8700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f27f40ce3a0 0x7f27f40c5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:12.164 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.132+0000 7f27fc4f8700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f27f40c6f40 0x7f27f4203b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:12.164 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.132+0000 7f27fc4f8700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27f40c74e0 con 0x7f27f40c6f40 2023-10-14T10:36:12.164 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.132+0000 7f27fc4f8700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27f40c77c0 con 0x7f27f40ce3a0 2023-10-14T10:36:12.164 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.132+0000 7f27fc4f8700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27f40c7aa0 con 0x7f27f40c9c50 2023-10-14T10:36:12.165 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.131+0000 7f3b19de6700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3b14157630 0x7f3b140c6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:12.165 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.131+0000 7f3b19de6700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3b14157630 0x7f3b140c6d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44270/0 (socket says 172.21.15.67:44270) 2023-10-14T10:36:12.165 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.131+0000 7f3b19de6700 1 -- 172.21.15.67:0/4133065744 learned_addr learned my addr 172.21.15.67:0/4133065744 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:12.165 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.131+0000 7f3b19de6700 1 -- 172.21.15.67:0/4133065744 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f3b14154920 msgr2=0x7f3b140c8bb0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-14T10:36:12.166 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.131+0000 7f3b19de6700 1 --2- 172.21.15.67:0/4133065744 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f3b14154920 0x7f3b140c8bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:12.166 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.131+0000 7f3b19de6700 1 -- 172.21.15.67:0/4133065744 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3b140c90f0 msgr2=0x7f3b14153cb0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-14T10:36:12.166 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.131+0000 7f3b19de6700 1 --2- 172.21.15.67:0/4133065744 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3b140c90f0 0x7f3b14153cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:12.166 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.131+0000 7f3b19de6700 1 -- 172.21.15.67:0/4133065744 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b0c009b30 con 0x7f3b14157630 2023-10-14T10:36:12.167 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.132+0000 7f3b19de6700 1 --2- 172.21.15.67:0/4133065744 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3b14157630 0x7f3b140c6d20 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3b1000c430 tx=0x7f3b1000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:12.167 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.132+0000 7f3b0a7fc700 1 -- 172.21.15.67:0/4133065744 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b10011790 con 0x7f3b14157630 2023-10-14T10:36:12.169 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.132+0000 7f27faa95700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f27f40c6f40 0x7f27f4203b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:12.169 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.133+0000 7f27faa95700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f27f40c6f40 0x7f27f4203b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44276/0 (socket says 172.21.15.67:44276) 2023-10-14T10:36:12.169 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.133+0000 7f27faa95700 1 -- 172.21.15.67:0/1792481374 learned_addr learned my addr 172.21.15.67:0/1792481374 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:12.170 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.133+0000 7f27f9a93700 1 --2- 172.21.15.67:0/1792481374 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f27f40ce3a0 0x7f27f40c5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:12.170 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.133+0000 7f27fa294700 1 --2- 172.21.15.67:0/1792481374 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f27f40c9c50 0x7f27f40c1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:12.170 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.133+0000 7f27faa95700 1 -- 172.21.15.67:0/1792481374 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f27f40c9c50 msgr2=0x7f27f40c1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:12.170 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.133+0000 7f27faa95700 1 --2- 172.21.15.67:0/1792481374 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f27f40c9c50 0x7f27f40c1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:12.170 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.133+0000 7f27faa95700 1 -- 172.21.15.67:0/1792481374 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f27f40ce3a0 msgr2=0x7f27f40c5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:12.171 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.133+0000 7f27faa95700 1 --2- 172.21.15.67:0/1792481374 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f27f40ce3a0 0x7f27f40c5580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:12.171 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.133+0000 7f27faa95700 1 -- 172.21.15.67:0/1792481374 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27f00094a0 con 0x7f27f40c6f40 2023-10-14T10:36:12.171 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.133+0000 7f27fa294700 1 --2- 172.21.15.67:0/1792481374 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f27f40c9c50 0x7f27f40c1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-14T10:36:12.171 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.133+0000 7f27faa95700 1 --2- 172.21.15.67:0/1792481374 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f27f40c6f40 0x7f27f4203b70 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f27ec00cc10 tx=0x7f27ec00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:12.172 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.133+0000 7f65c777d700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f65ac003fd0 0x7f65c00b4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:12.172 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.133+0000 7f65c777d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65c0153a30 0x7f65c00c2860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:12.172 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.133+0000 7f65c777d700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f65c015f1a0 0x7f65c015f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:12.172 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.133+0000 7f65c777d700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65c01512f0 con 0x7f65c0153a30 2023-10-14T10:36:12.172 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.133+0000 7f65c777d700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65c0151590 con 0x7f65c015f1a0 2023-10-14T10:36:12.173 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.133+0000 7f65c777d700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65c00c5570 con 0x7f65ac003fd0 2023-10-14T10:36:12.173 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.134+0000 7f65c4d18700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65c0153a30 0x7f65c00c2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:12.173 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.134+0000 7f65c4d18700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65c0153a30 0x7f65c00c2860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44282/0 (socket says 172.21.15.67:44282) 2023-10-14T10:36:12.173 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.134+0000 7f65c5d1a700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f65c015f1a0 0x7f65c015f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:12.173 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.134+0000 7f65c4d18700 1 -- 172.21.15.67:0/2392525451 learned_addr learned my addr 172.21.15.67:0/2392525451 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:12.174 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.134+0000 7f27eaffd700 1 -- 172.21.15.67:0/1792481374 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27ec00ab90 con 0x7f27f40c6f40 2023-10-14T10:36:12.174 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.134+0000 7f27eaffd700 1 -- 172.21.15.67:0/1792481374 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27ec00acf0 con 0x7f27f40c6f40 2023-10-14T10:36:12.174 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.134+0000 7f27eaffd700 1 -- 172.21.15.67:0/1792481374 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27ec007310 con 0x7f27f40c6f40 2023-10-14T10:36:12.175 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.135+0000 7f65c5d1a700 1 -- 172.21.15.67:0/2392525451 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f65ac003fd0 msgr2=0x7f65c00b4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:12.175 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.135+0000 7f65c5d1a700 1 --2- 172.21.15.67:0/2392525451 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f65ac003fd0 0x7f65c00b4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:12.175 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.135+0000 7f65c5d1a700 1 -- 172.21.15.67:0/2392525451 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65c0153a30 msgr2=0x7f65c00c2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:12.175 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.135+0000 7f65c5d1a700 1 --2- 172.21.15.67:0/2392525451 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65c0153a30 0x7f65c00c2860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:12.176 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.135+0000 7f65c5d1a700 1 -- 172.21.15.67:0/2392525451 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65c00c5850 con 0x7f65c015f1a0 2023-10-14T10:36:12.176 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.136+0000 7f65c5d1a700 1 --2- 172.21.15.67:0/2392525451 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f65c015f1a0 0x7f65c015f630 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f65c006f4d0 tx=0x7f65bc011770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:12.176 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.136+0000 7fab496fb700 1 -- 172.21.15.67:0/626474598 shutdown_connections 2023-10-14T10:36:12.177 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.132+0000 7f3b1c04a700 1 -- 172.21.15.67:0/4133065744 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b14158070 con 0x7f3b14157630 2023-10-14T10:36:12.177 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.132+0000 7f3b0a7fc700 1 -- 172.21.15.67:0/4133065744 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b100118f0 con 0x7f3b14157630 2023-10-14T10:36:12.177 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.132+0000 7f3b0a7fc700 1 -- 172.21.15.67:0/4133065744 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b10011ca0 con 0x7f3b14157630 2023-10-14T10:36:12.177 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.134+0000 7f3b1c04a700 1 -- 172.21.15.67:0/4133065744 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b140c7260 con 0x7f3b14157630 2023-10-14T10:36:12.177 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.136+0000 7f65b5ffb700 1 -- 172.21.15.67:0/2392525451 <== mon.1 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65bc004230 con 0x7f65c015f1a0 2023-10-14T10:36:12.178 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.136+0000 7f65b5ffb700 1 -- 172.21.15.67:0/2392525451 <== mon.1 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65bc004390 con 0x7f65c015f1a0 2023-10-14T10:36:12.178 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.136+0000 7f65b5ffb700 1 -- 172.21.15.67:0/2392525451 <== mon.1 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65bc0038e0 con 0x7f65c015f1a0 2023-10-14T10:36:12.178 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.135+0000 7f27fc4f8700 1 -- 172.21.15.67:0/1792481374 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27f40c7d80 con 0x7f27f40c6f40 2023-10-14T10:36:12.179 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.135+0000 7f27fc4f8700 1 -- 172.21.15.67:0/1792481374 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27f4163c20 con 0x7f27f40c6f40 2023-10-14T10:36:12.179 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.136+0000 7f3b1c04a700 1 -- 172.21.15.67:0/4133065744 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b14158870 con 0x7f3b14157630 2023-10-14T10:36:12.179 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.137+0000 7f65c777d700 1 -- 172.21.15.67:0/2392525451 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65c00c5b30 con 0x7f65c015f1a0 2023-10-14T10:36:12.180 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.137+0000 7f65c777d700 1 -- 172.21.15.67:0/2392525451 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65c00c2da0 con 0x7f65c015f1a0 2023-10-14T10:36:12.180 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.139+0000 7f27fc4f8700 1 -- 172.21.15.67:0/1792481374 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27f4163ea0 con 0x7f27f40c6f40 2023-10-14T10:36:12.181 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.139+0000 7f65b5ffb700 1 -- 172.21.15.67:0/2392525451 <== mon.1 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f65bc003a40 con 0x7f65c015f1a0 2023-10-14T10:36:12.181 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.140+0000 7f65c777d700 1 -- 172.21.15.67:0/2392525451 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65c00c5d20 con 0x7f65c015f1a0 2023-10-14T10:36:12.181 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.140+0000 7f65b5ffb700 1 --2- 172.21.15.67:0/2392525451 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f65a406af50 0x7f65a406d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:12.182 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.140+0000 7f65b5ffb700 1 -- 172.21.15.67:0/2392525451 <== mon.1 v2:172.21.15.72:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f65bc089440 con 0x7f65c015f1a0 2023-10-14T10:36:12.182 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.140+0000 7f65c5519700 1 --2- 172.21.15.67:0/2392525451 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f65a406af50 0x7f65a406d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:12.182 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.141+0000 7f3b0a7fc700 1 -- 172.21.15.67:0/4133065744 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f3b1001c460 con 0x7f3b14157630 2023-10-14T10:36:12.183 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.141+0000 7f3b0a7fc700 1 --2- 172.21.15.67:0/4133065744 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f3b0006b150 0x7f3b0006d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:12.183 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.141+0000 7f3b0a7fc700 1 -- 172.21.15.67:0/4133065744 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3b100892a0 con 0x7f3b14157630 2023-10-14T10:36:12.183 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.142+0000 7f27eaffd700 1 -- 172.21.15.67:0/1792481374 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f27ec007470 con 0x7f27f40c6f40 2023-10-14T10:36:12.184 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.143+0000 7f27eaffd700 1 --2- 172.21.15.67:0/1792481374 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f27d806af00 0x7f27d806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:12.184 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.143+0000 7f27eaffd700 1 -- 172.21.15.67:0/1792481374 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f27ec089400 con 0x7f27f40c6f40 2023-10-14T10:36:12.184 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.143+0000 7f27fa294700 1 --2- 172.21.15.67:0/1792481374 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f27d806af00 0x7f27d806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:12.184 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.144+0000 7f3b195e5700 1 --2- 172.21.15.67:0/4133065744 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f3b0006b150 0x7f3b0006d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:12.185 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.144+0000 7f65c5519700 1 --2- 172.21.15.67:0/2392525451 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f65a406af50 0x7f65a406d410 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f65b800ab10 tx=0x7f65b8009250 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:12.185 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.145+0000 7f27fa294700 1 --2- 172.21.15.67:0/1792481374 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f27d806af00 0x7f27d806d3c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f27f40cbe40 tx=0x7f27f0008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:12.186 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.145+0000 7f65b5ffb700 1 -- 172.21.15.67:0/2392525451 <== mon.1 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f65bc0549a0 con 0x7f65c015f1a0 2023-10-14T10:36:12.187 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.141+0000 7fab496fb700 1 -- 172.21.15.67:0/626474598 wait complete. 2023-10-14T10:36:12.187 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.144+0000 7fab496fb700 1 Processor -- start 2023-10-14T10:36:12.187 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.144+0000 7fab496fb700 1 -- start start 2023-10-14T10:36:12.187 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.145+0000 7fab496fb700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fab2c005410 0x7fab44152770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:12.187 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.145+0000 7fab496fb700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab44154430 0x7fab44154bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:12.188 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.145+0000 7fab496fb700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fab44155100 0x7fab440c77e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:12.188 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.145+0000 7fab496fb700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab44155590 con 0x7fab44154430 2023-10-14T10:36:12.188 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.145+0000 7fab496fb700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab4415de60 con 0x7fab2c005410 2023-10-14T10:36:12.188 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.145+0000 7fab496fb700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab4415e100 con 0x7fab44155100 2023-10-14T10:36:12.188 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.145+0000 7fab42ffd700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fab2c005410 0x7fab44152770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:12.189 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.145+0000 7fab42ffd700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fab2c005410 0x7fab44152770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.67:39082/0 (socket says 172.21.15.67:39082) 2023-10-14T10:36:12.189 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.145+0000 7fab42ffd700 1 -- 172.21.15.67:0/2056394436 learned_addr learned my addr 172.21.15.67:0/2056394436 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:12.189 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.145+0000 7fab437fe700 1 --2- 172.21.15.67:0/2056394436 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fab44155100 0x7fab440c77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:12.189 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.146+0000 7fab42ffd700 1 -- 172.21.15.67:0/2056394436 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fab44155100 msgr2=0x7fab440c77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:12.189 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.146+0000 7fab42ffd700 1 --2- 172.21.15.67:0/2056394436 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fab44155100 0x7fab440c77e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:12.190 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.146+0000 7fab42ffd700 1 -- 172.21.15.67:0/2056394436 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab44154430 msgr2=0x7fab44154bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:12.190 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.146+0000 7fab42ffd700 1 --2- 172.21.15.67:0/2056394436 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab44154430 0x7fab44154bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:12.190 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.146+0000 7fab42ffd700 1 -- 172.21.15.67:0/2056394436 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab4415e3e0 con 0x7fab2c005410 2023-10-14T10:36:12.191 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.145+0000 7f3b195e5700 1 --2- 172.21.15.67:0/4133065744 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f3b0006b150 0x7f3b0006d610 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3b14154d40 tx=0x7f3b04007a20 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:12.191 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.153+0000 7fab42ffd700 1 --2- 172.21.15.67:0/2056394436 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fab2c005410 0x7fab44152770 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fab3400c730 tx=0x7fab3400ca40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:12.192 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.153+0000 7fab2b7fe700 1 -- 172.21.15.67:0/2056394436 <== mon.1 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab34005d50 con 0x7fab2c005410 2023-10-14T10:36:12.192 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.153+0000 7fab2b7fe700 1 -- 172.21.15.67:0/2056394436 <== mon.1 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab340070d0 con 0x7fab2c005410 2023-10-14T10:36:12.192 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.153+0000 7fab2b7fe700 1 -- 172.21.15.67:0/2056394436 <== mon.1 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab34007480 con 0x7fab2c005410 2023-10-14T10:36:12.192 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.153+0000 7fab496fb700 1 -- 172.21.15.67:0/2056394436 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab4415e6c0 con 0x7fab2c005410 2023-10-14T10:36:12.192 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.153+0000 7fab496fb700 1 -- 172.21.15.67:0/2056394436 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab441548b0 con 0x7fab2c005410 2023-10-14T10:36:12.193 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.154+0000 7f3b0a7fc700 1 -- 172.21.15.67:0/4133065744 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3b14158870 con 0x7f3b14157630 2023-10-14T10:36:12.193 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.156+0000 7fab2b7fe700 1 -- 172.21.15.67:0/2056394436 <== mon.1 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fab340075e0 con 0x7fab2c005410 2023-10-14T10:36:12.193 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.157+0000 7fab2b7fe700 1 --2- 172.21.15.67:0/2056394436 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fab3006af00 0x7fab3006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:12.194 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.157+0000 7fab2b7fe700 1 -- 172.21.15.67:0/2056394436 <== mon.1 v2:172.21.15.72:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fab34088b80 con 0x7fab2c005410 2023-10-14T10:36:12.194 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.155+0000 7f27eaffd700 1 -- 172.21.15.67:0/1792481374 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f27ec054590 con 0x7f27f40c6f40 2023-10-14T10:36:12.195 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.162+0000 7fab496fb700 1 -- 172.21.15.67:0/2056394436 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab4415ebe0 con 0x7fab2c005410 2023-10-14T10:36:12.195 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.167+0000 7fab427fc700 1 --2- 172.21.15.67:0/2056394436 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fab3006af00 0x7fab3006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:12.195 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.167+0000 7fab2b7fe700 1 -- 172.21.15.67:0/2056394436 <== mon.1 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fab4415ebe0 con 0x7fab2c005410 2023-10-14T10:36:12.195 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.172+0000 7fab427fc700 1 --2- 172.21.15.67:0/2056394436 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fab3006af00 0x7fab3006d3c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fab440c9210 tx=0x7fab38008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:12.198 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.177+0000 7f713c25f700 1 Processor -- start 2023-10-14T10:36:12.198 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.177+0000 7f713c25f700 1 -- start start 2023-10-14T10:36:12.198 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.179+0000 7f713c25f700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f7134157250 0x7f713415df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:12.198 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.179+0000 7f713c25f700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f7134157630 0x7f713406c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:12.198 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.179+0000 7f713c25f700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f713415af20 con 0x7f71340c9fc0 2023-10-14T10:36:12.199 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.179+0000 7f713c25f700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f713415e470 con 0x7f7134157630 2023-10-14T10:36:12.199 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.179+0000 7f713c25f700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f713415a6b0 con 0x7f7134157250 2023-10-14T10:36:12.199 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.180+0000 7f713a7fc700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f7134157630 0x7f713406c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:12.199 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.180+0000 7f713a7fc700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f7134157630 0x7f713406c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.67:39092/0 (socket says 172.21.15.67:39092) 2023-10-14T10:36:12.199 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.180+0000 7f713a7fc700 1 -- 172.21.15.67:0/580023321 learned_addr learned my addr 172.21.15.67:0/580023321 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:12.200 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.180+0000 7f7139ffb700 1 --2- 172.21.15.67:0/580023321 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f7134157250 0x7f713415df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:12.200 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.181+0000 7f713a7fc700 1 -- 172.21.15.67:0/580023321 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f7134157250 msgr2=0x7f713415df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:12.200 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.181+0000 7f713a7fc700 1 --2- 172.21.15.67:0/580023321 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f7134157250 0x7f713415df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:12.200 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.181+0000 7f713a7fc700 1 -- 172.21.15.67:0/580023321 >> v1:172.21.15.67:6789/0 conn(0x7f71340c9fc0 legacy=0x7f7134159fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:12.200 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.181+0000 7f713a7fc700 1 -- 172.21.15.67:0/580023321 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f713415a960 con 0x7f7134157630 2023-10-14T10:36:12.201 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.181+0000 7f7139ffb700 1 --2- 172.21.15.67:0/580023321 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f7134157250 0x7f713415df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-14T10:36:12.201 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.181+0000 7f713a7fc700 1 --2- 172.21.15.67:0/580023321 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7134157630 0x7f713406c9b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f712c00d370 tx=0x7f712c00d680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec472ee7b997c9ea server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:12.201 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.182+0000 7f7138ff9700 1 -- 172.21.15.67:0/580023321 <== mon.1 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712c010070 con 0x7f7134157630 2023-10-14T10:36:12.201 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.182+0000 7f7138ff9700 1 -- 172.21.15.67:0/580023321 <== mon.1 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f712c00dc20 con 0x7f7134157630 2023-10-14T10:36:12.201 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.182+0000 7f7138ff9700 1 -- 172.21.15.67:0/580023321 <== mon.1 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712c007b30 con 0x7f7134157630 2023-10-14T10:36:12.202 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.183+0000 7f713c25f700 1 -- 172.21.15.67:0/580023321 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7134157630 msgr2=0x7f713406c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:12.202 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.183+0000 7f713c25f700 1 --2- 172.21.15.67:0/580023321 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7134157630 0x7f713406c9b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f712c00d370 tx=0x7f712c00d680 comp rx=0 tx=0).stop 2023-10-14T10:36:12.202 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.183+0000 7f713c25f700 1 -- 172.21.15.67:0/580023321 shutdown_connections 2023-10-14T10:36:12.202 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.183+0000 7f713c25f700 1 --2- 172.21.15.67:0/580023321 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7134157630 0x7f713406c9b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:12.202 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.183+0000 7f713c25f700 1 --2- 172.21.15.67:0/580023321 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f7134157250 0x7f713415df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:12.203 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.183+0000 7f713c25f700 1 -- 172.21.15.67:0/580023321 >> 172.21.15.67:0/580023321 conn(0x7f71340cf3f0 msgr2=0x7f71340cf800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:12.203 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.183+0000 7f713c25f700 1 -- 172.21.15.67:0/580023321 shutdown_connections 2023-10-14T10:36:12.203 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.183+0000 7f713c25f700 1 -- 172.21.15.67:0/580023321 wait complete. 2023-10-14T10:36:12.203 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.185+0000 7f713c25f700 1 Processor -- start 2023-10-14T10:36:12.203 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.186+0000 7f713c25f700 1 -- start start 2023-10-14T10:36:12.204 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.187+0000 7f713c25f700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f71340c9fc0 0x7f7134153500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:12.204 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.187+0000 7f713c25f700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7134157250 0x7f71340af820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:12.204 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.187+0000 7f713c25f700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7134157630 0x7f71341520d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:12.204 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.187+0000 7f713c25f700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71341581f0 con 0x7f7134157250 2023-10-14T10:36:12.204 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.187+0000 7f713c25f700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71340b0120 con 0x7f7134157630 2023-10-14T10:36:12.204 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.187+0000 7f713c25f700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71340b0400 con 0x7f71340c9fc0 2023-10-14T10:36:12.205 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.188+0000 7f71397fa700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7134157250 0x7f71340af820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:12.205 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.188+0000 7f71397fa700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7134157250 0x7f71340af820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44306/0 (socket says 172.21.15.67:44306) 2023-10-14T10:36:12.205 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.188+0000 7f71397fa700 1 -- 172.21.15.67:0/167551441 learned_addr learned my addr 172.21.15.67:0/167551441 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:12.205 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.188+0000 7f71397fa700 1 -- 172.21.15.67:0/167551441 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f71340c9fc0 msgr2=0x7f7134153500 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-14T10:36:12.205 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.188+0000 7f71397fa700 1 --2- 172.21.15.67:0/167551441 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f71340c9fc0 0x7f7134153500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:12.206 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.188+0000 7f71397fa700 1 -- 172.21.15.67:0/167551441 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7134157630 msgr2=0x7f71341520d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-14T10:36:12.206 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.188+0000 7f71397fa700 1 --2- 172.21.15.67:0/167551441 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7134157630 0x7f71341520d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:12.206 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.188+0000 7f71397fa700 1 -- 172.21.15.67:0/167551441 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71300094a0 con 0x7f7134157250 2023-10-14T10:36:12.206 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.189+0000 7f71397fa700 1 --2- 172.21.15.67:0/167551441 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7134157250 0x7f71340af820 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f712400b450 tx=0x7f712400b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:12.207 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.190+0000 7f712a7fc700 1 -- 172.21.15.67:0/167551441 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7124011830 con 0x7f7134157250 2023-10-14T10:36:12.207 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.190+0000 7f712a7fc700 1 -- 172.21.15.67:0/167551441 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7124011990 con 0x7f7134157250 2023-10-14T10:36:12.207 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.190+0000 7f712a7fc700 1 -- 172.21.15.67:0/167551441 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7124011d20 con 0x7f7134157250 2023-10-14T10:36:12.207 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.190+0000 7f713c25f700 1 -- 172.21.15.67:0/167551441 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f712c00d050 con 0x7f7134157250 2023-10-14T10:36:12.207 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.190+0000 7f713c25f700 1 -- 172.21.15.67:0/167551441 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7134157e80 con 0x7f7134157250 2023-10-14T10:36:12.207 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.194+0000 7f712a7fc700 1 -- 172.21.15.67:0/167551441 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f712401a400 con 0x7f7134157250 2023-10-14T10:36:12.208 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.195+0000 7f712a7fc700 1 --2- 172.21.15.67:0/167551441 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f711806afd0 0x7f711806d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:12.208 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.195+0000 7f712a7fc700 1 -- 172.21.15.67:0/167551441 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7124089470 con 0x7f7134157250 2023-10-14T10:36:12.208 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.196+0000 7f713c25f700 1 -- 172.21.15.67:0/167551441 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f711c0052f0 con 0x7f7134157250 2023-10-14T10:36:12.209 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.206+0000 7f712a7fc700 1 -- 172.21.15.67:0/167551441 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7124054590 con 0x7f7134157250 2023-10-14T10:36:12.209 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.206+0000 7f7139ffb700 1 --2- 172.21.15.67:0/167551441 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f711806afd0 0x7f711806d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:12.244 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.240+0000 7f7139ffb700 1 --2- 172.21.15.67:0/167551441 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f711806afd0 0x7f711806d490 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7130009b80 tx=0x7f7130009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:12.479 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.478+0000 7f39af8d4700 1 -- 172.21.15.67:0/3346455091 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f39880055a0 con 0x7f3994005410 2023-10-14T10:36:12.480 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.479+0000 7f399dffb700 1 -- 172.21.15.67:0/3346455091 <== mon.1 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f39a0055080 con 0x7f3994005410 2023-10-14T10:36:12.483 INFO:teuthology.orchestra.run.smithi067.stdout:60129542149 2023-10-14T10:36:12.484 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.483+0000 7f39af8d4700 1 -- 172.21.15.67:0/3346455091 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f398c06afd0 msgr2=0x7f398c06d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:12.484 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.483+0000 7f39af8d4700 1 --2- 172.21.15.67:0/3346455091 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f398c06afd0 0x7f398c06d490 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f39a80719f0 tx=0x7f3998008040 comp rx=0 tx=0).stop 2023-10-14T10:36:12.484 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.483+0000 7f39af8d4700 1 -- 172.21.15.67:0/3346455091 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3994005410 msgr2=0x7f39a814d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:12.485 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.483+0000 7f39af8d4700 1 --2- 172.21.15.67:0/3346455091 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3994005410 0x7f39a814d800 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f39a000c3d0 tx=0x7f39a000c790 comp rx=0 tx=0).stop 2023-10-14T10:36:12.485 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.483+0000 7f39af8d4700 1 -- 172.21.15.67:0/3346455091 shutdown_connections 2023-10-14T10:36:12.486 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.483+0000 7f39af8d4700 1 --2- 172.21.15.67:0/3346455091 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f398c06afd0 0x7f398c06d490 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:12.486 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.483+0000 7f39af8d4700 1 --2- 172.21.15.67:0/3346455091 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f39a80ad3f0 0x7f39a80c9520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:12.486 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.483+0000 7f39af8d4700 1 --2- 172.21.15.67:0/3346455091 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3994005410 0x7f39a814d800 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:12.486 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.483+0000 7f39af8d4700 1 --2- 172.21.15.67:0/3346455091 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f39a80aca30 0x7f39a80aceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:12.486 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.483+0000 7f39af8d4700 1 -- 172.21.15.67:0/3346455091 >> 172.21.15.67:0/3346455091 conn(0x7f39a806afb0 msgr2=0x7f39a815c7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:12.487 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.484+0000 7f39af8d4700 1 -- 172.21.15.67:0/3346455091 shutdown_connections 2023-10-14T10:36:12.487 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.484+0000 7f39af8d4700 1 -- 172.21.15.67:0/3346455091 wait complete. 2023-10-14T10:36:12.500 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.499+0000 7f1934830700 1 -- 172.21.15.67:0/94689802 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f19300af830 con 0x7f1930157630 2023-10-14T10:36:12.500 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.499+0000 7f191effd700 1 -- 172.21.15.67:0/94689802 <== mon.1 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f19300af830 con 0x7f1930157630 2023-10-14T10:36:12.501 INFO:teuthology.orchestra.run.smithi067.stdout:60129542150 2023-10-14T10:36:12.504 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.0 2023-10-14T10:36:12.504 DEBUG:teuthology.parallel:result is None 2023-10-14T10:36:12.505 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.503+0000 7f1934830700 1 -- 172.21.15.67:0/94689802 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f191006afd0 msgr2=0x7f191006d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:12.505 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.503+0000 7f1934830700 1 --2- 172.21.15.67:0/94689802 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f191006afd0 0x7f191006d490 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f192800d6c0 tx=0x7f1928012000 comp rx=0 tx=0).stop 2023-10-14T10:36:12.505 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.503+0000 7f1934830700 1 -- 172.21.15.67:0/94689802 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1930157630 msgr2=0x7f19300ac8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:12.505 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.503+0000 7f1934830700 1 --2- 172.21.15.67:0/94689802 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1930157630 0x7f19300ac8b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f192400c410 tx=0x7f192400c720 comp rx=0 tx=0).stop 2023-10-14T10:36:12.506 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.504+0000 7f1934830700 1 -- 172.21.15.67:0/94689802 shutdown_connections 2023-10-14T10:36:12.506 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.504+0000 7f1934830700 1 --2- 172.21.15.67:0/94689802 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f191006afd0 0x7f191006d490 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:12.506 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.504+0000 7f1934830700 1 --2- 172.21.15.67:0/94689802 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1930157630 0x7f19300ac8b0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:12.506 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.504+0000 7f1934830700 1 --2- 172.21.15.67:0/94689802 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f19300c9fc0 0x7f193006cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:12.507 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.504+0000 7f1934830700 1 --2- 172.21.15.67:0/94689802 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1930157250 0x7f1930071f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:12.507 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.504+0000 7f1934830700 1 -- 172.21.15.67:0/94689802 >> 172.21.15.67:0/94689802 conn(0x7f19300cf3f0 msgr2=0x7f19300b0ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:12.507 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.504+0000 7f1934830700 1 -- 172.21.15.67:0/94689802 shutdown_connections 2023-10-14T10:36:12.507 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.504+0000 7f1934830700 1 -- 172.21.15.67:0/94689802 wait complete. 2023-10-14T10:36:12.524 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.5 2023-10-14T10:36:12.524 DEBUG:teuthology.parallel:result is None 2023-10-14T10:36:12.535 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.534+0000 7ff43c2fd700 1 -- 172.21.15.67:0/954900309 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7ff4340c9be0 con 0x7ff434070ff0 2023-10-14T10:36:12.535 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.535+0000 7ff42affd700 1 -- 172.21.15.67:0/954900309 <== mon.1 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff4340c9be0 con 0x7ff434070ff0 2023-10-14T10:36:12.536 INFO:teuthology.orchestra.run.smithi067.stdout:60129542150 2023-10-14T10:36:12.540 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.539+0000 7ff43c2fd700 1 -- 172.21.15.67:0/954900309 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7ff42007c090 msgr2=0x7ff42007e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:12.540 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.539+0000 7ff43c2fd700 1 --2- 172.21.15.67:0/954900309 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7ff42007c090 0x7ff42007e550 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff434151ca0 tx=0x7ff430009190 comp rx=0 tx=0).stop 2023-10-14T10:36:12.541 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.539+0000 7ff43c2fd700 1 -- 172.21.15.67:0/954900309 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff434070ff0 msgr2=0x7ff4340b3420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:12.541 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.539+0000 7ff43c2fd700 1 --2- 172.21.15.67:0/954900309 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff434070ff0 0x7ff4340b3420 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff42400c750 tx=0x7ff42400ca60 comp rx=0 tx=0).stop 2023-10-14T10:36:12.541 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.539+0000 7ff43c2fd700 1 -- 172.21.15.67:0/954900309 shutdown_connections 2023-10-14T10:36:12.541 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.539+0000 7ff43c2fd700 1 --2- 172.21.15.67:0/954900309 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7ff42007c090 0x7ff42007e550 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:12.541 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.539+0000 7ff43c2fd700 1 --2- 172.21.15.67:0/954900309 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7ff4340714a0 0x7ff4341506a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:12.542 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.539+0000 7ff43c2fd700 1 --2- 172.21.15.67:0/954900309 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff434070c10 0x7ff4340b18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:12.542 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.539+0000 7ff43c2fd700 1 --2- 172.21.15.67:0/954900309 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff434070ff0 0x7ff4340b3420 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:12.542 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.539+0000 7ff43c2fd700 1 -- 172.21.15.67:0/954900309 >> 172.21.15.67:0/954900309 conn(0x7ff4340cf3f0 msgr2=0x7ff4340ce650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:12.542 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.539+0000 7ff43c2fd700 1 -- 172.21.15.67:0/954900309 shutdown_connections 2023-10-14T10:36:12.542 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.539+0000 7ff43c2fd700 1 -- 172.21.15.67:0/954900309 wait complete. 2023-10-14T10:36:12.546 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.545+0000 7f3b1c04a700 1 -- 172.21.15.67:0/4133065744 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f3b14158870 con 0x7f3b14157630 2023-10-14T10:36:12.546 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.546+0000 7f3b0a7fc700 1 -- 172.21.15.67:0/4133065744 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3b14158870 con 0x7f3b14157630 2023-10-14T10:36:12.547 INFO:teuthology.orchestra.run.smithi067.stdout:60129542151 2023-10-14T10:36:12.549 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.548+0000 7f3b1c04a700 1 -- 172.21.15.67:0/4133065744 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f3b0006b150 msgr2=0x7f3b0006d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:12.549 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.548+0000 7f3b1c04a700 1 --2- 172.21.15.67:0/4133065744 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f3b0006b150 0x7f3b0006d610 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3b14154d40 tx=0x7f3b04007a20 comp rx=0 tx=0).stop 2023-10-14T10:36:12.549 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.548+0000 7f3b1c04a700 1 -- 172.21.15.67:0/4133065744 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3b14157630 msgr2=0x7f3b140c6d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:12.549 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.548+0000 7f3b1c04a700 1 --2- 172.21.15.67:0/4133065744 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3b14157630 0x7f3b140c6d20 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3b1000c430 tx=0x7f3b1000c7f0 comp rx=0 tx=0).stop 2023-10-14T10:36:12.550 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.549+0000 7f3b1c04a700 1 -- 172.21.15.67:0/4133065744 shutdown_connections 2023-10-14T10:36:12.550 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.549+0000 7f3b1c04a700 1 --2- 172.21.15.67:0/4133065744 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f3b0006b150 0x7f3b0006d610 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:12.551 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.549+0000 7f3b1c04a700 1 --2- 172.21.15.67:0/4133065744 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3b140c90f0 0x7f3b14153cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:12.551 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.549+0000 7f3b1c04a700 1 --2- 172.21.15.67:0/4133065744 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3b14157630 0x7f3b140c6d20 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:12.551 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.549+0000 7f3b1c04a700 1 --2- 172.21.15.67:0/4133065744 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f3b14154920 0x7f3b140c8bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:12.551 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.549+0000 7f3b1c04a700 1 -- 172.21.15.67:0/4133065744 >> 172.21.15.67:0/4133065744 conn(0x7f3b140cf3f0 msgr2=0x7f3b14072750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:12.551 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.549+0000 7f3b1c04a700 1 -- 172.21.15.67:0/4133065744 shutdown_connections 2023-10-14T10:36:12.552 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.549+0000 7f3b1c04a700 1 -- 172.21.15.67:0/4133065744 wait complete. 2023-10-14T10:36:12.559 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.1 2023-10-14T10:36:12.559 DEBUG:teuthology.parallel:result is None 2023-10-14T10:36:12.560 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.559+0000 7f65c777d700 1 -- 172.21.15.67:0/2392525451 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f65c004ebb0 con 0x7f65c015f1a0 2023-10-14T10:36:12.561 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.560+0000 7f65b5ffb700 1 -- 172.21.15.67:0/2392525451 <== mon.1 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f65bc057bf0 con 0x7f65c015f1a0 2023-10-14T10:36:12.561 INFO:teuthology.orchestra.run.smithi067.stdout:60129542150 2023-10-14T10:36:12.563 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542151 for osd.6 2023-10-14T10:36:12.564 DEBUG:teuthology.parallel:result is None 2023-10-14T10:36:12.564 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.562+0000 7f65c777d700 1 -- 172.21.15.67:0/2392525451 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f65a406af50 msgr2=0x7f65a406d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:12.564 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.562+0000 7f65c777d700 1 --2- 172.21.15.67:0/2392525451 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f65a406af50 0x7f65a406d410 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f65b800ab10 tx=0x7f65b8009250 comp rx=0 tx=0).stop 2023-10-14T10:36:12.565 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.562+0000 7f65c777d700 1 -- 172.21.15.67:0/2392525451 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f65c015f1a0 msgr2=0x7f65c015f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:12.565 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.562+0000 7f65c777d700 1 --2- 172.21.15.67:0/2392525451 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f65c015f1a0 0x7f65c015f630 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f65c006f4d0 tx=0x7f65bc011770 comp rx=0 tx=0).stop 2023-10-14T10:36:12.568 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.567+0000 7f65c777d700 1 -- 172.21.15.67:0/2392525451 shutdown_connections 2023-10-14T10:36:12.568 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.567+0000 7f65c777d700 1 --2- 172.21.15.67:0/2392525451 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f65a406af50 0x7f65a406d410 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:12.568 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.567+0000 7f65c777d700 1 --2- 172.21.15.67:0/2392525451 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f65c015f1a0 0x7f65c015f630 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:12.568 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.567+0000 7f65c777d700 1 --2- 172.21.15.67:0/2392525451 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f65ac003fd0 0x7f65c00b4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:12.569 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.567+0000 7f65c777d700 1 --2- 172.21.15.67:0/2392525451 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65c0153a30 0x7f65c00c2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:12.569 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.567+0000 7f65c777d700 1 -- 172.21.15.67:0/2392525451 >> 172.21.15.67:0/2392525451 conn(0x7f65c00cf3f0 msgr2=0x7f65c015c310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:12.569 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.567+0000 7f65c777d700 1 -- 172.21.15.67:0/2392525451 shutdown_connections 2023-10-14T10:36:12.569 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.567+0000 7f65c777d700 1 -- 172.21.15.67:0/2392525451 wait complete. 2023-10-14T10:36:12.570 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.567+0000 7f27fc4f8700 1 -- 172.21.15.67:0/1792481374 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f27f404ebb0 con 0x7f27f40c6f40 2023-10-14T10:36:12.570 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.568+0000 7f27eaffd700 1 -- 172.21.15.67:0/1792481374 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f27ec018090 con 0x7f27f40c6f40 2023-10-14T10:36:12.571 INFO:teuthology.orchestra.run.smithi067.stdout:60129542150 2023-10-14T10:36:12.571 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.569+0000 7f713c25f700 1 -- 172.21.15.67:0/167551441 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f711c0055a0 con 0x7f7134157250 2023-10-14T10:36:12.572 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.569+0000 7f712a7fc700 1 -- 172.21.15.67:0/167551441 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7124015030 con 0x7f7134157250 2023-10-14T10:36:12.572 INFO:teuthology.orchestra.run.smithi067.stdout:60129542149 2023-10-14T10:36:12.572 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.571+0000 7f27fc4f8700 1 -- 172.21.15.67:0/1792481374 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f27d806af00 msgr2=0x7f27d806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:12.573 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.571+0000 7f27fc4f8700 1 --2- 172.21.15.67:0/1792481374 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f27d806af00 0x7f27d806d3c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f27f40cbe40 tx=0x7f27f0008030 comp rx=0 tx=0).stop 2023-10-14T10:36:12.573 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.571+0000 7f27fc4f8700 1 -- 172.21.15.67:0/1792481374 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f27f40c6f40 msgr2=0x7f27f4203b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:12.573 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.571+0000 7f27fc4f8700 1 --2- 172.21.15.67:0/1792481374 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f27f40c6f40 0x7f27f4203b70 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f27ec00cc10 tx=0x7f27ec00a060 comp rx=0 tx=0).stop 2023-10-14T10:36:12.574 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.571+0000 7f713c25f700 1 -- 172.21.15.67:0/167551441 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f711806afd0 msgr2=0x7f711806d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:12.574 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.571+0000 7f713c25f700 1 --2- 172.21.15.67:0/167551441 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f711806afd0 0x7f711806d490 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7130009b80 tx=0x7f7130009210 comp rx=0 tx=0).stop 2023-10-14T10:36:12.574 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.571+0000 7f713c25f700 1 -- 172.21.15.67:0/167551441 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7134157250 msgr2=0x7f71340af820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:12.575 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.571+0000 7f713c25f700 1 --2- 172.21.15.67:0/167551441 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7134157250 0x7f71340af820 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f712400b450 tx=0x7f712400b810 comp rx=0 tx=0).stop 2023-10-14T10:36:12.575 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.571+0000 7f27fc4f8700 1 -- 172.21.15.67:0/1792481374 shutdown_connections 2023-10-14T10:36:12.575 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.571+0000 7f27fc4f8700 1 --2- 172.21.15.67:0/1792481374 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f27d806af00 0x7f27d806d3c0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:12.575 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.572+0000 7f27fc4f8700 1 --2- 172.21.15.67:0/1792481374 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f27f40c6f40 0x7f27f4203b70 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:12.576 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.572+0000 7f27fc4f8700 1 --2- 172.21.15.67:0/1792481374 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f27f40c9c50 0x7f27f40c1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:12.576 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.572+0000 7f27fc4f8700 1 --2- 172.21.15.67:0/1792481374 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f27f40ce3a0 0x7f27f40c5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:12.576 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.572+0000 7f27fc4f8700 1 -- 172.21.15.67:0/1792481374 >> 172.21.15.67:0/1792481374 conn(0x7f27f40cf3f0 msgr2=0x7f27f40af8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:12.577 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.572+0000 7f713c25f700 1 -- 172.21.15.67:0/167551441 shutdown_connections 2023-10-14T10:36:12.577 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.572+0000 7f713c25f700 1 --2- 172.21.15.67:0/167551441 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f711806afd0 0x7f711806d490 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:12.577 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.572+0000 7f713c25f700 1 --2- 172.21.15.67:0/167551441 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7134157630 0x7f71341520d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:12.577 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.572+0000 7f713c25f700 1 --2- 172.21.15.67:0/167551441 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f71340c9fc0 0x7f7134153500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:12.578 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.572+0000 7f713c25f700 1 --2- 172.21.15.67:0/167551441 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7134157250 0x7f71340af820 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:12.578 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.572+0000 7f713c25f700 1 -- 172.21.15.67:0/167551441 >> 172.21.15.67:0/167551441 conn(0x7f71340cf3f0 msgr2=0x7f713406e5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:12.578 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.572+0000 7f713c25f700 1 -- 172.21.15.67:0/167551441 shutdown_connections 2023-10-14T10:36:12.578 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.572+0000 7f713c25f700 1 -- 172.21.15.67:0/167551441 wait complete. 2023-10-14T10:36:12.579 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.575+0000 7f27fc4f8700 1 -- 172.21.15.67:0/1792481374 shutdown_connections 2023-10-14T10:36:12.579 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.575+0000 7f27fc4f8700 1 -- 172.21.15.67:0/1792481374 wait complete. 2023-10-14T10:36:12.581 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.580+0000 7fab496fb700 1 -- 172.21.15.67:0/2056394436 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fab4415ebe0 con 0x7fab2c005410 2023-10-14T10:36:12.582 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.581+0000 7fab2b7fe700 1 -- 172.21.15.67:0/2056394436 <== mon.1 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fab4415ebe0 con 0x7fab2c005410 2023-10-14T10:36:12.582 INFO:teuthology.orchestra.run.smithi067.stdout:60129542150 2023-10-14T10:36:12.583 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.3 2023-10-14T10:36:12.584 DEBUG:teuthology.parallel:result is None 2023-10-14T10:36:12.585 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.583+0000 7fab496fb700 1 -- 172.21.15.67:0/2056394436 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fab3006af00 msgr2=0x7fab3006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:12.585 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.583+0000 7fab496fb700 1 --2- 172.21.15.67:0/2056394436 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fab3006af00 0x7fab3006d3c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fab440c9210 tx=0x7fab38008040 comp rx=0 tx=0).stop 2023-10-14T10:36:12.585 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.583+0000 7fab496fb700 1 -- 172.21.15.67:0/2056394436 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fab2c005410 msgr2=0x7fab44152770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:12.585 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.583+0000 7fab496fb700 1 --2- 172.21.15.67:0/2056394436 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fab2c005410 0x7fab44152770 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fab3400c730 tx=0x7fab3400ca40 comp rx=0 tx=0).stop 2023-10-14T10:36:12.586 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.583+0000 7fab496fb700 1 -- 172.21.15.67:0/2056394436 shutdown_connections 2023-10-14T10:36:12.586 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.583+0000 7fab496fb700 1 --2- 172.21.15.67:0/2056394436 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fab3006af00 0x7fab3006d3c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:12.586 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.583+0000 7fab496fb700 1 --2- 172.21.15.67:0/2056394436 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fab44155100 0x7fab440c77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:12.586 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.583+0000 7fab496fb700 1 --2- 172.21.15.67:0/2056394436 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fab2c005410 0x7fab44152770 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:12.587 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.583+0000 7fab496fb700 1 --2- 172.21.15.67:0/2056394436 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab44154430 0x7fab44154bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:12.587 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.583+0000 7fab496fb700 1 -- 172.21.15.67:0/2056394436 >> 172.21.15.67:0/2056394436 conn(0x7fab4406a840 msgr2=0x7fab4406ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:12.587 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.583+0000 7fab496fb700 1 -- 172.21.15.67:0/2056394436 shutdown_connections 2023-10-14T10:36:12.587 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.583+0000 7fab496fb700 1 -- 172.21.15.67:0/2056394436 wait complete. 2023-10-14T10:36:12.588 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.4 2023-10-14T10:36:12.588 DEBUG:teuthology.parallel:result is None 2023-10-14T10:36:12.592 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.7 2023-10-14T10:36:12.592 DEBUG:teuthology.parallel:result is None 2023-10-14T10:36:12.598 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.2 2023-10-14T10:36:12.598 DEBUG:teuthology.parallel:result is None 2023-10-14T10:36:12.598 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-14T10:36:12.599 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T10:36:12.763 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.762+0000 7f6f599fc700 1 Processor -- start 2023-10-14T10:36:12.764 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.762+0000 7f6f599fc700 1 -- start start 2023-10-14T10:36:12.766 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.763+0000 7f6f599fc700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f6f540b7050 0x7f6f5415dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:12.767 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.763+0000 7f6f599fc700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f5415e4e0 con 0x7f6f540d3fc0 2023-10-14T10:36:12.767 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.763+0000 7f6f599fc700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f540d28e0 con 0x7f6f5415dbc0 2023-10-14T10:36:12.767 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.763+0000 7f6f599fc700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f540b7560 con 0x7f6f540b7050 2023-10-14T10:36:12.767 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.763+0000 7f6f527fc700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f6f540d3fc0 0x7f6f540d21d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:52082/0 (socket says 172.21.15.67:52082) 2023-10-14T10:36:12.768 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.763+0000 7f6f527fc700 1 -- 172.21.15.67:0/1646468282 learned_addr learned my addr 172.21.15.67:0/1646468282 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:36:12.768 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.763+0000 7f6f537fe700 1 --2- 172.21.15.67:0/1646468282 >> v2:172.21.15.72:3300/0 conn(0x7f6f540b7050 0x7f6f5415dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:12.768 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.763+0000 7f6f51ffb700 1 -- 172.21.15.67:0/1646468282 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1889621525 0 0) 0x7f6f5415e4e0 con 0x7f6f540d3fc0 2023-10-14T10:36:12.768 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.763+0000 7f6f51ffb700 1 -- 172.21.15.67:0/1646468282 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f3c0035b0 con 0x7f6f540d3fc0 2023-10-14T10:36:12.769 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.763+0000 7f6f51ffb700 1 -- 172.21.15.67:0/1646468282 <== mon.1 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3979569395 0 0) 0x7f6f540d28e0 con 0x7f6f5415dbc0 2023-10-14T10:36:12.769 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.764+0000 7f6f51ffb700 1 -- 172.21.15.67:0/1646468282 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f5415e4e0 con 0x7f6f5415dbc0 2023-10-14T10:36:12.769 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.764+0000 7f6f51ffb700 1 -- 172.21.15.67:0/1646468282 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3187243643 0 0) 0x7f6f3c0035b0 con 0x7f6f540d3fc0 2023-10-14T10:36:12.769 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.764+0000 7f6f537fe700 1 -- 172.21.15.67:0/1646468282 >> v1:172.21.15.72:6789/0 conn(0x7f6f5415dbc0 legacy=0x7f6f540d1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:12.769 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.764+0000 7f6f537fe700 1 -- 172.21.15.67:0/1646468282 >> v1:172.21.15.67:6789/0 conn(0x7f6f540d3fc0 legacy=0x7f6f540d21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:12.770 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.764+0000 7f6f537fe700 1 -- 172.21.15.67:0/1646468282 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f540b7840 con 0x7f6f540b7050 2023-10-14T10:36:12.770 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.764+0000 7f6f537fe700 1 --2- 172.21.15.67:0/1646468282 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6f540b7050 0x7f6f5415dfa0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c009d90 tx=0x7f6f4c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f1774631d570138 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:12.770 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.764+0000 7f6f51ffb700 1 -- 172.21.15.67:0/1646468282 <== mon.1 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f4c00e070 con 0x7f6f540b7050 2023-10-14T10:36:12.770 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.764+0000 7f6f51ffb700 1 -- 172.21.15.67:0/1646468282 <== mon.1 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f4c003a20 con 0x7f6f540b7050 2023-10-14T10:36:12.771 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.764+0000 7f6f51ffb700 1 -- 172.21.15.67:0/1646468282 <== mon.1 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f44001670 con 0x7f6f540b7050 2023-10-14T10:36:12.771 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.764+0000 7f6f599fc700 1 -- 172.21.15.67:0/1646468282 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6f540b7050 msgr2=0x7f6f5415dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:12.771 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.764+0000 7f6f599fc700 1 --2- 172.21.15.67:0/1646468282 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6f540b7050 0x7f6f5415dfa0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c009d90 tx=0x7f6f4c00b330 comp rx=0 tx=0).stop 2023-10-14T10:36:12.771 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.765+0000 7f6f599fc700 1 -- 172.21.15.67:0/1646468282 shutdown_connections 2023-10-14T10:36:12.771 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.765+0000 7f6f599fc700 1 --2- 172.21.15.67:0/1646468282 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6f540b7050 0x7f6f5415dfa0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:12.772 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.765+0000 7f6f599fc700 1 -- 172.21.15.67:0/1646468282 >> 172.21.15.67:0/1646468282 conn(0x7f6f5409cb40 msgr2=0x7f6f540d3820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:12.772 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.765+0000 7f6f599fc700 1 -- 172.21.15.67:0/1646468282 shutdown_connections 2023-10-14T10:36:12.772 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.765+0000 7f6f599fc700 1 -- 172.21.15.67:0/1646468282 wait complete. 2023-10-14T10:36:12.772 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.766+0000 7f6f599fc700 1 Processor -- start 2023-10-14T10:36:12.775 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.766+0000 7f6f599fc700 1 -- start start 2023-10-14T10:36:12.775 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.767+0000 7f6f599fc700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f6f540b7050 0x7f6f54157fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:12.776 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.767+0000 7f6f599fc700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f540d3fc0 0x7f6f540ca860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:12.776 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.767+0000 7f6f599fc700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6f5415dbc0 0x7f6f540cada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:12.776 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.767+0000 7f6f599fc700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f54157b10 con 0x7f6f540d3fc0 2023-10-14T10:36:12.777 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.767+0000 7f6f599fc700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f54156f00 con 0x7f6f5415dbc0 2023-10-14T10:36:12.777 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.767+0000 7f6f599fc700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f541571e0 con 0x7f6f540b7050 2023-10-14T10:36:12.777 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.767+0000 7f6f527fc700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f540d3fc0 0x7f6f540ca860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:12.777 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.767+0000 7f6f527fc700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f540d3fc0 0x7f6f540ca860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44322/0 (socket says 172.21.15.67:44322) 2023-10-14T10:36:12.777 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.767+0000 7f6f527fc700 1 -- 172.21.15.67:0/415113285 learned_addr learned my addr 172.21.15.67:0/415113285 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:12.778 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.767+0000 7f6f52ffd700 1 --2- 172.21.15.67:0/415113285 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f6f540b7050 0x7f6f54157fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:12.778 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.767+0000 7f6f537fe700 1 --2- 172.21.15.67:0/415113285 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6f5415dbc0 0x7f6f540cada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:12.778 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.767+0000 7f6f527fc700 1 -- 172.21.15.67:0/415113285 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f6f540b7050 msgr2=0x7f6f54157fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:12.778 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.767+0000 7f6f527fc700 1 --2- 172.21.15.67:0/415113285 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f6f540b7050 0x7f6f54157fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:12.778 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.767+0000 7f6f527fc700 1 -- 172.21.15.67:0/415113285 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6f5415dbc0 msgr2=0x7f6f540cada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:12.779 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.767+0000 7f6f527fc700 1 --2- 172.21.15.67:0/415113285 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6f5415dbc0 0x7f6f540cada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:12.779 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.767+0000 7f6f527fc700 1 -- 172.21.15.67:0/415113285 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f4c009a70 con 0x7f6f540d3fc0 2023-10-14T10:36:12.779 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.767+0000 7f6f527fc700 1 --2- 172.21.15.67:0/415113285 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f540d3fc0 0x7f6f540ca860 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6f4400c750 tx=0x7f6f4400ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:12.779 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.768+0000 7f6f3bfff700 1 -- 172.21.15.67:0/415113285 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f44004d40 con 0x7f6f540d3fc0 2023-10-14T10:36:12.779 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.768+0000 7f6f3bfff700 1 -- 172.21.15.67:0/415113285 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f440070d0 con 0x7f6f540d3fc0 2023-10-14T10:36:12.780 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.768+0000 7f6f599fc700 1 -- 172.21.15.67:0/415113285 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f541574c0 con 0x7f6f540d3fc0 2023-10-14T10:36:12.780 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.768+0000 7f6f3bfff700 1 -- 172.21.15.67:0/415113285 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f44004ea0 con 0x7f6f540d3fc0 2023-10-14T10:36:12.780 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.768+0000 7f6f599fc700 1 -- 172.21.15.67:0/415113285 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f541584e0 con 0x7f6f540d3fc0 2023-10-14T10:36:12.780 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.768+0000 7f6f599fc700 1 -- 172.21.15.67:0/415113285 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f54157c80 con 0x7f6f540d3fc0 2023-10-14T10:36:12.780 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.769+0000 7f6f3bfff700 1 -- 172.21.15.67:0/415113285 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f6f44007440 con 0x7f6f540d3fc0 2023-10-14T10:36:12.781 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.769+0000 7f6f3bfff700 1 --2- 172.21.15.67:0/415113285 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f6f4006af00 0x7f6f4006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:12.781 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.769+0000 7f6f3bfff700 1 -- 172.21.15.67:0/415113285 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6f440894f0 con 0x7f6f540d3fc0 2023-10-14T10:36:12.781 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.770+0000 7f6f52ffd700 1 --2- 172.21.15.67:0/415113285 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f6f4006af00 0x7f6f4006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:12.781 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.770+0000 7f6f52ffd700 1 --2- 172.21.15.67:0/415113285 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f6f4006af00 0x7f6f4006d3c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6f48007fd0 tx=0x7f6f48009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:12.781 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.774+0000 7f6f3bfff700 1 -- 172.21.15.67:0/415113285 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6f44054500 con 0x7f6f540d3fc0 2023-10-14T10:36:12.971 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.969+0000 7f6f599fc700 1 -- 172.21.15.67:0/415113285 --> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6f540b7e00 con 0x7f6f4006af00 2023-10-14T10:36:12.978 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.977+0000 7f6f3bfff700 1 -- 172.21.15.67:0/415113285 <== mgr.4100 v2:172.21.15.72:6832/703488382 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264532 (secure 0 0 0) 0x7f6f48001d20 con 0x7f6f4006af00 2023-10-14T10:36:12.982 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-14T10:36:12.984 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.980+0000 7f6f599fc700 1 -- 172.21.15.67:0/415113285 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f6f4006af00 msgr2=0x7f6f4006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:12.984 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.980+0000 7f6f599fc700 1 --2- 172.21.15.67:0/415113285 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f6f4006af00 0x7f6f4006d3c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6f48007fd0 tx=0x7f6f48009040 comp rx=0 tx=0).stop 2023-10-14T10:36:12.985 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.980+0000 7f6f599fc700 1 -- 172.21.15.67:0/415113285 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f540d3fc0 msgr2=0x7f6f540ca860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:12.985 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.980+0000 7f6f599fc700 1 --2- 172.21.15.67:0/415113285 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f540d3fc0 0x7f6f540ca860 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6f4400c750 tx=0x7f6f4400ca60 comp rx=0 tx=0).stop 2023-10-14T10:36:12.985 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.980+0000 7f6f599fc700 1 -- 172.21.15.67:0/415113285 shutdown_connections 2023-10-14T10:36:12.985 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.980+0000 7f6f599fc700 1 --2- 172.21.15.67:0/415113285 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f6f4006af00 0x7f6f4006d3c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:12.985 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.980+0000 7f6f599fc700 1 --2- 172.21.15.67:0/415113285 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6f5415dbc0 0x7f6f540cada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:12.986 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.980+0000 7f6f599fc700 1 --2- 172.21.15.67:0/415113285 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f6f540b7050 0x7f6f54157fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:12.986 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.980+0000 7f6f599fc700 1 --2- 172.21.15.67:0/415113285 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f540d3fc0 0x7f6f540ca860 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:12.986 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.980+0000 7f6f599fc700 1 -- 172.21.15.67:0/415113285 >> 172.21.15.67:0/415113285 conn(0x7f6f5409cb40 msgr2=0x7f6f5406d0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:12.986 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.980+0000 7f6f599fc700 1 -- 172.21.15.67:0/415113285 shutdown_connections 2023-10-14T10:36:12.986 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:12.980+0000 7f6f599fc700 1 -- 172.21.15.67:0/415113285 wait complete. 2023-10-14T10:36:12.987 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2023-10-14T10:36:12.994 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-10-14T10:36:11.756762+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219164,"kb_used_data":2984,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755556,"statfs":{"total":773094113280,"available":772869689344,"internally_reserved":0,"allocated":3055616,"data_stored":1816468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12742,"internal_metadata":221236794},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002099"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.847283+0000","last_change":"2023-10-14T10:36:03.869543+0000","last_active":"2023-10-14T10:36:04.847283+0000","last_peered":"2023-10-14T10:36:04.847283+0000","last_clean":"2023-10-14T10:36:04.847283+0000","last_became_active":"2023-10-14T10:36:03.868884+0000","last_became_peered":"2023-10-14T10:36:03.868884+0000","last_unstale":"2023-10-14T10:36:04.847283+0000","last_undegraded":"2023-10-14T10:36:04.847283+0000","last_fullsized":"2023-10-14T10:36:04.847283+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:31:25.399614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.841189+0000","last_change":"2023-10-14T10:36:03.867145+0000","last_active":"2023-10-14T10:36:04.841189+0000","last_peered":"2023-10-14T10:36:04.841189+0000","last_clean":"2023-10-14T10:36:04.841189+0000","last_became_active":"2023-10-14T10:36:03.866743+0000","last_became_peered":"2023-10-14T10:36:03.866743+0000","last_unstale":"2023-10-14T10:36:04.841189+0000","last_undegraded":"2023-10-14T10:36:04.841189+0000","last_fullsized":"2023-10-14T10:36:04.841189+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T18:49:18.106622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.483814+0000","last_change":"2023-10-14T10:36:03.861171+0000","last_active":"2023-10-14T10:36:05.483814+0000","last_peered":"2023-10-14T10:36:05.483814+0000","last_clean":"2023-10-14T10:36:05.483814+0000","last_became_active":"2023-10-14T10:36:03.860860+0000","last_became_peered":"2023-10-14T10:36:03.860860+0000","last_unstale":"2023-10-14T10:36:05.483814+0000","last_undegraded":"2023-10-14T10:36:05.483814+0000","last_fullsized":"2023-10-14T10:36:05.483814+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T17:28:51.336361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.845699+0000","last_change":"2023-10-14T10:36:03.861631+0000","last_active":"2023-10-14T10:36:04.845699+0000","last_peered":"2023-10-14T10:36:04.845699+0000","last_clean":"2023-10-14T10:36:04.845699+0000","last_became_active":"2023-10-14T10:36:03.861291+0000","last_became_peered":"2023-10-14T10:36:03.861291+0000","last_unstale":"2023-10-14T10:36:04.845699+0000","last_undegraded":"2023-10-14T10:36:04.845699+0000","last_fullsized":"2023-10-14T10:36:04.845699+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T17:52:45.149486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.846931+0000","last_change":"2023-10-14T10:36:03.870469+0000","last_active":"2023-10-14T10:36:04.846931+0000","last_peered":"2023-10-14T10:36:04.846931+0000","last_clean":"2023-10-14T10:36:04.846931+0000","last_became_active":"2023-10-14T10:36:03.865743+0000","last_became_peered":"2023-10-14T10:36:03.865743+0000","last_unstale":"2023-10-14T10:36:04.846931+0000","last_undegraded":"2023-10-14T10:36:04.846931+0000","last_fullsized":"2023-10-14T10:36:04.846931+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:33:09.576282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.845553+0000","last_change":"2023-10-14T10:36:03.862386+0000","last_active":"2023-10-14T10:36:04.845553+0000","last_peered":"2023-10-14T10:36:04.845553+0000","last_clean":"2023-10-14T10:36:04.845553+0000","last_became_active":"2023-10-14T10:36:03.861705+0000","last_became_peered":"2023-10-14T10:36:03.861705+0000","last_unstale":"2023-10-14T10:36:04.845553+0000","last_undegraded":"2023-10-14T10:36:04.845553+0000","last_fullsized":"2023-10-14T10:36:04.845553+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:47:42.089047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.846829+0000","last_change":"2023-10-14T10:36:03.865502+0000","last_active":"2023-10-14T10:36:04.846829+0000","last_peered":"2023-10-14T10:36:04.846829+0000","last_clean":"2023-10-14T10:36:04.846829+0000","last_became_active":"2023-10-14T10:36:03.865238+0000","last_became_peered":"2023-10-14T10:36:03.865238+0000","last_unstale":"2023-10-14T10:36:04.846829+0000","last_undegraded":"2023-10-14T10:36:04.846829+0000","last_fullsized":"2023-10-14T10:36:04.846829+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T20:52:07.681036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.482589+0000","last_change":"2023-10-14T10:36:03.859238+0000","last_active":"2023-10-14T10:36:05.482589+0000","last_peered":"2023-10-14T10:36:05.482589+0000","last_clean":"2023-10-14T10:36:05.482589+0000","last_became_active":"2023-10-14T10:36:03.858981+0000","last_became_peered":"2023-10-14T10:36:03.858981+0000","last_unstale":"2023-10-14T10:36:05.482589+0000","last_undegraded":"2023-10-14T10:36:05.482589+0000","last_fullsized":"2023-10-14T10:36:05.482589+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T12:18:09.680616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.847259+0000","last_change":"2023-10-14T10:36:03.868883+0000","last_active":"2023-10-14T10:36:04.847259+0000","last_peered":"2023-10-14T10:36:04.847259+0000","last_clean":"2023-10-14T10:36:04.847259+0000","last_became_active":"2023-10-14T10:36:03.868019+0000","last_became_peered":"2023-10-14T10:36:03.868019+0000","last_unstale":"2023-10-14T10:36:04.847259+0000","last_undegraded":"2023-10-14T10:36:04.847259+0000","last_fullsized":"2023-10-14T10:36:04.847259+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T17:08:17.694068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.482896+0000","last_change":"2023-10-14T10:36:03.861523+0000","last_active":"2023-10-14T10:36:05.482896+0000","last_peered":"2023-10-14T10:36:05.482896+0000","last_clean":"2023-10-14T10:36:05.482896+0000","last_became_active":"2023-10-14T10:36:03.860708+0000","last_became_peered":"2023-10-14T10:36:03.860708+0000","last_unstale":"2023-10-14T10:36:05.482896+0000","last_undegraded":"2023-10-14T10:36:05.482896+0000","last_fullsized":"2023-10-14T10:36:05.482896+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:07:32.070142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.482672+0000","last_change":"2023-10-14T10:36:03.861278+0000","last_active":"2023-10-14T10:36:05.482672+0000","last_peered":"2023-10-14T10:36:05.482672+0000","last_clean":"2023-10-14T10:36:05.482672+0000","last_became_active":"2023-10-14T10:36:03.860994+0000","last_became_peered":"2023-10-14T10:36:03.860994+0000","last_unstale":"2023-10-14T10:36:05.482672+0000","last_undegraded":"2023-10-14T10:36:05.482672+0000","last_fullsized":"2023-10-14T10:36:05.482672+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T12:18:09.680616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.482785+0000","last_change":"2023-10-14T10:36:03.871181+0000","last_active":"2023-10-14T10:36:05.482785+0000","last_peered":"2023-10-14T10:36:05.482785+0000","last_clean":"2023-10-14T10:36:05.482785+0000","last_became_active":"2023-10-14T10:36:03.870875+0000","last_became_peered":"2023-10-14T10:36:03.870875+0000","last_unstale":"2023-10-14T10:36:05.482785+0000","last_undegraded":"2023-10-14T10:36:05.482785+0000","last_fullsized":"2023-10-14T10:36:05.482785+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:50:35.924528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.482529+0000","last_change":"2023-10-14T10:36:03.857251+0000","last_active":"2023-10-14T10:36:05.482529+0000","last_peered":"2023-10-14T10:36:05.482529+0000","last_clean":"2023-10-14T10:36:05.482529+0000","last_became_active":"2023-10-14T10:36:03.856908+0000","last_became_peered":"2023-10-14T10:36:03.856908+0000","last_unstale":"2023-10-14T10:36:05.482529+0000","last_undegraded":"2023-10-14T10:36:05.482529+0000","last_fullsized":"2023-10-14T10:36:05.482529+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T18:50:26.625744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.484165+0000","last_change":"2023-10-14T10:36:03.858760+0000","last_active":"2023-10-14T10:36:05.484165+0000","last_peered":"2023-10-14T10:36:05.484165+0000","last_clean":"2023-10-14T10:36:05.484165+0000","last_became_active":"2023-10-14T10:36:03.854779+0000","last_became_peered":"2023-10-14T10:36:03.854779+0000","last_unstale":"2023-10-14T10:36:05.484165+0000","last_undegraded":"2023-10-14T10:36:05.484165+0000","last_fullsized":"2023-10-14T10:36:05.484165+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T18:49:18.106622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.840868+0000","last_change":"2023-10-14T10:36:03.861476+0000","last_active":"2023-10-14T10:36:04.840868+0000","last_peered":"2023-10-14T10:36:04.840868+0000","last_clean":"2023-10-14T10:36:04.840868+0000","last_became_active":"2023-10-14T10:36:03.859641+0000","last_became_peered":"2023-10-14T10:36:03.859641+0000","last_unstale":"2023-10-14T10:36:04.840868+0000","last_undegraded":"2023-10-14T10:36:04.840868+0000","last_fullsized":"2023-10-14T10:36:04.840868+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T21:40:43.364841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.845130+0000","last_change":"2023-10-14T10:36:03.858710+0000","last_active":"2023-10-14T10:36:04.845130+0000","last_peered":"2023-10-14T10:36:04.845130+0000","last_clean":"2023-10-14T10:36:04.845130+0000","last_became_active":"2023-10-14T10:36:03.858456+0000","last_became_peered":"2023-10-14T10:36:03.858456+0000","last_unstale":"2023-10-14T10:36:04.845130+0000","last_undegraded":"2023-10-14T10:36:04.845130+0000","last_fullsized":"2023-10-14T10:36:04.845130+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:07:32.070142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.845836+0000","last_change":"2023-10-14T10:36:03.867067+0000","last_active":"2023-10-14T10:36:04.845836+0000","last_peered":"2023-10-14T10:36:04.845836+0000","last_clean":"2023-10-14T10:36:04.845836+0000","last_became_active":"2023-10-14T10:36:03.866802+0000","last_became_peered":"2023-10-14T10:36:03.866802+0000","last_unstale":"2023-10-14T10:36:04.845836+0000","last_undegraded":"2023-10-14T10:36:04.845836+0000","last_fullsized":"2023-10-14T10:36:04.845836+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:09:22.818455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.482788+0000","last_change":"2023-10-14T10:36:03.869409+0000","last_active":"2023-10-14T10:36:05.482788+0000","last_peered":"2023-10-14T10:36:05.482788+0000","last_clean":"2023-10-14T10:36:05.482788+0000","last_became_active":"2023-10-14T10:36:03.868961+0000","last_became_peered":"2023-10-14T10:36:03.868961+0000","last_unstale":"2023-10-14T10:36:05.482788+0000","last_undegraded":"2023-10-14T10:36:05.482788+0000","last_fullsized":"2023-10-14T10:36:05.482788+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:10:10.517040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.847011+0000","last_change":"2023-10-14T10:36:03.864794+0000","last_active":"2023-10-14T10:36:04.847011+0000","last_peered":"2023-10-14T10:36:04.847011+0000","last_clean":"2023-10-14T10:36:04.847011+0000","last_became_active":"2023-10-14T10:36:03.864195+0000","last_became_peered":"2023-10-14T10:36:03.864195+0000","last_unstale":"2023-10-14T10:36:04.847011+0000","last_undegraded":"2023-10-14T10:36:04.847011+0000","last_fullsized":"2023-10-14T10:36:04.847011+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T17:52:45.149486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.483618+0000","last_change":"2023-10-14T10:36:02.855169+0000","last_active":"2023-10-14T10:36:05.483618+0000","last_peered":"2023-10-14T10:36:05.483618+0000","last_clean":"2023-10-14T10:36:05.483618+0000","last_became_active":"2023-10-14T10:36:02.854817+0000","last_became_peered":"2023-10-14T10:36:02.854817+0000","last_unstale":"2023-10-14T10:36:05.483618+0000","last_undegraded":"2023-10-14T10:36:05.483618+0000","last_fullsized":"2023-10-14T10:36:05.483618+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T14:12:52.738066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.484369+0000","last_change":"2023-10-14T10:36:03.865118+0000","last_active":"2023-10-14T10:36:05.484369+0000","last_peered":"2023-10-14T10:36:05.484369+0000","last_clean":"2023-10-14T10:36:05.484369+0000","last_became_active":"2023-10-14T10:36:03.864579+0000","last_became_peered":"2023-10-14T10:36:03.864579+0000","last_unstale":"2023-10-14T10:36:05.484369+0000","last_undegraded":"2023-10-14T10:36:05.484369+0000","last_fullsized":"2023-10-14T10:36:05.484369+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:27:25.021169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.484313+0000","last_change":"2023-10-14T10:36:02.851587+0000","last_active":"2023-10-14T10:36:05.484313+0000","last_peered":"2023-10-14T10:36:05.484313+0000","last_clean":"2023-10-14T10:36:05.484313+0000","last_became_active":"2023-10-14T10:36:02.851276+0000","last_became_peered":"2023-10-14T10:36:02.851276+0000","last_unstale":"2023-10-14T10:36:05.484313+0000","last_undegraded":"2023-10-14T10:36:05.484313+0000","last_fullsized":"2023-10-14T10:36:05.484313+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T14:04:44.127519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.483564+0000","last_change":"2023-10-14T10:36:03.861320+0000","last_active":"2023-10-14T10:36:05.483564+0000","last_peered":"2023-10-14T10:36:05.483564+0000","last_clean":"2023-10-14T10:36:05.483564+0000","last_became_active":"2023-10-14T10:36:03.860684+0000","last_became_peered":"2023-10-14T10:36:03.860684+0000","last_unstale":"2023-10-14T10:36:05.483564+0000","last_undegraded":"2023-10-14T10:36:05.483564+0000","last_fullsized":"2023-10-14T10:36:05.483564+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:22:58.149187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.847230+0000","last_change":"2023-10-14T10:36:02.854569+0000","last_active":"2023-10-14T10:36:04.847230+0000","last_peered":"2023-10-14T10:36:04.847230+0000","last_clean":"2023-10-14T10:36:04.847230+0000","last_became_active":"2023-10-14T10:36:02.853518+0000","last_became_peered":"2023-10-14T10:36:02.853518+0000","last_unstale":"2023-10-14T10:36:04.847230+0000","last_undegraded":"2023-10-14T10:36:04.847230+0000","last_fullsized":"2023-10-14T10:36:04.847230+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:27:24.015552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.845887+0000","last_change":"2023-10-14T10:36:03.872077+0000","last_active":"2023-10-14T10:36:04.845887+0000","last_peered":"2023-10-14T10:36:04.845887+0000","last_clean":"2023-10-14T10:36:04.845887+0000","last_became_active":"2023-10-14T10:36:03.871740+0000","last_became_peered":"2023-10-14T10:36:03.871740+0000","last_unstale":"2023-10-14T10:36:04.845887+0000","last_undegraded":"2023-10-14T10:36:04.845887+0000","last_fullsized":"2023-10-14T10:36:04.845887+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T12:31:06.179885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.846883+0000","last_change":"2023-10-14T10:36:02.853780+0000","last_active":"2023-10-14T10:36:04.846883+0000","last_peered":"2023-10-14T10:36:04.846883+0000","last_clean":"2023-10-14T10:36:04.846883+0000","last_became_active":"2023-10-14T10:36:02.853450+0000","last_became_peered":"2023-10-14T10:36:02.853450+0000","last_unstale":"2023-10-14T10:36:04.846883+0000","last_undegraded":"2023-10-14T10:36:04.846883+0000","last_fullsized":"2023-10-14T10:36:04.846883+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T10:42:23.021229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.845567+0000","last_change":"2023-10-14T10:36:03.858379+0000","last_active":"2023-10-14T10:36:04.845567+0000","last_peered":"2023-10-14T10:36:04.845567+0000","last_clean":"2023-10-14T10:36:04.845567+0000","last_became_active":"2023-10-14T10:36:03.857904+0000","last_became_peered":"2023-10-14T10:36:03.857904+0000","last_unstale":"2023-10-14T10:36:04.845567+0000","last_undegraded":"2023-10-14T10:36:04.845567+0000","last_fullsized":"2023-10-14T10:36:04.845567+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:33:09.576282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.482737+0000","last_change":"2023-10-14T10:36:02.854396+0000","last_active":"2023-10-14T10:36:05.482737+0000","last_peered":"2023-10-14T10:36:05.482737+0000","last_clean":"2023-10-14T10:36:05.482737+0000","last_became_active":"2023-10-14T10:36:02.854085+0000","last_became_peered":"2023-10-14T10:36:02.854085+0000","last_unstale":"2023-10-14T10:36:05.482737+0000","last_undegraded":"2023-10-14T10:36:05.482737+0000","last_fullsized":"2023-10-14T10:36:05.482737+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T14:12:52.738066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.483060+0000","last_change":"2023-10-14T10:36:03.855010+0000","last_active":"2023-10-14T10:36:05.483060+0000","last_peered":"2023-10-14T10:36:05.483060+0000","last_clean":"2023-10-14T10:36:05.483060+0000","last_became_active":"2023-10-14T10:36:03.854714+0000","last_became_peered":"2023-10-14T10:36:03.854714+0000","last_unstale":"2023-10-14T10:36:05.483060+0000","last_undegraded":"2023-10-14T10:36:05.483060+0000","last_fullsized":"2023-10-14T10:36:05.483060+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:29:51.596547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-14T10:36:03.865241+0000","last_change":"2023-10-14T10:36:02.847573+0000","last_active":"2023-10-14T10:36:03.865241+0000","last_peered":"2023-10-14T10:36:03.865241+0000","last_clean":"2023-10-14T10:36:03.865241+0000","last_became_active":"2023-10-14T10:36:02.847061+0000","last_became_peered":"2023-10-14T10:36:02.847061+0000","last_unstale":"2023-10-14T10:36:03.865241+0000","last_undegraded":"2023-10-14T10:36:03.865241+0000","last_fullsized":"2023-10-14T10:36:03.865241+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:50:34.918912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.484012+0000","last_change":"2023-10-14T10:36:03.866134+0000","last_active":"2023-10-14T10:36:05.484012+0000","last_peered":"2023-10-14T10:36:05.484012+0000","last_clean":"2023-10-14T10:36:05.484012+0000","last_became_active":"2023-10-14T10:36:03.865614+0000","last_became_peered":"2023-10-14T10:36:03.865614+0000","last_unstale":"2023-10-14T10:36:05.484012+0000","last_undegraded":"2023-10-14T10:36:05.484012+0000","last_fullsized":"2023-10-14T10:36:05.484012+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T18:42:51.749474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.482644+0000","last_change":"2023-10-14T10:36:02.846515+0000","last_active":"2023-10-14T10:36:05.482644+0000","last_peered":"2023-10-14T10:36:05.482644+0000","last_clean":"2023-10-14T10:36:05.482644+0000","last_became_active":"2023-10-14T10:36:02.845841+0000","last_became_peered":"2023-10-14T10:36:02.845841+0000","last_unstale":"2023-10-14T10:36:05.482644+0000","last_undegraded":"2023-10-14T10:36:05.482644+0000","last_fullsized":"2023-10-14T10:36:05.482644+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:22:57.143570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.483031+0000","last_change":"2023-10-14T10:36:03.865280+0000","last_active":"2023-10-14T10:36:05.483031+0000","last_peered":"2023-10-14T10:36:05.483031+0000","last_clean":"2023-10-14T10:36:05.483031+0000","last_became_active":"2023-10-14T10:36:03.864926+0000","last_became_peered":"2023-10-14T10:36:03.864926+0000","last_unstale":"2023-10-14T10:36:05.483031+0000","last_undegraded":"2023-10-14T10:36:05.483031+0000","last_fullsized":"2023-10-14T10:36:05.483031+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:05:52.086793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.845177+0000","last_change":"2023-10-14T10:36:02.843680+0000","last_active":"2023-10-14T10:36:04.845177+0000","last_peered":"2023-10-14T10:36:04.845177+0000","last_clean":"2023-10-14T10:36:04.845177+0000","last_became_active":"2023-10-14T10:36:02.843364+0000","last_became_peered":"2023-10-14T10:36:02.843364+0000","last_unstale":"2023-10-14T10:36:04.845177+0000","last_undegraded":"2023-10-14T10:36:04.845177+0000","last_fullsized":"2023-10-14T10:36:04.845177+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T10:59:16.502697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.841021+0000","last_change":"2023-10-14T10:36:03.862822+0000","last_active":"2023-10-14T10:36:04.841021+0000","last_peered":"2023-10-14T10:36:04.841021+0000","last_clean":"2023-10-14T10:36:04.841021+0000","last_became_active":"2023-10-14T10:36:03.862547+0000","last_became_peered":"2023-10-14T10:36:03.862547+0000","last_unstale":"2023-10-14T10:36:04.841021+0000","last_undegraded":"2023-10-14T10:36:04.841021+0000","last_fullsized":"2023-10-14T10:36:04.841021+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T18:42:51.749474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-14T10:36:03.851585+0000","last_change":"2023-10-14T10:36:02.846160+0000","last_active":"2023-10-14T10:36:03.851585+0000","last_peered":"2023-10-14T10:36:03.851585+0000","last_clean":"2023-10-14T10:36:03.851585+0000","last_became_active":"2023-10-14T10:36:02.845784+0000","last_became_peered":"2023-10-14T10:36:02.845784+0000","last_unstale":"2023-10-14T10:36:03.851585+0000","last_undegraded":"2023-10-14T10:36:03.851585+0000","last_fullsized":"2023-10-14T10:36:03.851585+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T12:29:01.996039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.845511+0000","last_change":"2023-10-14T10:36:03.864605+0000","last_active":"2023-10-14T10:36:04.845511+0000","last_peered":"2023-10-14T10:36:04.845511+0000","last_clean":"2023-10-14T10:36:04.845511+0000","last_became_active":"2023-10-14T10:36:03.864160+0000","last_became_peered":"2023-10-14T10:36:03.864160+0000","last_unstale":"2023-10-14T10:36:04.845511+0000","last_undegraded":"2023-10-14T10:36:04.845511+0000","last_fullsized":"2023-10-14T10:36:04.845511+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T20:52:07.681036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.845294+0000","last_change":"2023-10-14T10:36:02.845777+0000","last_active":"2023-10-14T10:36:04.845294+0000","last_peered":"2023-10-14T10:36:04.845294+0000","last_clean":"2023-10-14T10:36:04.845294+0000","last_became_active":"2023-10-14T10:36:02.845482+0000","last_became_peered":"2023-10-14T10:36:02.845482+0000","last_unstale":"2023-10-14T10:36:04.845294+0000","last_undegraded":"2023-10-14T10:36:04.845294+0000","last_fullsized":"2023-10-14T10:36:04.845294+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T14:04:44.127519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.483717+0000","last_change":"2023-10-14T10:36:03.861072+0000","last_active":"2023-10-14T10:36:05.483717+0000","last_peered":"2023-10-14T10:36:05.483717+0000","last_clean":"2023-10-14T10:36:05.483717+0000","last_became_active":"2023-10-14T10:36:03.860335+0000","last_became_peered":"2023-10-14T10:36:03.860335+0000","last_unstale":"2023-10-14T10:36:05.483717+0000","last_undegraded":"2023-10-14T10:36:05.483717+0000","last_fullsized":"2023-10-14T10:36:05.483717+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T12:18:09.680616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.841103+0000","last_change":"2023-10-14T10:36:02.845951+0000","last_active":"2023-10-14T10:36:04.841103+0000","last_peered":"2023-10-14T10:36:04.841103+0000","last_clean":"2023-10-14T10:36:04.841103+0000","last_became_active":"2023-10-14T10:36:02.845667+0000","last_became_peered":"2023-10-14T10:36:02.845667+0000","last_unstale":"2023-10-14T10:36:04.841103+0000","last_undegraded":"2023-10-14T10:36:04.841103+0000","last_fullsized":"2023-10-14T10:36:04.841103+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:00:17.008189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.482746+0000","last_change":"2023-10-14T10:36:03.871035+0000","last_active":"2023-10-14T10:36:05.482746+0000","last_peered":"2023-10-14T10:36:05.482746+0000","last_clean":"2023-10-14T10:36:05.482746+0000","last_became_active":"2023-10-14T10:36:03.870542+0000","last_became_peered":"2023-10-14T10:36:03.870542+0000","last_unstale":"2023-10-14T10:36:05.482746+0000","last_undegraded":"2023-10-14T10:36:05.482746+0000","last_fullsized":"2023-10-14T10:36:05.482746+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T14:05:05.847145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-14T10:36:03.874236+0000","last_change":"2023-10-14T10:36:02.847268+0000","last_active":"2023-10-14T10:36:03.874236+0000","last_peered":"2023-10-14T10:36:03.874236+0000","last_clean":"2023-10-14T10:36:03.874236+0000","last_became_active":"2023-10-14T10:36:02.846840+0000","last_became_peered":"2023-10-14T10:36:02.846840+0000","last_unstale":"2023-10-14T10:36:03.874236+0000","last_undegraded":"2023-10-14T10:36:03.874236+0000","last_fullsized":"2023-10-14T10:36:03.874236+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:22:57.143570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.847057+0000","last_change":"2023-10-14T10:36:03.864955+0000","last_active":"2023-10-14T10:36:04.847057+0000","last_peered":"2023-10-14T10:36:04.847057+0000","last_clean":"2023-10-14T10:36:04.847057+0000","last_became_active":"2023-10-14T10:36:03.864505+0000","last_became_peered":"2023-10-14T10:36:03.864505+0000","last_unstale":"2023-10-14T10:36:04.847057+0000","last_undegraded":"2023-10-14T10:36:04.847057+0000","last_fullsized":"2023-10-14T10:36:04.847057+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:00:36.181878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.483879+0000","last_change":"2023-10-14T10:36:02.856271+0000","last_active":"2023-10-14T10:36:05.483879+0000","last_peered":"2023-10-14T10:36:05.483879+0000","last_clean":"2023-10-14T10:36:05.483879+0000","last_became_active":"2023-10-14T10:36:02.856039+0000","last_became_peered":"2023-10-14T10:36:02.856039+0000","last_unstale":"2023-10-14T10:36:05.483879+0000","last_undegraded":"2023-10-14T10:36:05.483879+0000","last_fullsized":"2023-10-14T10:36:05.483879+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:29:50.590931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.484124+0000","last_change":"2023-10-14T10:36:03.868295+0000","last_active":"2023-10-14T10:36:05.484124+0000","last_peered":"2023-10-14T10:36:05.484124+0000","last_clean":"2023-10-14T10:36:05.484124+0000","last_became_active":"2023-10-14T10:36:03.867969+0000","last_became_peered":"2023-10-14T10:36:03.867969+0000","last_unstale":"2023-10-14T10:36:05.484124+0000","last_undegraded":"2023-10-14T10:36:05.484124+0000","last_fullsized":"2023-10-14T10:36:05.484124+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:05:52.086793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.845748+0000","last_change":"2023-10-14T10:36:02.845900+0000","last_active":"2023-10-14T10:36:04.845748+0000","last_peered":"2023-10-14T10:36:04.845748+0000","last_clean":"2023-10-14T10:36:04.845748+0000","last_became_active":"2023-10-14T10:36:02.845596+0000","last_became_peered":"2023-10-14T10:36:02.845596+0000","last_unstale":"2023-10-14T10:36:04.845748+0000","last_undegraded":"2023-10-14T10:36:04.845748+0000","last_fullsized":"2023-10-14T10:36:04.845748+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T21:40:42.359224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.846941+0000","last_change":"2023-10-14T10:36:03.870846+0000","last_active":"2023-10-14T10:36:04.846941+0000","last_peered":"2023-10-14T10:36:04.846941+0000","last_clean":"2023-10-14T10:36:04.846941+0000","last_became_active":"2023-10-14T10:36:03.869379+0000","last_became_peered":"2023-10-14T10:36:03.869379+0000","last_unstale":"2023-10-14T10:36:04.846941+0000","last_undegraded":"2023-10-14T10:36:04.846941+0000","last_fullsized":"2023-10-14T10:36:04.846941+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T10:36:37.449003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-14T10:36:03.851688+0000","last_change":"2023-10-14T10:36:02.847080+0000","last_active":"2023-10-14T10:36:03.851688+0000","last_peered":"2023-10-14T10:36:03.851688+0000","last_clean":"2023-10-14T10:36:03.851688+0000","last_became_active":"2023-10-14T10:36:02.846784+0000","last_became_peered":"2023-10-14T10:36:02.846784+0000","last_unstale":"2023-10-14T10:36:03.851688+0000","last_undegraded":"2023-10-14T10:36:03.851688+0000","last_fullsized":"2023-10-14T10:36:03.851688+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:29:50.590931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.482758+0000","last_change":"2023-10-14T10:36:03.862316+0000","last_active":"2023-10-14T10:36:05.482758+0000","last_peered":"2023-10-14T10:36:05.482758+0000","last_clean":"2023-10-14T10:36:05.482758+0000","last_became_active":"2023-10-14T10:36:03.862022+0000","last_became_peered":"2023-10-14T10:36:03.862022+0000","last_unstale":"2023-10-14T10:36:05.482758+0000","last_undegraded":"2023-10-14T10:36:05.482758+0000","last_fullsized":"2023-10-14T10:36:05.482758+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T21:40:43.364841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-14T10:36:03.867586+0000","last_change":"2023-10-14T10:36:02.846811+0000","last_active":"2023-10-14T10:36:03.867586+0000","last_peered":"2023-10-14T10:36:03.867586+0000","last_clean":"2023-10-14T10:36:03.867586+0000","last_became_active":"2023-10-14T10:36:02.846487+0000","last_became_peered":"2023-10-14T10:36:02.846487+0000","last_unstale":"2023-10-14T10:36:03.867586+0000","last_undegraded":"2023-10-14T10:36:03.867586+0000","last_fullsized":"2023-10-14T10:36:03.867586+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T18:47:26.337837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.846699+0000","last_change":"2023-10-14T10:36:03.872480+0000","last_active":"2023-10-14T10:36:04.846699+0000","last_peered":"2023-10-14T10:36:04.846699+0000","last_clean":"2023-10-14T10:36:04.846699+0000","last_became_active":"2023-10-14T10:36:03.872231+0000","last_became_peered":"2023-10-14T10:36:03.872231+0000","last_unstale":"2023-10-14T10:36:04.846699+0000","last_undegraded":"2023-10-14T10:36:04.846699+0000","last_fullsized":"2023-10-14T10:36:04.846699+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:47:42.089047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.482611+0000","last_change":"2023-10-14T10:36:02.851223+0000","last_active":"2023-10-14T10:36:05.482611+0000","last_peered":"2023-10-14T10:36:05.482611+0000","last_clean":"2023-10-14T10:36:05.482611+0000","last_became_active":"2023-10-14T10:36:02.850955+0000","last_became_peered":"2023-10-14T10:36:02.850955+0000","last_unstale":"2023-10-14T10:36:05.482611+0000","last_undegraded":"2023-10-14T10:36:05.482611+0000","last_fullsized":"2023-10-14T10:36:05.482611+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:08:47.187941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.847107+0000","last_change":"2023-10-14T10:36:03.871637+0000","last_active":"2023-10-14T10:36:04.847107+0000","last_peered":"2023-10-14T10:36:04.847107+0000","last_clean":"2023-10-14T10:36:04.847107+0000","last_became_active":"2023-10-14T10:36:03.871362+0000","last_became_peered":"2023-10-14T10:36:03.871362+0000","last_unstale":"2023-10-14T10:36:04.847107+0000","last_undegraded":"2023-10-14T10:36:04.847107+0000","last_fullsized":"2023-10-14T10:36:04.847107+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:09:22.818455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.841156+0000","last_change":"2023-10-14T10:36:02.851439+0000","last_active":"2023-10-14T10:36:04.841156+0000","last_peered":"2023-10-14T10:36:04.841156+0000","last_clean":"2023-10-14T10:36:04.841156+0000","last_became_active":"2023-10-14T10:36:02.851121+0000","last_became_peered":"2023-10-14T10:36:02.851121+0000","last_unstale":"2023-10-14T10:36:04.841156+0000","last_undegraded":"2023-10-14T10:36:04.841156+0000","last_fullsized":"2023-10-14T10:36:04.841156+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T10:42:23.021229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.845753+0000","last_change":"2023-10-14T10:36:03.863345+0000","last_active":"2023-10-14T10:36:04.845753+0000","last_peered":"2023-10-14T10:36:04.845753+0000","last_clean":"2023-10-14T10:36:04.845753+0000","last_became_active":"2023-10-14T10:36:03.861227+0000","last_became_peered":"2023-10-14T10:36:03.861227+0000","last_unstale":"2023-10-14T10:36:04.845753+0000","last_undegraded":"2023-10-14T10:36:04.845753+0000","last_fullsized":"2023-10-14T10:36:04.845753+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:00:36.181878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.845649+0000","last_change":"2023-10-14T10:36:02.846354+0000","last_active":"2023-10-14T10:36:04.845649+0000","last_peered":"2023-10-14T10:36:04.845649+0000","last_clean":"2023-10-14T10:36:04.845649+0000","last_became_active":"2023-10-14T10:36:02.845950+0000","last_became_peered":"2023-10-14T10:36:02.845950+0000","last_unstale":"2023-10-14T10:36:04.845649+0000","last_undegraded":"2023-10-14T10:36:04.845649+0000","last_fullsized":"2023-10-14T10:36:04.845649+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T17:28:50.330745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.482785+0000","last_change":"2023-10-14T10:36:03.860805+0000","last_active":"2023-10-14T10:36:05.482785+0000","last_peered":"2023-10-14T10:36:05.482785+0000","last_clean":"2023-10-14T10:36:05.482785+0000","last_became_active":"2023-10-14T10:36:03.860218+0000","last_became_peered":"2023-10-14T10:36:03.860218+0000","last_unstale":"2023-10-14T10:36:05.482785+0000","last_undegraded":"2023-10-14T10:36:05.482785+0000","last_fullsized":"2023-10-14T10:36:05.482785+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T12:29:03.001656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.840832+0000","last_change":"2023-10-14T10:36:03.857512+0000","last_active":"2023-10-14T10:36:04.840832+0000","last_peered":"2023-10-14T10:36:04.840832+0000","last_clean":"2023-10-14T10:36:04.840832+0000","last_became_active":"2023-10-14T10:36:03.855687+0000","last_became_peered":"2023-10-14T10:36:03.855687+0000","last_unstale":"2023-10-14T10:36:04.840832+0000","last_undegraded":"2023-10-14T10:36:04.840832+0000","last_fullsized":"2023-10-14T10:36:04.840832+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T17:28:51.336361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-14T10:36:03.843979+0000","last_change":"2023-10-14T10:36:02.843620+0000","last_active":"2023-10-14T10:36:03.843979+0000","last_peered":"2023-10-14T10:36:03.843979+0000","last_clean":"2023-10-14T10:36:03.843979+0000","last_became_active":"2023-10-14T10:36:02.843324+0000","last_became_peered":"2023-10-14T10:36:02.843324+0000","last_unstale":"2023-10-14T10:36:03.843979+0000","last_undegraded":"2023-10-14T10:36:03.843979+0000","last_fullsized":"2023-10-14T10:36:03.843979+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T14:12:49.629312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.482850+0000","last_change":"2023-10-14T10:36:03.858691+0000","last_active":"2023-10-14T10:36:05.482850+0000","last_peered":"2023-10-14T10:36:05.482850+0000","last_clean":"2023-10-14T10:36:05.482850+0000","last_became_active":"2023-10-14T10:36:03.858420+0000","last_became_peered":"2023-10-14T10:36:03.858420+0000","last_unstale":"2023-10-14T10:36:05.482850+0000","last_undegraded":"2023-10-14T10:36:05.482850+0000","last_fullsized":"2023-10-14T10:36:05.482850+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T18:42:51.749474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.845260+0000","last_change":"2023-10-14T10:36:02.840533+0000","last_active":"2023-10-14T10:36:04.845260+0000","last_peered":"2023-10-14T10:36:04.845260+0000","last_clean":"2023-10-14T10:36:04.845260+0000","last_became_active":"2023-10-14T10:36:02.839919+0000","last_became_peered":"2023-10-14T10:36:02.839919+0000","last_unstale":"2023-10-14T10:36:04.845260+0000","last_undegraded":"2023-10-14T10:36:04.845260+0000","last_fullsized":"2023-10-14T10:36:04.845260+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:29:50.590931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":165,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-14T10:36:03.847417+0000","last_change":"2023-10-14T10:35:59.770163+0000","last_active":"2023-10-14T10:36:03.847417+0000","last_peered":"2023-10-14T10:36:03.847417+0000","last_clean":"2023-10-14T10:36:03.847417+0000","last_became_active":"2023-10-14T10:35:59.769199+0000","last_became_peered":"2023-10-14T10:35:59.769199+0000","last_unstale":"2023-10-14T10:36:03.847417+0000","last_undegraded":"2023-10-14T10:36:03.847417+0000","last_fullsized":"2023-10-14T10:36:03.847417+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:35:58.675572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:35:58.675572+0000","last_clean_scrub_stamp":"2023-10-14T10:35:58.675572+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:03:25.281187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.484102+0000","last_change":"2023-10-14T10:36:03.872244+0000","last_active":"2023-10-14T10:36:05.484102+0000","last_peered":"2023-10-14T10:36:05.484102+0000","last_clean":"2023-10-14T10:36:05.484102+0000","last_became_active":"2023-10-14T10:36:03.871841+0000","last_became_peered":"2023-10-14T10:36:03.871841+0000","last_unstale":"2023-10-14T10:36:05.484102+0000","last_undegraded":"2023-10-14T10:36:05.484102+0000","last_fullsized":"2023-10-14T10:36:05.484102+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:07:32.070142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.846600+0000","last_change":"2023-10-14T10:36:02.854777+0000","last_active":"2023-10-14T10:36:04.846600+0000","last_peered":"2023-10-14T10:36:04.846600+0000","last_clean":"2023-10-14T10:36:04.846600+0000","last_became_active":"2023-10-14T10:36:02.854126+0000","last_became_peered":"2023-10-14T10:36:02.854126+0000","last_unstale":"2023-10-14T10:36:04.846600+0000","last_undegraded":"2023-10-14T10:36:04.846600+0000","last_fullsized":"2023-10-14T10:36:04.846600+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:22:57.143570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.847151+0000","last_change":"2023-10-14T10:36:02.853800+0000","last_active":"2023-10-14T10:36:04.847151+0000","last_peered":"2023-10-14T10:36:04.847151+0000","last_clean":"2023-10-14T10:36:04.847151+0000","last_became_active":"2023-10-14T10:36:02.853508+0000","last_became_peered":"2023-10-14T10:36:02.853508+0000","last_unstale":"2023-10-14T10:36:04.847151+0000","last_undegraded":"2023-10-14T10:36:04.847151+0000","last_fullsized":"2023-10-14T10:36:04.847151+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:05:51.081176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.841103+0000","last_change":"2023-10-14T10:36:03.856928+0000","last_active":"2023-10-14T10:36:04.841103+0000","last_peered":"2023-10-14T10:36:04.841103+0000","last_clean":"2023-10-14T10:36:04.841103+0000","last_became_active":"2023-10-14T10:36:03.856616+0000","last_became_peered":"2023-10-14T10:36:03.856616+0000","last_unstale":"2023-10-14T10:36:04.841103+0000","last_undegraded":"2023-10-14T10:36:04.841103+0000","last_fullsized":"2023-10-14T10:36:04.841103+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:07:32.070142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:06.814986+0000","last_change":"2023-10-14T10:36:02.847787+0000","last_active":"2023-10-14T10:36:06.814986+0000","last_peered":"2023-10-14T10:36:06.814986+0000","last_clean":"2023-10-14T10:36:06.814986+0000","last_became_active":"2023-10-14T10:36:02.847468+0000","last_became_peered":"2023-10-14T10:36:02.847468+0000","last_unstale":"2023-10-14T10:36:06.814986+0000","last_undegraded":"2023-10-14T10:36:06.814986+0000","last_fullsized":"2023-10-14T10:36:06.814986+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T18:50:25.620127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.484430+0000","last_change":"2023-10-14T10:36:03.869759+0000","last_active":"2023-10-14T10:36:05.484430+0000","last_peered":"2023-10-14T10:36:05.484430+0000","last_clean":"2023-10-14T10:36:05.484430+0000","last_became_active":"2023-10-14T10:36:03.869404+0000","last_became_peered":"2023-10-14T10:36:03.869404+0000","last_unstale":"2023-10-14T10:36:05.484430+0000","last_undegraded":"2023-10-14T10:36:05.484430+0000","last_fullsized":"2023-10-14T10:36:05.484430+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:47:42.089047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-14T10:36:03.868372+0000","last_change":"2023-10-14T10:36:02.853796+0000","last_active":"2023-10-14T10:36:03.868372+0000","last_peered":"2023-10-14T10:36:03.868372+0000","last_clean":"2023-10-14T10:36:03.868372+0000","last_became_active":"2023-10-14T10:36:02.853468+0000","last_became_peered":"2023-10-14T10:36:02.853468+0000","last_unstale":"2023-10-14T10:36:03.868372+0000","last_undegraded":"2023-10-14T10:36:03.868372+0000","last_fullsized":"2023-10-14T10:36:03.868372+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:14:10.501109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.483634+0000","last_change":"2023-10-14T10:36:03.854935+0000","last_active":"2023-10-14T10:36:05.483634+0000","last_peered":"2023-10-14T10:36:05.483634+0000","last_clean":"2023-10-14T10:36:05.483634+0000","last_became_active":"2023-10-14T10:36:03.854648+0000","last_became_peered":"2023-10-14T10:36:03.854648+0000","last_unstale":"2023-10-14T10:36:05.483634+0000","last_undegraded":"2023-10-14T10:36:05.483634+0000","last_fullsized":"2023-10-14T10:36:05.483634+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T10:42:24.026846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.846475+0000","last_change":"2023-10-14T10:36:02.854850+0000","last_active":"2023-10-14T10:36:04.846475+0000","last_peered":"2023-10-14T10:36:04.846475+0000","last_clean":"2023-10-14T10:36:04.846475+0000","last_became_active":"2023-10-14T10:36:02.854442+0000","last_became_peered":"2023-10-14T10:36:02.854442+0000","last_unstale":"2023-10-14T10:36:04.846475+0000","last_undegraded":"2023-10-14T10:36:04.846475+0000","last_fullsized":"2023-10-14T10:36:04.846475+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T12:29:01.996039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.482660+0000","last_change":"2023-10-14T10:36:03.859500+0000","last_active":"2023-10-14T10:36:05.482660+0000","last_peered":"2023-10-14T10:36:05.482660+0000","last_clean":"2023-10-14T10:36:05.482660+0000","last_became_active":"2023-10-14T10:36:03.859247+0000","last_became_peered":"2023-10-14T10:36:03.859247+0000","last_unstale":"2023-10-14T10:36:05.482660+0000","last_undegraded":"2023-10-14T10:36:05.482660+0000","last_fullsized":"2023-10-14T10:36:05.482660+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:14:11.506725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.840826+0000","last_change":"2023-10-14T10:36:02.854036+0000","last_active":"2023-10-14T10:36:04.840826+0000","last_peered":"2023-10-14T10:36:04.840826+0000","last_clean":"2023-10-14T10:36:04.840826+0000","last_became_active":"2023-10-14T10:36:02.853723+0000","last_became_peered":"2023-10-14T10:36:02.853723+0000","last_unstale":"2023-10-14T10:36:04.840826+0000","last_undegraded":"2023-10-14T10:36:04.840826+0000","last_fullsized":"2023-10-14T10:36:04.840826+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:29:50.590931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.482772+0000","last_change":"2023-10-14T10:36:03.856047+0000","last_active":"2023-10-14T10:36:05.482772+0000","last_peered":"2023-10-14T10:36:05.482772+0000","last_clean":"2023-10-14T10:36:05.482772+0000","last_became_active":"2023-10-14T10:36:03.855747+0000","last_became_peered":"2023-10-14T10:36:03.855747+0000","last_unstale":"2023-10-14T10:36:05.482772+0000","last_undegraded":"2023-10-14T10:36:05.482772+0000","last_fullsized":"2023-10-14T10:36:05.482772+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T21:40:43.364841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.841238+0000","last_change":"2023-10-14T10:36:02.845255+0000","last_active":"2023-10-14T10:36:04.841238+0000","last_peered":"2023-10-14T10:36:04.841238+0000","last_clean":"2023-10-14T10:36:04.841238+0000","last_became_active":"2023-10-14T10:36:02.844939+0000","last_became_peered":"2023-10-14T10:36:02.844939+0000","last_unstale":"2023-10-14T10:36:04.841238+0000","last_undegraded":"2023-10-14T10:36:04.841238+0000","last_fullsized":"2023-10-14T10:36:04.841238+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T12:29:01.996039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.482888+0000","last_change":"2023-10-14T10:36:03.865197+0000","last_active":"2023-10-14T10:36:05.482888+0000","last_peered":"2023-10-14T10:36:05.482888+0000","last_clean":"2023-10-14T10:36:05.482888+0000","last_became_active":"2023-10-14T10:36:03.864885+0000","last_became_peered":"2023-10-14T10:36:03.864885+0000","last_unstale":"2023-10-14T10:36:05.482888+0000","last_undegraded":"2023-10-14T10:36:05.482888+0000","last_fullsized":"2023-10-14T10:36:05.482888+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:07:32.070142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.847061+0000","last_change":"2023-10-14T10:36:02.853880+0000","last_active":"2023-10-14T10:36:04.847061+0000","last_peered":"2023-10-14T10:36:04.847061+0000","last_clean":"2023-10-14T10:36:04.847061+0000","last_became_active":"2023-10-14T10:36:02.853574+0000","last_became_peered":"2023-10-14T10:36:02.853574+0000","last_unstale":"2023-10-14T10:36:04.847061+0000","last_undegraded":"2023-10-14T10:36:04.847061+0000","last_fullsized":"2023-10-14T10:36:04.847061+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T10:59:16.502697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.482245+0000","last_change":"2023-10-14T10:36:03.865872+0000","last_active":"2023-10-14T10:36:05.482245+0000","last_peered":"2023-10-14T10:36:05.482245+0000","last_clean":"2023-10-14T10:36:05.482245+0000","last_became_active":"2023-10-14T10:36:03.865624+0000","last_became_peered":"2023-10-14T10:36:03.865624+0000","last_unstale":"2023-10-14T10:36:05.482245+0000","last_undegraded":"2023-10-14T10:36:05.482245+0000","last_fullsized":"2023-10-14T10:36:05.482245+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T18:50:26.625744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-14T10:36:03.842738+0000","last_change":"2023-10-14T10:36:02.840500+0000","last_active":"2023-10-14T10:36:03.842738+0000","last_peered":"2023-10-14T10:36:03.842738+0000","last_clean":"2023-10-14T10:36:03.842738+0000","last_became_active":"2023-10-14T10:36:02.839996+0000","last_became_peered":"2023-10-14T10:36:02.839996+0000","last_unstale":"2023-10-14T10:36:03.842738+0000","last_undegraded":"2023-10-14T10:36:03.842738+0000","last_fullsized":"2023-10-14T10:36:03.842738+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T21:06:31.726029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.847199+0000","last_change":"2023-10-14T10:36:03.868941+0000","last_active":"2023-10-14T10:36:04.847199+0000","last_peered":"2023-10-14T10:36:04.847199+0000","last_clean":"2023-10-14T10:36:04.847199+0000","last_became_active":"2023-10-14T10:36:03.868609+0000","last_became_peered":"2023-10-14T10:36:03.868609+0000","last_unstale":"2023-10-14T10:36:04.847199+0000","last_undegraded":"2023-10-14T10:36:04.847199+0000","last_fullsized":"2023-10-14T10:36:04.847199+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:14:41.057137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.841048+0000","last_change":"2023-10-14T10:36:02.848004+0000","last_active":"2023-10-14T10:36:04.841048+0000","last_peered":"2023-10-14T10:36:04.841048+0000","last_clean":"2023-10-14T10:36:04.841048+0000","last_became_active":"2023-10-14T10:36:02.847494+0000","last_became_peered":"2023-10-14T10:36:02.847494+0000","last_unstale":"2023-10-14T10:36:04.841048+0000","last_undegraded":"2023-10-14T10:36:04.841048+0000","last_fullsized":"2023-10-14T10:36:04.841048+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T14:12:52.738066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.845506+0000","last_change":"2023-10-14T10:36:03.863211+0000","last_active":"2023-10-14T10:36:04.845506+0000","last_peered":"2023-10-14T10:36:04.845506+0000","last_clean":"2023-10-14T10:36:04.845506+0000","last_became_active":"2023-10-14T10:36:03.862894+0000","last_became_peered":"2023-10-14T10:36:03.862894+0000","last_unstale":"2023-10-14T10:36:04.845506+0000","last_undegraded":"2023-10-14T10:36:04.845506+0000","last_fullsized":"2023-10-14T10:36:04.845506+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:27:25.021169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-14T10:36:03.864535+0000","last_change":"2023-10-14T10:36:02.842899+0000","last_active":"2023-10-14T10:36:03.864535+0000","last_peered":"2023-10-14T10:36:03.864535+0000","last_clean":"2023-10-14T10:36:03.864535+0000","last_became_active":"2023-10-14T10:36:02.842620+0000","last_became_peered":"2023-10-14T10:36:02.842620+0000","last_unstale":"2023-10-14T10:36:03.864535+0000","last_undegraded":"2023-10-14T10:36:03.864535+0000","last_fullsized":"2023-10-14T10:36:03.864535+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T12:44:16.134070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.847179+0000","last_change":"2023-10-14T10:36:03.872451+0000","last_active":"2023-10-14T10:36:04.847179+0000","last_peered":"2023-10-14T10:36:04.847179+0000","last_clean":"2023-10-14T10:36:04.847179+0000","last_became_active":"2023-10-14T10:36:03.872212+0000","last_became_peered":"2023-10-14T10:36:03.872212+0000","last_unstale":"2023-10-14T10:36:04.847179+0000","last_undegraded":"2023-10-14T10:36:04.847179+0000","last_fullsized":"2023-10-14T10:36:04.847179+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T12:31:06.179885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.482245+0000","last_change":"2023-10-14T10:36:03.862676+0000","last_active":"2023-10-14T10:36:05.482245+0000","last_peered":"2023-10-14T10:36:05.482245+0000","last_clean":"2023-10-14T10:36:05.482245+0000","last_became_active":"2023-10-14T10:36:03.862364+0000","last_became_peered":"2023-10-14T10:36:03.862364+0000","last_unstale":"2023-10-14T10:36:05.482245+0000","last_undegraded":"2023-10-14T10:36:05.482245+0000","last_fullsized":"2023-10-14T10:36:05.482245+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T10:42:24.026846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.483021+0000","last_change":"2023-10-14T10:36:03.856448+0000","last_active":"2023-10-14T10:36:05.483021+0000","last_peered":"2023-10-14T10:36:05.483021+0000","last_clean":"2023-10-14T10:36:05.483021+0000","last_became_active":"2023-10-14T10:36:03.856165+0000","last_became_peered":"2023-10-14T10:36:03.856165+0000","last_unstale":"2023-10-14T10:36:05.483021+0000","last_undegraded":"2023-10-14T10:36:05.483021+0000","last_fullsized":"2023-10-14T10:36:05.483021+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:21:06.827004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.845673+0000","last_change":"2023-10-14T10:36:03.862566+0000","last_active":"2023-10-14T10:36:04.845673+0000","last_peered":"2023-10-14T10:36:04.845673+0000","last_clean":"2023-10-14T10:36:04.845673+0000","last_became_active":"2023-10-14T10:36:03.862028+0000","last_became_peered":"2023-10-14T10:36:03.862028+0000","last_unstale":"2023-10-14T10:36:04.845673+0000","last_undegraded":"2023-10-14T10:36:04.845673+0000","last_fullsized":"2023-10-14T10:36:04.845673+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T10:36:37.449003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.483846+0000","last_change":"2023-10-14T10:36:03.865325+0000","last_active":"2023-10-14T10:36:05.483846+0000","last_peered":"2023-10-14T10:36:05.483846+0000","last_clean":"2023-10-14T10:36:05.483846+0000","last_became_active":"2023-10-14T10:36:03.865034+0000","last_became_peered":"2023-10-14T10:36:03.865034+0000","last_unstale":"2023-10-14T10:36:05.483846+0000","last_undegraded":"2023-10-14T10:36:05.483846+0000","last_fullsized":"2023-10-14T10:36:05.483846+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T21:40:43.364841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.483268+0000","last_change":"2023-10-14T10:36:03.857566+0000","last_active":"2023-10-14T10:36:05.483268+0000","last_peered":"2023-10-14T10:36:05.483268+0000","last_clean":"2023-10-14T10:36:05.483268+0000","last_became_active":"2023-10-14T10:36:03.857301+0000","last_became_peered":"2023-10-14T10:36:03.857301+0000","last_unstale":"2023-10-14T10:36:05.483268+0000","last_undegraded":"2023-10-14T10:36:05.483268+0000","last_fullsized":"2023-10-14T10:36:05.483268+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T18:49:18.106622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.841002+0000","last_change":"2023-10-14T10:36:03.863392+0000","last_active":"2023-10-14T10:36:04.841002+0000","last_peered":"2023-10-14T10:36:04.841002+0000","last_clean":"2023-10-14T10:36:04.841002+0000","last_became_active":"2023-10-14T10:36:03.863097+0000","last_became_peered":"2023-10-14T10:36:03.863097+0000","last_unstale":"2023-10-14T10:36:04.841002+0000","last_undegraded":"2023-10-14T10:36:04.841002+0000","last_fullsized":"2023-10-14T10:36:04.841002+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:05:52.086793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.482815+0000","last_change":"2023-10-14T10:36:03.864988+0000","last_active":"2023-10-14T10:36:05.482815+0000","last_peered":"2023-10-14T10:36:05.482815+0000","last_clean":"2023-10-14T10:36:05.482815+0000","last_became_active":"2023-10-14T10:36:03.864699+0000","last_became_peered":"2023-10-14T10:36:03.864699+0000","last_unstale":"2023-10-14T10:36:05.482815+0000","last_undegraded":"2023-10-14T10:36:05.482815+0000","last_fullsized":"2023-10-14T10:36:05.482815+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T18:42:51.749474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.484472+0000","last_change":"2023-10-14T10:36:03.855927+0000","last_active":"2023-10-14T10:36:05.484472+0000","last_peered":"2023-10-14T10:36:05.484472+0000","last_clean":"2023-10-14T10:36:05.484472+0000","last_became_active":"2023-10-14T10:36:03.855643+0000","last_became_peered":"2023-10-14T10:36:03.855643+0000","last_unstale":"2023-10-14T10:36:05.484472+0000","last_undegraded":"2023-10-14T10:36:05.484472+0000","last_fullsized":"2023-10-14T10:36:05.484472+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T20:52:07.681036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.845305+0000","last_change":"2023-10-14T10:36:03.864752+0000","last_active":"2023-10-14T10:36:04.845305+0000","last_peered":"2023-10-14T10:36:04.845305+0000","last_clean":"2023-10-14T10:36:04.845305+0000","last_became_active":"2023-10-14T10:36:03.864480+0000","last_became_peered":"2023-10-14T10:36:03.864480+0000","last_unstale":"2023-10-14T10:36:04.845305+0000","last_undegraded":"2023-10-14T10:36:04.845305+0000","last_fullsized":"2023-10-14T10:36:04.845305+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:05:52.086793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.482676+0000","last_change":"2023-10-14T10:36:03.858250+0000","last_active":"2023-10-14T10:36:05.482676+0000","last_peered":"2023-10-14T10:36:05.482676+0000","last_clean":"2023-10-14T10:36:05.482676+0000","last_became_active":"2023-10-14T10:36:03.857940+0000","last_became_peered":"2023-10-14T10:36:03.857940+0000","last_unstale":"2023-10-14T10:36:05.482676+0000","last_undegraded":"2023-10-14T10:36:05.482676+0000","last_fullsized":"2023-10-14T10:36:05.482676+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T17:28:51.336361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.845474+0000","last_change":"2023-10-14T10:36:03.866904+0000","last_active":"2023-10-14T10:36:04.845474+0000","last_peered":"2023-10-14T10:36:04.845474+0000","last_clean":"2023-10-14T10:36:04.845474+0000","last_became_active":"2023-10-14T10:36:03.866439+0000","last_became_peered":"2023-10-14T10:36:03.866439+0000","last_unstale":"2023-10-14T10:36:04.845474+0000","last_undegraded":"2023-10-14T10:36:04.845474+0000","last_fullsized":"2023-10-14T10:36:04.845474+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T18:49:18.106622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.483685+0000","last_change":"2023-10-14T10:36:03.868286+0000","last_active":"2023-10-14T10:36:05.483685+0000","last_peered":"2023-10-14T10:36:05.483685+0000","last_clean":"2023-10-14T10:36:05.483685+0000","last_became_active":"2023-10-14T10:36:03.868050+0000","last_became_peered":"2023-10-14T10:36:03.868050+0000","last_unstale":"2023-10-14T10:36:05.483685+0000","last_undegraded":"2023-10-14T10:36:05.483685+0000","last_fullsized":"2023-10-14T10:36:05.483685+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T18:50:26.625744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.482746+0000","last_change":"2023-10-14T10:36:03.872106+0000","last_active":"2023-10-14T10:36:05.482746+0000","last_peered":"2023-10-14T10:36:05.482746+0000","last_clean":"2023-10-14T10:36:05.482746+0000","last_became_active":"2023-10-14T10:36:03.871788+0000","last_became_peered":"2023-10-14T10:36:03.871788+0000","last_unstale":"2023-10-14T10:36:05.482746+0000","last_undegraded":"2023-10-14T10:36:05.482746+0000","last_fullsized":"2023-10-14T10:36:05.482746+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T17:28:51.336361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542150,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":823296,"data_stored":669814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45500000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.376}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]}]},{"osd":6,"up_from":14,"seq":60129542151,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":237568,"data_stored":80476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63800000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66800000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.372}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51400000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47199999999999998}]}]},{"osd":1,"up_from":14,"seq":60129542150,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":233472,"data_stored":79024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32900000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27600000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60199999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.629}]}]},{"osd":0,"up_from":14,"seq":60129542149,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27832,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344008,"statfs":{"total":96636764160,"available":96608264192,"internally_reserved":0,"allocated":831488,"data_stored":669870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40500000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35399999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60399999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65400000000000003}]}]},{"osd":2,"up_from":14,"seq":60129542150,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":225280,"data_stored":78864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.429}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5269999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1890000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4950000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.099}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0580000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98099999999999998}]}]},{"osd":3,"up_from":14,"seq":60129542150,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":237568,"data_stored":80476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.311}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46600000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47699999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542149,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":237568,"data_stored":78990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79300000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.872}]}]},{"osd":5,"up_from":14,"seq":60129542150,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":78954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56399999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66200000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.436}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41499999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-14T10:36:12.999 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T10:36:13.158 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.157+0000 7fb0cf5b4700 1 Processor -- start 2023-10-14T10:36:13.159 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.158+0000 7fb0cf5b4700 1 -- start start 2023-10-14T10:36:13.161 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.158+0000 7fb0cf5b4700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb0c815f1f0 0x7fb0c80cf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:13.161 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.158+0000 7fb0cf5b4700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fb0c80cfc60 0x7fb0c80cbf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:13.161 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.158+0000 7fb0cf5b4700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0c815e7a0 con 0x7fb0c815ab10 2023-10-14T10:36:13.162 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.158+0000 7fb0cf5b4700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0c80d0090 con 0x7fb0c815f1f0 2023-10-14T10:36:13.162 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.158+0000 7fb0cf5b4700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0c80b29a0 con 0x7fb0c80cfc60 2023-10-14T10:36:13.162 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.159+0000 7fb0ccb4f700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb0c815f1f0 0x7fb0c80cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:13.162 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.159+0000 7fb0ccb4f700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb0c815f1f0 0x7fb0c80cf720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44330/0 (socket says 172.21.15.67:44330) 2023-10-14T10:36:13.163 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.159+0000 7fb0ccb4f700 1 -- 172.21.15.67:0/678641583 learned_addr learned my addr 172.21.15.67:0/678641583 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:13.163 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.159+0000 7fb0ccb4f700 1 -- 172.21.15.67:0/678641583 >> v2:172.21.15.72:3300/0 conn(0x7fb0c80cfc60 msgr2=0x7fb0c80cbf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:13.163 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.159+0000 7fb0cdb51700 1 --2- 172.21.15.67:0/678641583 >> v2:172.21.15.72:3300/0 conn(0x7fb0c80cfc60 0x7fb0c80cbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:13.163 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.159+0000 7fb0ccb4f700 1 --2- 172.21.15.67:0/678641583 >> v2:172.21.15.72:3300/0 conn(0x7fb0c80cfc60 0x7fb0c80cbf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:13.163 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.159+0000 7fb0ccb4f700 1 -- 172.21.15.67:0/678641583 >> v1:172.21.15.72:6789/0 conn(0x7fb0c815ab10 legacy=0x7fb0c80b1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:13.164 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.159+0000 7fb0ccb4f700 1 -- 172.21.15.67:0/678641583 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0c80b2c80 con 0x7fb0c815f1f0 2023-10-14T10:36:13.164 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.159+0000 7fb0cdb51700 1 --2- 172.21.15.67:0/678641583 >> v2:172.21.15.72:3300/0 conn(0x7fb0c80cfc60 0x7fb0c80cbf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-14T10:36:13.164 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.159+0000 7fb0ccb4f700 1 --2- 172.21.15.67:0/678641583 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0c815f1f0 0x7fb0c80cf720 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb0c80d03e0 tx=0x7fb0c4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e3485c71ba422ce server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:13.164 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.159+0000 7fb0bffff700 1 -- 172.21.15.67:0/678641583 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c4005e30 con 0x7fb0c815f1f0 2023-10-14T10:36:13.164 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.159+0000 7fb0bffff700 1 -- 172.21.15.67:0/678641583 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0c4006030 con 0x7fb0c815f1f0 2023-10-14T10:36:13.165 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.159+0000 7fb0bffff700 1 -- 172.21.15.67:0/678641583 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c40063e0 con 0x7fb0c815f1f0 2023-10-14T10:36:13.165 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.160+0000 7fb0cf5b4700 1 -- 172.21.15.67:0/678641583 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0c815f1f0 msgr2=0x7fb0c80cf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:13.165 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.160+0000 7fb0cf5b4700 1 --2- 172.21.15.67:0/678641583 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0c815f1f0 0x7fb0c80cf720 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb0c80d03e0 tx=0x7fb0c4009bd0 comp rx=0 tx=0).stop 2023-10-14T10:36:13.165 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.160+0000 7fb0cf5b4700 1 -- 172.21.15.67:0/678641583 shutdown_connections 2023-10-14T10:36:13.165 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.160+0000 7fb0cf5b4700 1 --2- 172.21.15.67:0/678641583 >> v2:172.21.15.72:3300/0 conn(0x7fb0c80cfc60 0x7fb0c80cbf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:13.166 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.160+0000 7fb0cf5b4700 1 --2- 172.21.15.67:0/678641583 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0c815f1f0 0x7fb0c80cf720 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:13.166 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.160+0000 7fb0cf5b4700 1 -- 172.21.15.67:0/678641583 >> 172.21.15.67:0/678641583 conn(0x7fb0c806dae0 msgr2=0x7fb0c806def0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:13.166 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.160+0000 7fb0cf5b4700 1 -- 172.21.15.67:0/678641583 shutdown_connections 2023-10-14T10:36:13.166 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.160+0000 7fb0cf5b4700 1 -- 172.21.15.67:0/678641583 wait complete. 2023-10-14T10:36:13.170 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.161+0000 7fb0cf5b4700 1 Processor -- start 2023-10-14T10:36:13.170 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.161+0000 7fb0cf5b4700 1 -- start start 2023-10-14T10:36:13.170 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.162+0000 7fb0cf5b4700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb0c80cfc60 0x7fb0c81efb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:13.170 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.162+0000 7fb0cf5b4700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0c815ab10 0x7fb0c80aff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:13.171 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.162+0000 7fb0cf5b4700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fb0c815f1f0 0x7fb0c80b0470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:13.171 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.162+0000 7fb0cf5b4700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c80c72f0 con 0x7fb0c815ab10 2023-10-14T10:36:13.171 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.162+0000 7fb0cf5b4700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c815b240 con 0x7fb0c80cfc60 2023-10-14T10:36:13.171 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.162+0000 7fb0cf5b4700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c815b520 con 0x7fb0c815f1f0 2023-10-14T10:36:13.171 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.162+0000 7fb0ccb4f700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0c815ab10 0x7fb0c80aff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:13.172 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.162+0000 7fb0ccb4f700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0c815ab10 0x7fb0c80aff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44336/0 (socket says 172.21.15.67:44336) 2023-10-14T10:36:13.172 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.162+0000 7fb0ccb4f700 1 -- 172.21.15.67:0/2528048860 learned_addr learned my addr 172.21.15.67:0/2528048860 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:13.172 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.162+0000 7fb0cd350700 1 --2- 172.21.15.67:0/2528048860 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb0c80cfc60 0x7fb0c81efb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:13.172 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.162+0000 7fb0cdb51700 1 --2- 172.21.15.67:0/2528048860 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fb0c815f1f0 0x7fb0c80b0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:13.172 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.162+0000 7fb0ccb4f700 1 -- 172.21.15.67:0/2528048860 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fb0c815f1f0 msgr2=0x7fb0c80b0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:13.173 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.162+0000 7fb0ccb4f700 1 --2- 172.21.15.67:0/2528048860 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fb0c815f1f0 0x7fb0c80b0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:13.173 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.162+0000 7fb0ccb4f700 1 -- 172.21.15.67:0/2528048860 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb0c80cfc60 msgr2=0x7fb0c81efb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:13.173 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.162+0000 7fb0ccb4f700 1 --2- 172.21.15.67:0/2528048860 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb0c80cfc60 0x7fb0c81efb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:13.173 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.162+0000 7fb0ccb4f700 1 -- 172.21.15.67:0/2528048860 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0c4009500 con 0x7fb0c815ab10 2023-10-14T10:36:13.173 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.162+0000 7fb0cdb51700 1 --2- 172.21.15.67:0/2528048860 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fb0c815f1f0 0x7fb0c80b0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-14T10:36:13.174 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.162+0000 7fb0ccb4f700 1 --2- 172.21.15.67:0/2528048860 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0c815ab10 0x7fb0c80aff30 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb0c4013040 tx=0x7fb0c4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:13.174 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.162+0000 7fb0bdffb700 1 -- 172.21.15.67:0/2528048860 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c4020070 con 0x7fb0c815ab10 2023-10-14T10:36:13.174 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.162+0000 7fb0bdffb700 1 -- 172.21.15.67:0/2528048860 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0c40068d0 con 0x7fb0c815ab10 2023-10-14T10:36:13.174 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.162+0000 7fb0cf5b4700 1 -- 172.21.15.67:0/2528048860 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0c815b800 con 0x7fb0c815ab10 2023-10-14T10:36:13.174 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.162+0000 7fb0bdffb700 1 -- 172.21.15.67:0/2528048860 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c4004290 con 0x7fb0c815ab10 2023-10-14T10:36:13.175 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.163+0000 7fb0cf5b4700 1 -- 172.21.15.67:0/2528048860 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0c81f0050 con 0x7fb0c815ab10 2023-10-14T10:36:13.175 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.164+0000 7fb0cf5b4700 1 -- 172.21.15.67:0/2528048860 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0c815bd20 con 0x7fb0c815ab10 2023-10-14T10:36:13.175 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.164+0000 7fb0bdffb700 1 -- 172.21.15.67:0/2528048860 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fb0c40043f0 con 0x7fb0c815ab10 2023-10-14T10:36:13.175 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.164+0000 7fb0bdffb700 1 --2- 172.21.15.67:0/2528048860 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fb0ac06af00 0x7fb0ac06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:13.175 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.165+0000 7fb0bdffb700 1 -- 172.21.15.67:0/2528048860 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb0c408c060 con 0x7fb0c815ab10 2023-10-14T10:36:13.176 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.165+0000 7fb0cd350700 1 --2- 172.21.15.67:0/2528048860 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fb0ac06af00 0x7fb0ac06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:13.176 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.165+0000 7fb0cd350700 1 --2- 172.21.15.67:0/2528048860 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fb0ac06af00 0x7fb0ac06d3c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb0b8009960 tx=0x7fb0b80090e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:13.176 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.169+0000 7fb0bdffb700 1 -- 172.21.15.67:0/2528048860 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb0c40beca0 con 0x7fb0c815ab10 2023-10-14T10:36:13.364 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.363+0000 7fb0cf5b4700 1 -- 172.21.15.67:0/2528048860 --> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb0c815ecd0 con 0x7fb0ac06af00 2023-10-14T10:36:13.371 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.371+0000 7fb0bdffb700 1 -- 172.21.15.67:0/2528048860 <== mgr.4100 v2:172.21.15.72:6832/703488382 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264532 (secure 0 0 0) 0x7fb0b8009e30 con 0x7fb0ac06af00 2023-10-14T10:36:13.375 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-14T10:36:13.378 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.374+0000 7fb0cf5b4700 1 -- 172.21.15.67:0/2528048860 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fb0ac06af00 msgr2=0x7fb0ac06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:13.378 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.374+0000 7fb0cf5b4700 1 --2- 172.21.15.67:0/2528048860 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fb0ac06af00 0x7fb0ac06d3c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb0b8009960 tx=0x7fb0b80090e0 comp rx=0 tx=0).stop 2023-10-14T10:36:13.378 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.374+0000 7fb0cf5b4700 1 -- 172.21.15.67:0/2528048860 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0c815ab10 msgr2=0x7fb0c80aff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:13.379 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.374+0000 7fb0cf5b4700 1 --2- 172.21.15.67:0/2528048860 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0c815ab10 0x7fb0c80aff30 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb0c4013040 tx=0x7fb0c4004750 comp rx=0 tx=0).stop 2023-10-14T10:36:13.379 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.374+0000 7fb0cf5b4700 1 -- 172.21.15.67:0/2528048860 shutdown_connections 2023-10-14T10:36:13.379 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.374+0000 7fb0cf5b4700 1 --2- 172.21.15.67:0/2528048860 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fb0ac06af00 0x7fb0ac06d3c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:13.379 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.374+0000 7fb0cf5b4700 1 --2- 172.21.15.67:0/2528048860 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fb0c815f1f0 0x7fb0c80b0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:13.379 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.374+0000 7fb0cf5b4700 1 --2- 172.21.15.67:0/2528048860 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb0c80cfc60 0x7fb0c81efb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:13.380 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.374+0000 7fb0cf5b4700 1 --2- 172.21.15.67:0/2528048860 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0c815ab10 0x7fb0c80aff30 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:13.380 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.374+0000 7fb0cf5b4700 1 -- 172.21.15.67:0/2528048860 >> 172.21.15.67:0/2528048860 conn(0x7fb0c806dae0 msgr2=0x7fb0c8158b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:13.380 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.374+0000 7fb0cf5b4700 1 -- 172.21.15.67:0/2528048860 shutdown_connections 2023-10-14T10:36:13.380 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.374+0000 7fb0cf5b4700 1 -- 172.21.15.67:0/2528048860 wait complete. 2023-10-14T10:36:13.381 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2023-10-14T10:36:13.387 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-10-14T10:36:11.756762+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219164,"kb_used_data":2984,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755556,"statfs":{"total":773094113280,"available":772869689344,"internally_reserved":0,"allocated":3055616,"data_stored":1816468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12742,"internal_metadata":221236794},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002099"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.847283+0000","last_change":"2023-10-14T10:36:03.869543+0000","last_active":"2023-10-14T10:36:04.847283+0000","last_peered":"2023-10-14T10:36:04.847283+0000","last_clean":"2023-10-14T10:36:04.847283+0000","last_became_active":"2023-10-14T10:36:03.868884+0000","last_became_peered":"2023-10-14T10:36:03.868884+0000","last_unstale":"2023-10-14T10:36:04.847283+0000","last_undegraded":"2023-10-14T10:36:04.847283+0000","last_fullsized":"2023-10-14T10:36:04.847283+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:31:25.399614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.841189+0000","last_change":"2023-10-14T10:36:03.867145+0000","last_active":"2023-10-14T10:36:04.841189+0000","last_peered":"2023-10-14T10:36:04.841189+0000","last_clean":"2023-10-14T10:36:04.841189+0000","last_became_active":"2023-10-14T10:36:03.866743+0000","last_became_peered":"2023-10-14T10:36:03.866743+0000","last_unstale":"2023-10-14T10:36:04.841189+0000","last_undegraded":"2023-10-14T10:36:04.841189+0000","last_fullsized":"2023-10-14T10:36:04.841189+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T18:49:18.106622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.483814+0000","last_change":"2023-10-14T10:36:03.861171+0000","last_active":"2023-10-14T10:36:05.483814+0000","last_peered":"2023-10-14T10:36:05.483814+0000","last_clean":"2023-10-14T10:36:05.483814+0000","last_became_active":"2023-10-14T10:36:03.860860+0000","last_became_peered":"2023-10-14T10:36:03.860860+0000","last_unstale":"2023-10-14T10:36:05.483814+0000","last_undegraded":"2023-10-14T10:36:05.483814+0000","last_fullsized":"2023-10-14T10:36:05.483814+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T17:28:51.336361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.845699+0000","last_change":"2023-10-14T10:36:03.861631+0000","last_active":"2023-10-14T10:36:04.845699+0000","last_peered":"2023-10-14T10:36:04.845699+0000","last_clean":"2023-10-14T10:36:04.845699+0000","last_became_active":"2023-10-14T10:36:03.861291+0000","last_became_peered":"2023-10-14T10:36:03.861291+0000","last_unstale":"2023-10-14T10:36:04.845699+0000","last_undegraded":"2023-10-14T10:36:04.845699+0000","last_fullsized":"2023-10-14T10:36:04.845699+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T17:52:45.149486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.846931+0000","last_change":"2023-10-14T10:36:03.870469+0000","last_active":"2023-10-14T10:36:04.846931+0000","last_peered":"2023-10-14T10:36:04.846931+0000","last_clean":"2023-10-14T10:36:04.846931+0000","last_became_active":"2023-10-14T10:36:03.865743+0000","last_became_peered":"2023-10-14T10:36:03.865743+0000","last_unstale":"2023-10-14T10:36:04.846931+0000","last_undegraded":"2023-10-14T10:36:04.846931+0000","last_fullsized":"2023-10-14T10:36:04.846931+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:33:09.576282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.845553+0000","last_change":"2023-10-14T10:36:03.862386+0000","last_active":"2023-10-14T10:36:04.845553+0000","last_peered":"2023-10-14T10:36:04.845553+0000","last_clean":"2023-10-14T10:36:04.845553+0000","last_became_active":"2023-10-14T10:36:03.861705+0000","last_became_peered":"2023-10-14T10:36:03.861705+0000","last_unstale":"2023-10-14T10:36:04.845553+0000","last_undegraded":"2023-10-14T10:36:04.845553+0000","last_fullsized":"2023-10-14T10:36:04.845553+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:47:42.089047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.846829+0000","last_change":"2023-10-14T10:36:03.865502+0000","last_active":"2023-10-14T10:36:04.846829+0000","last_peered":"2023-10-14T10:36:04.846829+0000","last_clean":"2023-10-14T10:36:04.846829+0000","last_became_active":"2023-10-14T10:36:03.865238+0000","last_became_peered":"2023-10-14T10:36:03.865238+0000","last_unstale":"2023-10-14T10:36:04.846829+0000","last_undegraded":"2023-10-14T10:36:04.846829+0000","last_fullsized":"2023-10-14T10:36:04.846829+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T20:52:07.681036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.482589+0000","last_change":"2023-10-14T10:36:03.859238+0000","last_active":"2023-10-14T10:36:05.482589+0000","last_peered":"2023-10-14T10:36:05.482589+0000","last_clean":"2023-10-14T10:36:05.482589+0000","last_became_active":"2023-10-14T10:36:03.858981+0000","last_became_peered":"2023-10-14T10:36:03.858981+0000","last_unstale":"2023-10-14T10:36:05.482589+0000","last_undegraded":"2023-10-14T10:36:05.482589+0000","last_fullsized":"2023-10-14T10:36:05.482589+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T12:18:09.680616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.847259+0000","last_change":"2023-10-14T10:36:03.868883+0000","last_active":"2023-10-14T10:36:04.847259+0000","last_peered":"2023-10-14T10:36:04.847259+0000","last_clean":"2023-10-14T10:36:04.847259+0000","last_became_active":"2023-10-14T10:36:03.868019+0000","last_became_peered":"2023-10-14T10:36:03.868019+0000","last_unstale":"2023-10-14T10:36:04.847259+0000","last_undegraded":"2023-10-14T10:36:04.847259+0000","last_fullsized":"2023-10-14T10:36:04.847259+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T17:08:17.694068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.482896+0000","last_change":"2023-10-14T10:36:03.861523+0000","last_active":"2023-10-14T10:36:05.482896+0000","last_peered":"2023-10-14T10:36:05.482896+0000","last_clean":"2023-10-14T10:36:05.482896+0000","last_became_active":"2023-10-14T10:36:03.860708+0000","last_became_peered":"2023-10-14T10:36:03.860708+0000","last_unstale":"2023-10-14T10:36:05.482896+0000","last_undegraded":"2023-10-14T10:36:05.482896+0000","last_fullsized":"2023-10-14T10:36:05.482896+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:07:32.070142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.482672+0000","last_change":"2023-10-14T10:36:03.861278+0000","last_active":"2023-10-14T10:36:05.482672+0000","last_peered":"2023-10-14T10:36:05.482672+0000","last_clean":"2023-10-14T10:36:05.482672+0000","last_became_active":"2023-10-14T10:36:03.860994+0000","last_became_peered":"2023-10-14T10:36:03.860994+0000","last_unstale":"2023-10-14T10:36:05.482672+0000","last_undegraded":"2023-10-14T10:36:05.482672+0000","last_fullsized":"2023-10-14T10:36:05.482672+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T12:18:09.680616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.482785+0000","last_change":"2023-10-14T10:36:03.871181+0000","last_active":"2023-10-14T10:36:05.482785+0000","last_peered":"2023-10-14T10:36:05.482785+0000","last_clean":"2023-10-14T10:36:05.482785+0000","last_became_active":"2023-10-14T10:36:03.870875+0000","last_became_peered":"2023-10-14T10:36:03.870875+0000","last_unstale":"2023-10-14T10:36:05.482785+0000","last_undegraded":"2023-10-14T10:36:05.482785+0000","last_fullsized":"2023-10-14T10:36:05.482785+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:50:35.924528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.482529+0000","last_change":"2023-10-14T10:36:03.857251+0000","last_active":"2023-10-14T10:36:05.482529+0000","last_peered":"2023-10-14T10:36:05.482529+0000","last_clean":"2023-10-14T10:36:05.482529+0000","last_became_active":"2023-10-14T10:36:03.856908+0000","last_became_peered":"2023-10-14T10:36:03.856908+0000","last_unstale":"2023-10-14T10:36:05.482529+0000","last_undegraded":"2023-10-14T10:36:05.482529+0000","last_fullsized":"2023-10-14T10:36:05.482529+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T18:50:26.625744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.484165+0000","last_change":"2023-10-14T10:36:03.858760+0000","last_active":"2023-10-14T10:36:05.484165+0000","last_peered":"2023-10-14T10:36:05.484165+0000","last_clean":"2023-10-14T10:36:05.484165+0000","last_became_active":"2023-10-14T10:36:03.854779+0000","last_became_peered":"2023-10-14T10:36:03.854779+0000","last_unstale":"2023-10-14T10:36:05.484165+0000","last_undegraded":"2023-10-14T10:36:05.484165+0000","last_fullsized":"2023-10-14T10:36:05.484165+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T18:49:18.106622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.840868+0000","last_change":"2023-10-14T10:36:03.861476+0000","last_active":"2023-10-14T10:36:04.840868+0000","last_peered":"2023-10-14T10:36:04.840868+0000","last_clean":"2023-10-14T10:36:04.840868+0000","last_became_active":"2023-10-14T10:36:03.859641+0000","last_became_peered":"2023-10-14T10:36:03.859641+0000","last_unstale":"2023-10-14T10:36:04.840868+0000","last_undegraded":"2023-10-14T10:36:04.840868+0000","last_fullsized":"2023-10-14T10:36:04.840868+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T21:40:43.364841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.845130+0000","last_change":"2023-10-14T10:36:03.858710+0000","last_active":"2023-10-14T10:36:04.845130+0000","last_peered":"2023-10-14T10:36:04.845130+0000","last_clean":"2023-10-14T10:36:04.845130+0000","last_became_active":"2023-10-14T10:36:03.858456+0000","last_became_peered":"2023-10-14T10:36:03.858456+0000","last_unstale":"2023-10-14T10:36:04.845130+0000","last_undegraded":"2023-10-14T10:36:04.845130+0000","last_fullsized":"2023-10-14T10:36:04.845130+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:07:32.070142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.845836+0000","last_change":"2023-10-14T10:36:03.867067+0000","last_active":"2023-10-14T10:36:04.845836+0000","last_peered":"2023-10-14T10:36:04.845836+0000","last_clean":"2023-10-14T10:36:04.845836+0000","last_became_active":"2023-10-14T10:36:03.866802+0000","last_became_peered":"2023-10-14T10:36:03.866802+0000","last_unstale":"2023-10-14T10:36:04.845836+0000","last_undegraded":"2023-10-14T10:36:04.845836+0000","last_fullsized":"2023-10-14T10:36:04.845836+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:09:22.818455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.482788+0000","last_change":"2023-10-14T10:36:03.869409+0000","last_active":"2023-10-14T10:36:05.482788+0000","last_peered":"2023-10-14T10:36:05.482788+0000","last_clean":"2023-10-14T10:36:05.482788+0000","last_became_active":"2023-10-14T10:36:03.868961+0000","last_became_peered":"2023-10-14T10:36:03.868961+0000","last_unstale":"2023-10-14T10:36:05.482788+0000","last_undegraded":"2023-10-14T10:36:05.482788+0000","last_fullsized":"2023-10-14T10:36:05.482788+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:10:10.517040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.847011+0000","last_change":"2023-10-14T10:36:03.864794+0000","last_active":"2023-10-14T10:36:04.847011+0000","last_peered":"2023-10-14T10:36:04.847011+0000","last_clean":"2023-10-14T10:36:04.847011+0000","last_became_active":"2023-10-14T10:36:03.864195+0000","last_became_peered":"2023-10-14T10:36:03.864195+0000","last_unstale":"2023-10-14T10:36:04.847011+0000","last_undegraded":"2023-10-14T10:36:04.847011+0000","last_fullsized":"2023-10-14T10:36:04.847011+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T17:52:45.149486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.483618+0000","last_change":"2023-10-14T10:36:02.855169+0000","last_active":"2023-10-14T10:36:05.483618+0000","last_peered":"2023-10-14T10:36:05.483618+0000","last_clean":"2023-10-14T10:36:05.483618+0000","last_became_active":"2023-10-14T10:36:02.854817+0000","last_became_peered":"2023-10-14T10:36:02.854817+0000","last_unstale":"2023-10-14T10:36:05.483618+0000","last_undegraded":"2023-10-14T10:36:05.483618+0000","last_fullsized":"2023-10-14T10:36:05.483618+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T14:12:52.738066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.484369+0000","last_change":"2023-10-14T10:36:03.865118+0000","last_active":"2023-10-14T10:36:05.484369+0000","last_peered":"2023-10-14T10:36:05.484369+0000","last_clean":"2023-10-14T10:36:05.484369+0000","last_became_active":"2023-10-14T10:36:03.864579+0000","last_became_peered":"2023-10-14T10:36:03.864579+0000","last_unstale":"2023-10-14T10:36:05.484369+0000","last_undegraded":"2023-10-14T10:36:05.484369+0000","last_fullsized":"2023-10-14T10:36:05.484369+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:27:25.021169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.484313+0000","last_change":"2023-10-14T10:36:02.851587+0000","last_active":"2023-10-14T10:36:05.484313+0000","last_peered":"2023-10-14T10:36:05.484313+0000","last_clean":"2023-10-14T10:36:05.484313+0000","last_became_active":"2023-10-14T10:36:02.851276+0000","last_became_peered":"2023-10-14T10:36:02.851276+0000","last_unstale":"2023-10-14T10:36:05.484313+0000","last_undegraded":"2023-10-14T10:36:05.484313+0000","last_fullsized":"2023-10-14T10:36:05.484313+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T14:04:44.127519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.483564+0000","last_change":"2023-10-14T10:36:03.861320+0000","last_active":"2023-10-14T10:36:05.483564+0000","last_peered":"2023-10-14T10:36:05.483564+0000","last_clean":"2023-10-14T10:36:05.483564+0000","last_became_active":"2023-10-14T10:36:03.860684+0000","last_became_peered":"2023-10-14T10:36:03.860684+0000","last_unstale":"2023-10-14T10:36:05.483564+0000","last_undegraded":"2023-10-14T10:36:05.483564+0000","last_fullsized":"2023-10-14T10:36:05.483564+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:22:58.149187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.847230+0000","last_change":"2023-10-14T10:36:02.854569+0000","last_active":"2023-10-14T10:36:04.847230+0000","last_peered":"2023-10-14T10:36:04.847230+0000","last_clean":"2023-10-14T10:36:04.847230+0000","last_became_active":"2023-10-14T10:36:02.853518+0000","last_became_peered":"2023-10-14T10:36:02.853518+0000","last_unstale":"2023-10-14T10:36:04.847230+0000","last_undegraded":"2023-10-14T10:36:04.847230+0000","last_fullsized":"2023-10-14T10:36:04.847230+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:27:24.015552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.845887+0000","last_change":"2023-10-14T10:36:03.872077+0000","last_active":"2023-10-14T10:36:04.845887+0000","last_peered":"2023-10-14T10:36:04.845887+0000","last_clean":"2023-10-14T10:36:04.845887+0000","last_became_active":"2023-10-14T10:36:03.871740+0000","last_became_peered":"2023-10-14T10:36:03.871740+0000","last_unstale":"2023-10-14T10:36:04.845887+0000","last_undegraded":"2023-10-14T10:36:04.845887+0000","last_fullsized":"2023-10-14T10:36:04.845887+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T12:31:06.179885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.846883+0000","last_change":"2023-10-14T10:36:02.853780+0000","last_active":"2023-10-14T10:36:04.846883+0000","last_peered":"2023-10-14T10:36:04.846883+0000","last_clean":"2023-10-14T10:36:04.846883+0000","last_became_active":"2023-10-14T10:36:02.853450+0000","last_became_peered":"2023-10-14T10:36:02.853450+0000","last_unstale":"2023-10-14T10:36:04.846883+0000","last_undegraded":"2023-10-14T10:36:04.846883+0000","last_fullsized":"2023-10-14T10:36:04.846883+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T10:42:23.021229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.845567+0000","last_change":"2023-10-14T10:36:03.858379+0000","last_active":"2023-10-14T10:36:04.845567+0000","last_peered":"2023-10-14T10:36:04.845567+0000","last_clean":"2023-10-14T10:36:04.845567+0000","last_became_active":"2023-10-14T10:36:03.857904+0000","last_became_peered":"2023-10-14T10:36:03.857904+0000","last_unstale":"2023-10-14T10:36:04.845567+0000","last_undegraded":"2023-10-14T10:36:04.845567+0000","last_fullsized":"2023-10-14T10:36:04.845567+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:33:09.576282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.482737+0000","last_change":"2023-10-14T10:36:02.854396+0000","last_active":"2023-10-14T10:36:05.482737+0000","last_peered":"2023-10-14T10:36:05.482737+0000","last_clean":"2023-10-14T10:36:05.482737+0000","last_became_active":"2023-10-14T10:36:02.854085+0000","last_became_peered":"2023-10-14T10:36:02.854085+0000","last_unstale":"2023-10-14T10:36:05.482737+0000","last_undegraded":"2023-10-14T10:36:05.482737+0000","last_fullsized":"2023-10-14T10:36:05.482737+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T14:12:52.738066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.483060+0000","last_change":"2023-10-14T10:36:03.855010+0000","last_active":"2023-10-14T10:36:05.483060+0000","last_peered":"2023-10-14T10:36:05.483060+0000","last_clean":"2023-10-14T10:36:05.483060+0000","last_became_active":"2023-10-14T10:36:03.854714+0000","last_became_peered":"2023-10-14T10:36:03.854714+0000","last_unstale":"2023-10-14T10:36:05.483060+0000","last_undegraded":"2023-10-14T10:36:05.483060+0000","last_fullsized":"2023-10-14T10:36:05.483060+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:29:51.596547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-14T10:36:03.865241+0000","last_change":"2023-10-14T10:36:02.847573+0000","last_active":"2023-10-14T10:36:03.865241+0000","last_peered":"2023-10-14T10:36:03.865241+0000","last_clean":"2023-10-14T10:36:03.865241+0000","last_became_active":"2023-10-14T10:36:02.847061+0000","last_became_peered":"2023-10-14T10:36:02.847061+0000","last_unstale":"2023-10-14T10:36:03.865241+0000","last_undegraded":"2023-10-14T10:36:03.865241+0000","last_fullsized":"2023-10-14T10:36:03.865241+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:50:34.918912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.484012+0000","last_change":"2023-10-14T10:36:03.866134+0000","last_active":"2023-10-14T10:36:05.484012+0000","last_peered":"2023-10-14T10:36:05.484012+0000","last_clean":"2023-10-14T10:36:05.484012+0000","last_became_active":"2023-10-14T10:36:03.865614+0000","last_became_peered":"2023-10-14T10:36:03.865614+0000","last_unstale":"2023-10-14T10:36:05.484012+0000","last_undegraded":"2023-10-14T10:36:05.484012+0000","last_fullsized":"2023-10-14T10:36:05.484012+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T18:42:51.749474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.482644+0000","last_change":"2023-10-14T10:36:02.846515+0000","last_active":"2023-10-14T10:36:05.482644+0000","last_peered":"2023-10-14T10:36:05.482644+0000","last_clean":"2023-10-14T10:36:05.482644+0000","last_became_active":"2023-10-14T10:36:02.845841+0000","last_became_peered":"2023-10-14T10:36:02.845841+0000","last_unstale":"2023-10-14T10:36:05.482644+0000","last_undegraded":"2023-10-14T10:36:05.482644+0000","last_fullsized":"2023-10-14T10:36:05.482644+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:22:57.143570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.483031+0000","last_change":"2023-10-14T10:36:03.865280+0000","last_active":"2023-10-14T10:36:05.483031+0000","last_peered":"2023-10-14T10:36:05.483031+0000","last_clean":"2023-10-14T10:36:05.483031+0000","last_became_active":"2023-10-14T10:36:03.864926+0000","last_became_peered":"2023-10-14T10:36:03.864926+0000","last_unstale":"2023-10-14T10:36:05.483031+0000","last_undegraded":"2023-10-14T10:36:05.483031+0000","last_fullsized":"2023-10-14T10:36:05.483031+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:05:52.086793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.845177+0000","last_change":"2023-10-14T10:36:02.843680+0000","last_active":"2023-10-14T10:36:04.845177+0000","last_peered":"2023-10-14T10:36:04.845177+0000","last_clean":"2023-10-14T10:36:04.845177+0000","last_became_active":"2023-10-14T10:36:02.843364+0000","last_became_peered":"2023-10-14T10:36:02.843364+0000","last_unstale":"2023-10-14T10:36:04.845177+0000","last_undegraded":"2023-10-14T10:36:04.845177+0000","last_fullsized":"2023-10-14T10:36:04.845177+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T10:59:16.502697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.841021+0000","last_change":"2023-10-14T10:36:03.862822+0000","last_active":"2023-10-14T10:36:04.841021+0000","last_peered":"2023-10-14T10:36:04.841021+0000","last_clean":"2023-10-14T10:36:04.841021+0000","last_became_active":"2023-10-14T10:36:03.862547+0000","last_became_peered":"2023-10-14T10:36:03.862547+0000","last_unstale":"2023-10-14T10:36:04.841021+0000","last_undegraded":"2023-10-14T10:36:04.841021+0000","last_fullsized":"2023-10-14T10:36:04.841021+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T18:42:51.749474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-14T10:36:03.851585+0000","last_change":"2023-10-14T10:36:02.846160+0000","last_active":"2023-10-14T10:36:03.851585+0000","last_peered":"2023-10-14T10:36:03.851585+0000","last_clean":"2023-10-14T10:36:03.851585+0000","last_became_active":"2023-10-14T10:36:02.845784+0000","last_became_peered":"2023-10-14T10:36:02.845784+0000","last_unstale":"2023-10-14T10:36:03.851585+0000","last_undegraded":"2023-10-14T10:36:03.851585+0000","last_fullsized":"2023-10-14T10:36:03.851585+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T12:29:01.996039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.845511+0000","last_change":"2023-10-14T10:36:03.864605+0000","last_active":"2023-10-14T10:36:04.845511+0000","last_peered":"2023-10-14T10:36:04.845511+0000","last_clean":"2023-10-14T10:36:04.845511+0000","last_became_active":"2023-10-14T10:36:03.864160+0000","last_became_peered":"2023-10-14T10:36:03.864160+0000","last_unstale":"2023-10-14T10:36:04.845511+0000","last_undegraded":"2023-10-14T10:36:04.845511+0000","last_fullsized":"2023-10-14T10:36:04.845511+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T20:52:07.681036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.845294+0000","last_change":"2023-10-14T10:36:02.845777+0000","last_active":"2023-10-14T10:36:04.845294+0000","last_peered":"2023-10-14T10:36:04.845294+0000","last_clean":"2023-10-14T10:36:04.845294+0000","last_became_active":"2023-10-14T10:36:02.845482+0000","last_became_peered":"2023-10-14T10:36:02.845482+0000","last_unstale":"2023-10-14T10:36:04.845294+0000","last_undegraded":"2023-10-14T10:36:04.845294+0000","last_fullsized":"2023-10-14T10:36:04.845294+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T14:04:44.127519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.483717+0000","last_change":"2023-10-14T10:36:03.861072+0000","last_active":"2023-10-14T10:36:05.483717+0000","last_peered":"2023-10-14T10:36:05.483717+0000","last_clean":"2023-10-14T10:36:05.483717+0000","last_became_active":"2023-10-14T10:36:03.860335+0000","last_became_peered":"2023-10-14T10:36:03.860335+0000","last_unstale":"2023-10-14T10:36:05.483717+0000","last_undegraded":"2023-10-14T10:36:05.483717+0000","last_fullsized":"2023-10-14T10:36:05.483717+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T12:18:09.680616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.841103+0000","last_change":"2023-10-14T10:36:02.845951+0000","last_active":"2023-10-14T10:36:04.841103+0000","last_peered":"2023-10-14T10:36:04.841103+0000","last_clean":"2023-10-14T10:36:04.841103+0000","last_became_active":"2023-10-14T10:36:02.845667+0000","last_became_peered":"2023-10-14T10:36:02.845667+0000","last_unstale":"2023-10-14T10:36:04.841103+0000","last_undegraded":"2023-10-14T10:36:04.841103+0000","last_fullsized":"2023-10-14T10:36:04.841103+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:00:17.008189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.482746+0000","last_change":"2023-10-14T10:36:03.871035+0000","last_active":"2023-10-14T10:36:05.482746+0000","last_peered":"2023-10-14T10:36:05.482746+0000","last_clean":"2023-10-14T10:36:05.482746+0000","last_became_active":"2023-10-14T10:36:03.870542+0000","last_became_peered":"2023-10-14T10:36:03.870542+0000","last_unstale":"2023-10-14T10:36:05.482746+0000","last_undegraded":"2023-10-14T10:36:05.482746+0000","last_fullsized":"2023-10-14T10:36:05.482746+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T14:05:05.847145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-14T10:36:03.874236+0000","last_change":"2023-10-14T10:36:02.847268+0000","last_active":"2023-10-14T10:36:03.874236+0000","last_peered":"2023-10-14T10:36:03.874236+0000","last_clean":"2023-10-14T10:36:03.874236+0000","last_became_active":"2023-10-14T10:36:02.846840+0000","last_became_peered":"2023-10-14T10:36:02.846840+0000","last_unstale":"2023-10-14T10:36:03.874236+0000","last_undegraded":"2023-10-14T10:36:03.874236+0000","last_fullsized":"2023-10-14T10:36:03.874236+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:22:57.143570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.847057+0000","last_change":"2023-10-14T10:36:03.864955+0000","last_active":"2023-10-14T10:36:04.847057+0000","last_peered":"2023-10-14T10:36:04.847057+0000","last_clean":"2023-10-14T10:36:04.847057+0000","last_became_active":"2023-10-14T10:36:03.864505+0000","last_became_peered":"2023-10-14T10:36:03.864505+0000","last_unstale":"2023-10-14T10:36:04.847057+0000","last_undegraded":"2023-10-14T10:36:04.847057+0000","last_fullsized":"2023-10-14T10:36:04.847057+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:00:36.181878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.483879+0000","last_change":"2023-10-14T10:36:02.856271+0000","last_active":"2023-10-14T10:36:05.483879+0000","last_peered":"2023-10-14T10:36:05.483879+0000","last_clean":"2023-10-14T10:36:05.483879+0000","last_became_active":"2023-10-14T10:36:02.856039+0000","last_became_peered":"2023-10-14T10:36:02.856039+0000","last_unstale":"2023-10-14T10:36:05.483879+0000","last_undegraded":"2023-10-14T10:36:05.483879+0000","last_fullsized":"2023-10-14T10:36:05.483879+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:29:50.590931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.484124+0000","last_change":"2023-10-14T10:36:03.868295+0000","last_active":"2023-10-14T10:36:05.484124+0000","last_peered":"2023-10-14T10:36:05.484124+0000","last_clean":"2023-10-14T10:36:05.484124+0000","last_became_active":"2023-10-14T10:36:03.867969+0000","last_became_peered":"2023-10-14T10:36:03.867969+0000","last_unstale":"2023-10-14T10:36:05.484124+0000","last_undegraded":"2023-10-14T10:36:05.484124+0000","last_fullsized":"2023-10-14T10:36:05.484124+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:05:52.086793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.845748+0000","last_change":"2023-10-14T10:36:02.845900+0000","last_active":"2023-10-14T10:36:04.845748+0000","last_peered":"2023-10-14T10:36:04.845748+0000","last_clean":"2023-10-14T10:36:04.845748+0000","last_became_active":"2023-10-14T10:36:02.845596+0000","last_became_peered":"2023-10-14T10:36:02.845596+0000","last_unstale":"2023-10-14T10:36:04.845748+0000","last_undegraded":"2023-10-14T10:36:04.845748+0000","last_fullsized":"2023-10-14T10:36:04.845748+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T21:40:42.359224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.846941+0000","last_change":"2023-10-14T10:36:03.870846+0000","last_active":"2023-10-14T10:36:04.846941+0000","last_peered":"2023-10-14T10:36:04.846941+0000","last_clean":"2023-10-14T10:36:04.846941+0000","last_became_active":"2023-10-14T10:36:03.869379+0000","last_became_peered":"2023-10-14T10:36:03.869379+0000","last_unstale":"2023-10-14T10:36:04.846941+0000","last_undegraded":"2023-10-14T10:36:04.846941+0000","last_fullsized":"2023-10-14T10:36:04.846941+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T10:36:37.449003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-14T10:36:03.851688+0000","last_change":"2023-10-14T10:36:02.847080+0000","last_active":"2023-10-14T10:36:03.851688+0000","last_peered":"2023-10-14T10:36:03.851688+0000","last_clean":"2023-10-14T10:36:03.851688+0000","last_became_active":"2023-10-14T10:36:02.846784+0000","last_became_peered":"2023-10-14T10:36:02.846784+0000","last_unstale":"2023-10-14T10:36:03.851688+0000","last_undegraded":"2023-10-14T10:36:03.851688+0000","last_fullsized":"2023-10-14T10:36:03.851688+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:29:50.590931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.482758+0000","last_change":"2023-10-14T10:36:03.862316+0000","last_active":"2023-10-14T10:36:05.482758+0000","last_peered":"2023-10-14T10:36:05.482758+0000","last_clean":"2023-10-14T10:36:05.482758+0000","last_became_active":"2023-10-14T10:36:03.862022+0000","last_became_peered":"2023-10-14T10:36:03.862022+0000","last_unstale":"2023-10-14T10:36:05.482758+0000","last_undegraded":"2023-10-14T10:36:05.482758+0000","last_fullsized":"2023-10-14T10:36:05.482758+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T21:40:43.364841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-14T10:36:03.867586+0000","last_change":"2023-10-14T10:36:02.846811+0000","last_active":"2023-10-14T10:36:03.867586+0000","last_peered":"2023-10-14T10:36:03.867586+0000","last_clean":"2023-10-14T10:36:03.867586+0000","last_became_active":"2023-10-14T10:36:02.846487+0000","last_became_peered":"2023-10-14T10:36:02.846487+0000","last_unstale":"2023-10-14T10:36:03.867586+0000","last_undegraded":"2023-10-14T10:36:03.867586+0000","last_fullsized":"2023-10-14T10:36:03.867586+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T18:47:26.337837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.846699+0000","last_change":"2023-10-14T10:36:03.872480+0000","last_active":"2023-10-14T10:36:04.846699+0000","last_peered":"2023-10-14T10:36:04.846699+0000","last_clean":"2023-10-14T10:36:04.846699+0000","last_became_active":"2023-10-14T10:36:03.872231+0000","last_became_peered":"2023-10-14T10:36:03.872231+0000","last_unstale":"2023-10-14T10:36:04.846699+0000","last_undegraded":"2023-10-14T10:36:04.846699+0000","last_fullsized":"2023-10-14T10:36:04.846699+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:47:42.089047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.482611+0000","last_change":"2023-10-14T10:36:02.851223+0000","last_active":"2023-10-14T10:36:05.482611+0000","last_peered":"2023-10-14T10:36:05.482611+0000","last_clean":"2023-10-14T10:36:05.482611+0000","last_became_active":"2023-10-14T10:36:02.850955+0000","last_became_peered":"2023-10-14T10:36:02.850955+0000","last_unstale":"2023-10-14T10:36:05.482611+0000","last_undegraded":"2023-10-14T10:36:05.482611+0000","last_fullsized":"2023-10-14T10:36:05.482611+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:08:47.187941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.847107+0000","last_change":"2023-10-14T10:36:03.871637+0000","last_active":"2023-10-14T10:36:04.847107+0000","last_peered":"2023-10-14T10:36:04.847107+0000","last_clean":"2023-10-14T10:36:04.847107+0000","last_became_active":"2023-10-14T10:36:03.871362+0000","last_became_peered":"2023-10-14T10:36:03.871362+0000","last_unstale":"2023-10-14T10:36:04.847107+0000","last_undegraded":"2023-10-14T10:36:04.847107+0000","last_fullsized":"2023-10-14T10:36:04.847107+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:09:22.818455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.841156+0000","last_change":"2023-10-14T10:36:02.851439+0000","last_active":"2023-10-14T10:36:04.841156+0000","last_peered":"2023-10-14T10:36:04.841156+0000","last_clean":"2023-10-14T10:36:04.841156+0000","last_became_active":"2023-10-14T10:36:02.851121+0000","last_became_peered":"2023-10-14T10:36:02.851121+0000","last_unstale":"2023-10-14T10:36:04.841156+0000","last_undegraded":"2023-10-14T10:36:04.841156+0000","last_fullsized":"2023-10-14T10:36:04.841156+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T10:42:23.021229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.845753+0000","last_change":"2023-10-14T10:36:03.863345+0000","last_active":"2023-10-14T10:36:04.845753+0000","last_peered":"2023-10-14T10:36:04.845753+0000","last_clean":"2023-10-14T10:36:04.845753+0000","last_became_active":"2023-10-14T10:36:03.861227+0000","last_became_peered":"2023-10-14T10:36:03.861227+0000","last_unstale":"2023-10-14T10:36:04.845753+0000","last_undegraded":"2023-10-14T10:36:04.845753+0000","last_fullsized":"2023-10-14T10:36:04.845753+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:00:36.181878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.845649+0000","last_change":"2023-10-14T10:36:02.846354+0000","last_active":"2023-10-14T10:36:04.845649+0000","last_peered":"2023-10-14T10:36:04.845649+0000","last_clean":"2023-10-14T10:36:04.845649+0000","last_became_active":"2023-10-14T10:36:02.845950+0000","last_became_peered":"2023-10-14T10:36:02.845950+0000","last_unstale":"2023-10-14T10:36:04.845649+0000","last_undegraded":"2023-10-14T10:36:04.845649+0000","last_fullsized":"2023-10-14T10:36:04.845649+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T17:28:50.330745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.482785+0000","last_change":"2023-10-14T10:36:03.860805+0000","last_active":"2023-10-14T10:36:05.482785+0000","last_peered":"2023-10-14T10:36:05.482785+0000","last_clean":"2023-10-14T10:36:05.482785+0000","last_became_active":"2023-10-14T10:36:03.860218+0000","last_became_peered":"2023-10-14T10:36:03.860218+0000","last_unstale":"2023-10-14T10:36:05.482785+0000","last_undegraded":"2023-10-14T10:36:05.482785+0000","last_fullsized":"2023-10-14T10:36:05.482785+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T12:29:03.001656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.840832+0000","last_change":"2023-10-14T10:36:03.857512+0000","last_active":"2023-10-14T10:36:04.840832+0000","last_peered":"2023-10-14T10:36:04.840832+0000","last_clean":"2023-10-14T10:36:04.840832+0000","last_became_active":"2023-10-14T10:36:03.855687+0000","last_became_peered":"2023-10-14T10:36:03.855687+0000","last_unstale":"2023-10-14T10:36:04.840832+0000","last_undegraded":"2023-10-14T10:36:04.840832+0000","last_fullsized":"2023-10-14T10:36:04.840832+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T17:28:51.336361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-14T10:36:03.843979+0000","last_change":"2023-10-14T10:36:02.843620+0000","last_active":"2023-10-14T10:36:03.843979+0000","last_peered":"2023-10-14T10:36:03.843979+0000","last_clean":"2023-10-14T10:36:03.843979+0000","last_became_active":"2023-10-14T10:36:02.843324+0000","last_became_peered":"2023-10-14T10:36:02.843324+0000","last_unstale":"2023-10-14T10:36:03.843979+0000","last_undegraded":"2023-10-14T10:36:03.843979+0000","last_fullsized":"2023-10-14T10:36:03.843979+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T14:12:49.629312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.482850+0000","last_change":"2023-10-14T10:36:03.858691+0000","last_active":"2023-10-14T10:36:05.482850+0000","last_peered":"2023-10-14T10:36:05.482850+0000","last_clean":"2023-10-14T10:36:05.482850+0000","last_became_active":"2023-10-14T10:36:03.858420+0000","last_became_peered":"2023-10-14T10:36:03.858420+0000","last_unstale":"2023-10-14T10:36:05.482850+0000","last_undegraded":"2023-10-14T10:36:05.482850+0000","last_fullsized":"2023-10-14T10:36:05.482850+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T18:42:51.749474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.845260+0000","last_change":"2023-10-14T10:36:02.840533+0000","last_active":"2023-10-14T10:36:04.845260+0000","last_peered":"2023-10-14T10:36:04.845260+0000","last_clean":"2023-10-14T10:36:04.845260+0000","last_became_active":"2023-10-14T10:36:02.839919+0000","last_became_peered":"2023-10-14T10:36:02.839919+0000","last_unstale":"2023-10-14T10:36:04.845260+0000","last_undegraded":"2023-10-14T10:36:04.845260+0000","last_fullsized":"2023-10-14T10:36:04.845260+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:29:50.590931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":165,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-14T10:36:03.847417+0000","last_change":"2023-10-14T10:35:59.770163+0000","last_active":"2023-10-14T10:36:03.847417+0000","last_peered":"2023-10-14T10:36:03.847417+0000","last_clean":"2023-10-14T10:36:03.847417+0000","last_became_active":"2023-10-14T10:35:59.769199+0000","last_became_peered":"2023-10-14T10:35:59.769199+0000","last_unstale":"2023-10-14T10:36:03.847417+0000","last_undegraded":"2023-10-14T10:36:03.847417+0000","last_fullsized":"2023-10-14T10:36:03.847417+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:35:58.675572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:35:58.675572+0000","last_clean_scrub_stamp":"2023-10-14T10:35:58.675572+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:03:25.281187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.484102+0000","last_change":"2023-10-14T10:36:03.872244+0000","last_active":"2023-10-14T10:36:05.484102+0000","last_peered":"2023-10-14T10:36:05.484102+0000","last_clean":"2023-10-14T10:36:05.484102+0000","last_became_active":"2023-10-14T10:36:03.871841+0000","last_became_peered":"2023-10-14T10:36:03.871841+0000","last_unstale":"2023-10-14T10:36:05.484102+0000","last_undegraded":"2023-10-14T10:36:05.484102+0000","last_fullsized":"2023-10-14T10:36:05.484102+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:07:32.070142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.846600+0000","last_change":"2023-10-14T10:36:02.854777+0000","last_active":"2023-10-14T10:36:04.846600+0000","last_peered":"2023-10-14T10:36:04.846600+0000","last_clean":"2023-10-14T10:36:04.846600+0000","last_became_active":"2023-10-14T10:36:02.854126+0000","last_became_peered":"2023-10-14T10:36:02.854126+0000","last_unstale":"2023-10-14T10:36:04.846600+0000","last_undegraded":"2023-10-14T10:36:04.846600+0000","last_fullsized":"2023-10-14T10:36:04.846600+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:22:57.143570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.847151+0000","last_change":"2023-10-14T10:36:02.853800+0000","last_active":"2023-10-14T10:36:04.847151+0000","last_peered":"2023-10-14T10:36:04.847151+0000","last_clean":"2023-10-14T10:36:04.847151+0000","last_became_active":"2023-10-14T10:36:02.853508+0000","last_became_peered":"2023-10-14T10:36:02.853508+0000","last_unstale":"2023-10-14T10:36:04.847151+0000","last_undegraded":"2023-10-14T10:36:04.847151+0000","last_fullsized":"2023-10-14T10:36:04.847151+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:05:51.081176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.841103+0000","last_change":"2023-10-14T10:36:03.856928+0000","last_active":"2023-10-14T10:36:04.841103+0000","last_peered":"2023-10-14T10:36:04.841103+0000","last_clean":"2023-10-14T10:36:04.841103+0000","last_became_active":"2023-10-14T10:36:03.856616+0000","last_became_peered":"2023-10-14T10:36:03.856616+0000","last_unstale":"2023-10-14T10:36:04.841103+0000","last_undegraded":"2023-10-14T10:36:04.841103+0000","last_fullsized":"2023-10-14T10:36:04.841103+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:07:32.070142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:06.814986+0000","last_change":"2023-10-14T10:36:02.847787+0000","last_active":"2023-10-14T10:36:06.814986+0000","last_peered":"2023-10-14T10:36:06.814986+0000","last_clean":"2023-10-14T10:36:06.814986+0000","last_became_active":"2023-10-14T10:36:02.847468+0000","last_became_peered":"2023-10-14T10:36:02.847468+0000","last_unstale":"2023-10-14T10:36:06.814986+0000","last_undegraded":"2023-10-14T10:36:06.814986+0000","last_fullsized":"2023-10-14T10:36:06.814986+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T18:50:25.620127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.484430+0000","last_change":"2023-10-14T10:36:03.869759+0000","last_active":"2023-10-14T10:36:05.484430+0000","last_peered":"2023-10-14T10:36:05.484430+0000","last_clean":"2023-10-14T10:36:05.484430+0000","last_became_active":"2023-10-14T10:36:03.869404+0000","last_became_peered":"2023-10-14T10:36:03.869404+0000","last_unstale":"2023-10-14T10:36:05.484430+0000","last_undegraded":"2023-10-14T10:36:05.484430+0000","last_fullsized":"2023-10-14T10:36:05.484430+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:47:42.089047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-14T10:36:03.868372+0000","last_change":"2023-10-14T10:36:02.853796+0000","last_active":"2023-10-14T10:36:03.868372+0000","last_peered":"2023-10-14T10:36:03.868372+0000","last_clean":"2023-10-14T10:36:03.868372+0000","last_became_active":"2023-10-14T10:36:02.853468+0000","last_became_peered":"2023-10-14T10:36:02.853468+0000","last_unstale":"2023-10-14T10:36:03.868372+0000","last_undegraded":"2023-10-14T10:36:03.868372+0000","last_fullsized":"2023-10-14T10:36:03.868372+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:14:10.501109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.483634+0000","last_change":"2023-10-14T10:36:03.854935+0000","last_active":"2023-10-14T10:36:05.483634+0000","last_peered":"2023-10-14T10:36:05.483634+0000","last_clean":"2023-10-14T10:36:05.483634+0000","last_became_active":"2023-10-14T10:36:03.854648+0000","last_became_peered":"2023-10-14T10:36:03.854648+0000","last_unstale":"2023-10-14T10:36:05.483634+0000","last_undegraded":"2023-10-14T10:36:05.483634+0000","last_fullsized":"2023-10-14T10:36:05.483634+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T10:42:24.026846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.846475+0000","last_change":"2023-10-14T10:36:02.854850+0000","last_active":"2023-10-14T10:36:04.846475+0000","last_peered":"2023-10-14T10:36:04.846475+0000","last_clean":"2023-10-14T10:36:04.846475+0000","last_became_active":"2023-10-14T10:36:02.854442+0000","last_became_peered":"2023-10-14T10:36:02.854442+0000","last_unstale":"2023-10-14T10:36:04.846475+0000","last_undegraded":"2023-10-14T10:36:04.846475+0000","last_fullsized":"2023-10-14T10:36:04.846475+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T12:29:01.996039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.482660+0000","last_change":"2023-10-14T10:36:03.859500+0000","last_active":"2023-10-14T10:36:05.482660+0000","last_peered":"2023-10-14T10:36:05.482660+0000","last_clean":"2023-10-14T10:36:05.482660+0000","last_became_active":"2023-10-14T10:36:03.859247+0000","last_became_peered":"2023-10-14T10:36:03.859247+0000","last_unstale":"2023-10-14T10:36:05.482660+0000","last_undegraded":"2023-10-14T10:36:05.482660+0000","last_fullsized":"2023-10-14T10:36:05.482660+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:14:11.506725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.840826+0000","last_change":"2023-10-14T10:36:02.854036+0000","last_active":"2023-10-14T10:36:04.840826+0000","last_peered":"2023-10-14T10:36:04.840826+0000","last_clean":"2023-10-14T10:36:04.840826+0000","last_became_active":"2023-10-14T10:36:02.853723+0000","last_became_peered":"2023-10-14T10:36:02.853723+0000","last_unstale":"2023-10-14T10:36:04.840826+0000","last_undegraded":"2023-10-14T10:36:04.840826+0000","last_fullsized":"2023-10-14T10:36:04.840826+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:29:50.590931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.482772+0000","last_change":"2023-10-14T10:36:03.856047+0000","last_active":"2023-10-14T10:36:05.482772+0000","last_peered":"2023-10-14T10:36:05.482772+0000","last_clean":"2023-10-14T10:36:05.482772+0000","last_became_active":"2023-10-14T10:36:03.855747+0000","last_became_peered":"2023-10-14T10:36:03.855747+0000","last_unstale":"2023-10-14T10:36:05.482772+0000","last_undegraded":"2023-10-14T10:36:05.482772+0000","last_fullsized":"2023-10-14T10:36:05.482772+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T21:40:43.364841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.841238+0000","last_change":"2023-10-14T10:36:02.845255+0000","last_active":"2023-10-14T10:36:04.841238+0000","last_peered":"2023-10-14T10:36:04.841238+0000","last_clean":"2023-10-14T10:36:04.841238+0000","last_became_active":"2023-10-14T10:36:02.844939+0000","last_became_peered":"2023-10-14T10:36:02.844939+0000","last_unstale":"2023-10-14T10:36:04.841238+0000","last_undegraded":"2023-10-14T10:36:04.841238+0000","last_fullsized":"2023-10-14T10:36:04.841238+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T12:29:01.996039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.482888+0000","last_change":"2023-10-14T10:36:03.865197+0000","last_active":"2023-10-14T10:36:05.482888+0000","last_peered":"2023-10-14T10:36:05.482888+0000","last_clean":"2023-10-14T10:36:05.482888+0000","last_became_active":"2023-10-14T10:36:03.864885+0000","last_became_peered":"2023-10-14T10:36:03.864885+0000","last_unstale":"2023-10-14T10:36:05.482888+0000","last_undegraded":"2023-10-14T10:36:05.482888+0000","last_fullsized":"2023-10-14T10:36:05.482888+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:07:32.070142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.847061+0000","last_change":"2023-10-14T10:36:02.853880+0000","last_active":"2023-10-14T10:36:04.847061+0000","last_peered":"2023-10-14T10:36:04.847061+0000","last_clean":"2023-10-14T10:36:04.847061+0000","last_became_active":"2023-10-14T10:36:02.853574+0000","last_became_peered":"2023-10-14T10:36:02.853574+0000","last_unstale":"2023-10-14T10:36:04.847061+0000","last_undegraded":"2023-10-14T10:36:04.847061+0000","last_fullsized":"2023-10-14T10:36:04.847061+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T10:59:16.502697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.482245+0000","last_change":"2023-10-14T10:36:03.865872+0000","last_active":"2023-10-14T10:36:05.482245+0000","last_peered":"2023-10-14T10:36:05.482245+0000","last_clean":"2023-10-14T10:36:05.482245+0000","last_became_active":"2023-10-14T10:36:03.865624+0000","last_became_peered":"2023-10-14T10:36:03.865624+0000","last_unstale":"2023-10-14T10:36:05.482245+0000","last_undegraded":"2023-10-14T10:36:05.482245+0000","last_fullsized":"2023-10-14T10:36:05.482245+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T18:50:26.625744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-14T10:36:03.842738+0000","last_change":"2023-10-14T10:36:02.840500+0000","last_active":"2023-10-14T10:36:03.842738+0000","last_peered":"2023-10-14T10:36:03.842738+0000","last_clean":"2023-10-14T10:36:03.842738+0000","last_became_active":"2023-10-14T10:36:02.839996+0000","last_became_peered":"2023-10-14T10:36:02.839996+0000","last_unstale":"2023-10-14T10:36:03.842738+0000","last_undegraded":"2023-10-14T10:36:03.842738+0000","last_fullsized":"2023-10-14T10:36:03.842738+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T21:06:31.726029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.847199+0000","last_change":"2023-10-14T10:36:03.868941+0000","last_active":"2023-10-14T10:36:04.847199+0000","last_peered":"2023-10-14T10:36:04.847199+0000","last_clean":"2023-10-14T10:36:04.847199+0000","last_became_active":"2023-10-14T10:36:03.868609+0000","last_became_peered":"2023-10-14T10:36:03.868609+0000","last_unstale":"2023-10-14T10:36:04.847199+0000","last_undegraded":"2023-10-14T10:36:04.847199+0000","last_fullsized":"2023-10-14T10:36:04.847199+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:14:41.057137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.841048+0000","last_change":"2023-10-14T10:36:02.848004+0000","last_active":"2023-10-14T10:36:04.841048+0000","last_peered":"2023-10-14T10:36:04.841048+0000","last_clean":"2023-10-14T10:36:04.841048+0000","last_became_active":"2023-10-14T10:36:02.847494+0000","last_became_peered":"2023-10-14T10:36:02.847494+0000","last_unstale":"2023-10-14T10:36:04.841048+0000","last_undegraded":"2023-10-14T10:36:04.841048+0000","last_fullsized":"2023-10-14T10:36:04.841048+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T14:12:52.738066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.845506+0000","last_change":"2023-10-14T10:36:03.863211+0000","last_active":"2023-10-14T10:36:04.845506+0000","last_peered":"2023-10-14T10:36:04.845506+0000","last_clean":"2023-10-14T10:36:04.845506+0000","last_became_active":"2023-10-14T10:36:03.862894+0000","last_became_peered":"2023-10-14T10:36:03.862894+0000","last_unstale":"2023-10-14T10:36:04.845506+0000","last_undegraded":"2023-10-14T10:36:04.845506+0000","last_fullsized":"2023-10-14T10:36:04.845506+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:27:25.021169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-14T10:36:03.864535+0000","last_change":"2023-10-14T10:36:02.842899+0000","last_active":"2023-10-14T10:36:03.864535+0000","last_peered":"2023-10-14T10:36:03.864535+0000","last_clean":"2023-10-14T10:36:03.864535+0000","last_became_active":"2023-10-14T10:36:02.842620+0000","last_became_peered":"2023-10-14T10:36:02.842620+0000","last_unstale":"2023-10-14T10:36:03.864535+0000","last_undegraded":"2023-10-14T10:36:03.864535+0000","last_fullsized":"2023-10-14T10:36:03.864535+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:01.818489+0000","last_clean_scrub_stamp":"2023-10-14T10:36:01.818489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T12:44:16.134070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.847179+0000","last_change":"2023-10-14T10:36:03.872451+0000","last_active":"2023-10-14T10:36:04.847179+0000","last_peered":"2023-10-14T10:36:04.847179+0000","last_clean":"2023-10-14T10:36:04.847179+0000","last_became_active":"2023-10-14T10:36:03.872212+0000","last_became_peered":"2023-10-14T10:36:03.872212+0000","last_unstale":"2023-10-14T10:36:04.847179+0000","last_undegraded":"2023-10-14T10:36:04.847179+0000","last_fullsized":"2023-10-14T10:36:04.847179+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T12:31:06.179885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.482245+0000","last_change":"2023-10-14T10:36:03.862676+0000","last_active":"2023-10-14T10:36:05.482245+0000","last_peered":"2023-10-14T10:36:05.482245+0000","last_clean":"2023-10-14T10:36:05.482245+0000","last_became_active":"2023-10-14T10:36:03.862364+0000","last_became_peered":"2023-10-14T10:36:03.862364+0000","last_unstale":"2023-10-14T10:36:05.482245+0000","last_undegraded":"2023-10-14T10:36:05.482245+0000","last_fullsized":"2023-10-14T10:36:05.482245+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T10:42:24.026846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.483021+0000","last_change":"2023-10-14T10:36:03.856448+0000","last_active":"2023-10-14T10:36:05.483021+0000","last_peered":"2023-10-14T10:36:05.483021+0000","last_clean":"2023-10-14T10:36:05.483021+0000","last_became_active":"2023-10-14T10:36:03.856165+0000","last_became_peered":"2023-10-14T10:36:03.856165+0000","last_unstale":"2023-10-14T10:36:05.483021+0000","last_undegraded":"2023-10-14T10:36:05.483021+0000","last_fullsized":"2023-10-14T10:36:05.483021+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:21:06.827004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.845673+0000","last_change":"2023-10-14T10:36:03.862566+0000","last_active":"2023-10-14T10:36:04.845673+0000","last_peered":"2023-10-14T10:36:04.845673+0000","last_clean":"2023-10-14T10:36:04.845673+0000","last_became_active":"2023-10-14T10:36:03.862028+0000","last_became_peered":"2023-10-14T10:36:03.862028+0000","last_unstale":"2023-10-14T10:36:04.845673+0000","last_undegraded":"2023-10-14T10:36:04.845673+0000","last_fullsized":"2023-10-14T10:36:04.845673+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T10:36:37.449003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.483846+0000","last_change":"2023-10-14T10:36:03.865325+0000","last_active":"2023-10-14T10:36:05.483846+0000","last_peered":"2023-10-14T10:36:05.483846+0000","last_clean":"2023-10-14T10:36:05.483846+0000","last_became_active":"2023-10-14T10:36:03.865034+0000","last_became_peered":"2023-10-14T10:36:03.865034+0000","last_unstale":"2023-10-14T10:36:05.483846+0000","last_undegraded":"2023-10-14T10:36:05.483846+0000","last_fullsized":"2023-10-14T10:36:05.483846+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T21:40:43.364841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.483268+0000","last_change":"2023-10-14T10:36:03.857566+0000","last_active":"2023-10-14T10:36:05.483268+0000","last_peered":"2023-10-14T10:36:05.483268+0000","last_clean":"2023-10-14T10:36:05.483268+0000","last_became_active":"2023-10-14T10:36:03.857301+0000","last_became_peered":"2023-10-14T10:36:03.857301+0000","last_unstale":"2023-10-14T10:36:05.483268+0000","last_undegraded":"2023-10-14T10:36:05.483268+0000","last_fullsized":"2023-10-14T10:36:05.483268+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T18:49:18.106622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.841002+0000","last_change":"2023-10-14T10:36:03.863392+0000","last_active":"2023-10-14T10:36:04.841002+0000","last_peered":"2023-10-14T10:36:04.841002+0000","last_clean":"2023-10-14T10:36:04.841002+0000","last_became_active":"2023-10-14T10:36:03.863097+0000","last_became_peered":"2023-10-14T10:36:03.863097+0000","last_unstale":"2023-10-14T10:36:04.841002+0000","last_undegraded":"2023-10-14T10:36:04.841002+0000","last_fullsized":"2023-10-14T10:36:04.841002+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:05:52.086793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.482815+0000","last_change":"2023-10-14T10:36:03.864988+0000","last_active":"2023-10-14T10:36:05.482815+0000","last_peered":"2023-10-14T10:36:05.482815+0000","last_clean":"2023-10-14T10:36:05.482815+0000","last_became_active":"2023-10-14T10:36:03.864699+0000","last_became_peered":"2023-10-14T10:36:03.864699+0000","last_unstale":"2023-10-14T10:36:05.482815+0000","last_undegraded":"2023-10-14T10:36:05.482815+0000","last_fullsized":"2023-10-14T10:36:05.482815+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T18:42:51.749474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.484472+0000","last_change":"2023-10-14T10:36:03.855927+0000","last_active":"2023-10-14T10:36:05.484472+0000","last_peered":"2023-10-14T10:36:05.484472+0000","last_clean":"2023-10-14T10:36:05.484472+0000","last_became_active":"2023-10-14T10:36:03.855643+0000","last_became_peered":"2023-10-14T10:36:03.855643+0000","last_unstale":"2023-10-14T10:36:05.484472+0000","last_undegraded":"2023-10-14T10:36:05.484472+0000","last_fullsized":"2023-10-14T10:36:05.484472+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T20:52:07.681036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.845305+0000","last_change":"2023-10-14T10:36:03.864752+0000","last_active":"2023-10-14T10:36:04.845305+0000","last_peered":"2023-10-14T10:36:04.845305+0000","last_clean":"2023-10-14T10:36:04.845305+0000","last_became_active":"2023-10-14T10:36:03.864480+0000","last_became_peered":"2023-10-14T10:36:03.864480+0000","last_unstale":"2023-10-14T10:36:04.845305+0000","last_undegraded":"2023-10-14T10:36:04.845305+0000","last_fullsized":"2023-10-14T10:36:04.845305+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:05:52.086793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.482676+0000","last_change":"2023-10-14T10:36:03.858250+0000","last_active":"2023-10-14T10:36:05.482676+0000","last_peered":"2023-10-14T10:36:05.482676+0000","last_clean":"2023-10-14T10:36:05.482676+0000","last_became_active":"2023-10-14T10:36:03.857940+0000","last_became_peered":"2023-10-14T10:36:03.857940+0000","last_unstale":"2023-10-14T10:36:05.482676+0000","last_undegraded":"2023-10-14T10:36:05.482676+0000","last_fullsized":"2023-10-14T10:36:05.482676+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T17:28:51.336361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:04.845474+0000","last_change":"2023-10-14T10:36:03.866904+0000","last_active":"2023-10-14T10:36:04.845474+0000","last_peered":"2023-10-14T10:36:04.845474+0000","last_clean":"2023-10-14T10:36:04.845474+0000","last_became_active":"2023-10-14T10:36:03.866439+0000","last_became_peered":"2023-10-14T10:36:03.866439+0000","last_unstale":"2023-10-14T10:36:04.845474+0000","last_undegraded":"2023-10-14T10:36:04.845474+0000","last_fullsized":"2023-10-14T10:36:04.845474+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T18:49:18.106622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.483685+0000","last_change":"2023-10-14T10:36:03.868286+0000","last_active":"2023-10-14T10:36:05.483685+0000","last_peered":"2023-10-14T10:36:05.483685+0000","last_clean":"2023-10-14T10:36:05.483685+0000","last_became_active":"2023-10-14T10:36:03.868050+0000","last_became_peered":"2023-10-14T10:36:03.868050+0000","last_unstale":"2023-10-14T10:36:05.483685+0000","last_undegraded":"2023-10-14T10:36:05.483685+0000","last_fullsized":"2023-10-14T10:36:05.483685+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T18:50:26.625744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-14T10:36:05.482746+0000","last_change":"2023-10-14T10:36:03.872106+0000","last_active":"2023-10-14T10:36:05.482746+0000","last_peered":"2023-10-14T10:36:05.482746+0000","last_clean":"2023-10-14T10:36:05.482746+0000","last_became_active":"2023-10-14T10:36:03.871788+0000","last_became_peered":"2023-10-14T10:36:03.871788+0000","last_unstale":"2023-10-14T10:36:05.482746+0000","last_undegraded":"2023-10-14T10:36:05.482746+0000","last_fullsized":"2023-10-14T10:36:05.482746+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:02.824105+0000","last_clean_scrub_stamp":"2023-10-14T10:36:02.824105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T17:28:51.336361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542150,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":823296,"data_stored":669814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45500000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.376}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]}]},{"osd":6,"up_from":14,"seq":60129542151,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":237568,"data_stored":80476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63800000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66800000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.372}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51400000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47199999999999998}]}]},{"osd":1,"up_from":14,"seq":60129542150,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":233472,"data_stored":79024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32900000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27600000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60199999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.629}]}]},{"osd":0,"up_from":14,"seq":60129542149,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27832,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344008,"statfs":{"total":96636764160,"available":96608264192,"internally_reserved":0,"allocated":831488,"data_stored":669870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40500000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35399999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60399999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65400000000000003}]}]},{"osd":2,"up_from":14,"seq":60129542150,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":225280,"data_stored":78864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.429}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5269999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1890000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4950000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.099}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0580000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98099999999999998}]}]},{"osd":3,"up_from":14,"seq":60129542150,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":237568,"data_stored":80476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.311}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46600000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47699999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542149,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":237568,"data_stored":78990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79300000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.872}]}]},{"osd":5,"up_from":14,"seq":60129542150,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":78954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56399999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66200000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.436}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41499999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-14T10:36:13.393 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-14T10:36:13.393 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-10-14T10:36:13.394 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-10-14T10:36:13.394 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-14T10:36:13.551 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.550+0000 7fd3a2ca5700 1 Processor -- start 2023-10-14T10:36:13.552 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.550+0000 7fd3a2ca5700 1 -- start start 2023-10-14T10:36:13.553 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.551+0000 7fd3a2ca5700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd39c06af50 0x7fd39c168ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:13.554 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.551+0000 7fd3a2ca5700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd39c06d7e0 con 0x7fd39c0af800 2023-10-14T10:36:13.554 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.551+0000 7fd3a2ca5700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd39c06da80 con 0x7fd39c0afbe0 2023-10-14T10:36:13.554 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.551+0000 7fd3a2ca5700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd39c06e420 con 0x7fd39c06af50 2023-10-14T10:36:13.554 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.551+0000 7fd3a1242700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd39c06af50 0x7fd39c168ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:13.555 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.551+0000 7fd3a0a41700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fd39c0af800 0x7fd39c06edd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:52110/0 (socket says 172.21.15.67:52110) 2023-10-14T10:36:13.555 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.551+0000 7fd3a0a41700 1 -- 172.21.15.67:0/1452921131 learned_addr learned my addr 172.21.15.67:0/1452921131 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:36:13.555 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.551+0000 7fd3a1242700 1 -- 172.21.15.67:0/1452921131 >> v1:172.21.15.72:6789/0 conn(0x7fd39c0afbe0 legacy=0x7fd39c06a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:13.555 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.551+0000 7fd3a1242700 1 -- 172.21.15.67:0/1452921131 >> v1:172.21.15.67:6789/0 conn(0x7fd39c0af800 legacy=0x7fd39c06edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:13.556 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.551+0000 7fd3a1242700 1 -- 172.21.15.67:0/1452921131 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd39c06dd60 con 0x7fd39c06af50 2023-10-14T10:36:13.556 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.551+0000 7fd39b7fe700 1 -- 172.21.15.67:0/1452921131 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1474792224 0 0) 0x7fd39c06d7e0 con 0x7fd39c0af800 2023-10-14T10:36:13.556 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.552+0000 7fd3a1242700 1 --2- 172.21.15.67:0/1452921131 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd39c06af50 0x7fd39c168ad0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd394009d90 tx=0x7fd39400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9bca60bf983934d server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:13.556 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.552+0000 7fd39b7fe700 1 -- 172.21.15.67:0/1452921131 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd39400e070 con 0x7fd39c06af50 2023-10-14T10:36:13.556 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.552+0000 7fd39b7fe700 1 -- 172.21.15.67:0/1452921131 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd39400bbd0 con 0x7fd39c06af50 2023-10-14T10:36:13.557 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.552+0000 7fd39b7fe700 1 -- 172.21.15.67:0/1452921131 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd394012430 con 0x7fd39c06af50 2023-10-14T10:36:13.557 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.552+0000 7fd3a2ca5700 1 -- 172.21.15.67:0/1452921131 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd39c06af50 msgr2=0x7fd39c168ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:13.557 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.552+0000 7fd3a2ca5700 1 --2- 172.21.15.67:0/1452921131 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd39c06af50 0x7fd39c168ad0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd394009d90 tx=0x7fd39400b330 comp rx=0 tx=0).stop 2023-10-14T10:36:13.557 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.552+0000 7fd3a2ca5700 1 -- 172.21.15.67:0/1452921131 shutdown_connections 2023-10-14T10:36:13.557 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.552+0000 7fd3a2ca5700 1 --2- 172.21.15.67:0/1452921131 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd39c06af50 0x7fd39c168ad0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:13.558 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.552+0000 7fd3a2ca5700 1 -- 172.21.15.67:0/1452921131 >> 172.21.15.67:0/1452921131 conn(0x7fd39c099a30 msgr2=0x7fd39c0d05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:13.558 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.552+0000 7fd3a2ca5700 1 -- 172.21.15.67:0/1452921131 shutdown_connections 2023-10-14T10:36:13.558 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.552+0000 7fd3a2ca5700 1 -- 172.21.15.67:0/1452921131 wait complete. 2023-10-14T10:36:13.560 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.554+0000 7fd3a2ca5700 1 Processor -- start 2023-10-14T10:36:13.561 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.554+0000 7fd3a2ca5700 1 -- start start 2023-10-14T10:36:13.561 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.554+0000 7fd3a2ca5700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd39c06af50 0x7fd39c0c6c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:13.561 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.554+0000 7fd3a2ca5700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd39c0af800 0x7fd39c0b15f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:13.561 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.555+0000 7fd3a2ca5700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fd39c0afbe0 0x7fd39c0b1b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:13.561 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.555+0000 7fd3a2ca5700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd39c0cad00 con 0x7fd39c06af50 2023-10-14T10:36:13.562 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.555+0000 7fd3a2ca5700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd39c0b3ef0 con 0x7fd39c0af800 2023-10-14T10:36:13.562 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.555+0000 7fd3a2ca5700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd39c0cafa0 con 0x7fd39c0afbe0 2023-10-14T10:36:13.562 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.555+0000 7fd3a0a41700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd39c06af50 0x7fd39c0c6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:13.562 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.555+0000 7fd3a0a41700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd39c06af50 0x7fd39c0c6c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44348/0 (socket says 172.21.15.67:44348) 2023-10-14T10:36:13.562 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.555+0000 7fd3a0a41700 1 -- 172.21.15.67:0/51517755 learned_addr learned my addr 172.21.15.67:0/51517755 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:13.563 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.555+0000 7fd39bfff700 1 --2- 172.21.15.67:0/51517755 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd39c0af800 0x7fd39c0b15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:13.563 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.555+0000 7fd3a1242700 1 --2- 172.21.15.67:0/51517755 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fd39c0afbe0 0x7fd39c0b1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:13.563 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.555+0000 7fd3a0a41700 1 -- 172.21.15.67:0/51517755 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fd39c0afbe0 msgr2=0x7fd39c0b1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:13.563 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.555+0000 7fd3a0a41700 1 --2- 172.21.15.67:0/51517755 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fd39c0afbe0 0x7fd39c0b1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:13.563 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.555+0000 7fd3a0a41700 1 -- 172.21.15.67:0/51517755 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd39c0af800 msgr2=0x7fd39c0b15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:13.564 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.555+0000 7fd3a0a41700 1 --2- 172.21.15.67:0/51517755 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd39c0af800 0x7fd39c0b15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:13.564 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.555+0000 7fd3a0a41700 1 -- 172.21.15.67:0/51517755 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd394009a70 con 0x7fd39c06af50 2023-10-14T10:36:13.564 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.555+0000 7fd3a0a41700 1 --2- 172.21.15.67:0/51517755 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd39c06af50 0x7fd39c0c6c20 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd39000c470 tx=0x7fd39000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:13.564 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.555+0000 7fd3997fa700 1 -- 172.21.15.67:0/51517755 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3900074e0 con 0x7fd39c06af50 2023-10-14T10:36:13.564 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.555+0000 7fd3997fa700 1 -- 172.21.15.67:0/51517755 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd390007640 con 0x7fd39c06af50 2023-10-14T10:36:13.565 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.555+0000 7fd3997fa700 1 -- 172.21.15.67:0/51517755 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd390005a50 con 0x7fd39c06af50 2023-10-14T10:36:13.565 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.555+0000 7fd3a2ca5700 1 -- 172.21.15.67:0/51517755 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd39c0cb280 con 0x7fd39c06af50 2023-10-14T10:36:13.565 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.555+0000 7fd3a2ca5700 1 -- 172.21.15.67:0/51517755 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd39c0b3b80 con 0x7fd39c06af50 2023-10-14T10:36:13.565 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.556+0000 7fd3a2ca5700 1 -- 172.21.15.67:0/51517755 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd39c06e820 con 0x7fd39c06af50 2023-10-14T10:36:13.565 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.557+0000 7fd3997fa700 1 -- 172.21.15.67:0/51517755 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd390005bb0 con 0x7fd39c06af50 2023-10-14T10:36:13.565 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.557+0000 7fd3997fa700 1 --2- 172.21.15.67:0/51517755 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fd38006af00 0x7fd38006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:13.566 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.557+0000 7fd3997fa700 1 -- 172.21.15.67:0/51517755 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd390088a00 con 0x7fd39c06af50 2023-10-14T10:36:13.566 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.557+0000 7fd39bfff700 1 --2- 172.21.15.67:0/51517755 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fd38006af00 0x7fd38006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:13.566 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.558+0000 7fd39bfff700 1 --2- 172.21.15.67:0/51517755 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fd38006af00 0x7fd38006d3c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd39c0b2a00 tx=0x7fd388009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:13.567 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.562+0000 7fd3997fa700 1 -- 172.21.15.67:0/51517755 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd39c06e820 con 0x7fd39c06af50 2023-10-14T10:36:13.809 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.805+0000 7fd3a2ca5700 1 -- 172.21.15.67:0/51517755 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fd39c0cb4c0 con 0x7fd39c06af50 2023-10-14T10:36:13.809 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.806+0000 7fd3997fa700 1 -- 172.21.15.67:0/51517755 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fd39c0cb4c0 con 0x7fd39c06af50 2023-10-14T10:36:13.810 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-14T10:36:13.810 INFO:teuthology.orchestra.run.smithi067.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-10-14T10:36:13.811 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.808+0000 7fd3a2ca5700 1 -- 172.21.15.67:0/51517755 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fd38006af00 msgr2=0x7fd38006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:13.811 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.808+0000 7fd3a2ca5700 1 --2- 172.21.15.67:0/51517755 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fd38006af00 0x7fd38006d3c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd39c0b2a00 tx=0x7fd388009040 comp rx=0 tx=0).stop 2023-10-14T10:36:13.812 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.808+0000 7fd3a2ca5700 1 -- 172.21.15.67:0/51517755 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd39c06af50 msgr2=0x7fd39c0c6c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:13.812 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.808+0000 7fd3a2ca5700 1 --2- 172.21.15.67:0/51517755 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd39c06af50 0x7fd39c0c6c20 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd39000c470 tx=0x7fd39000c830 comp rx=0 tx=0).stop 2023-10-14T10:36:13.812 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.808+0000 7fd3a2ca5700 1 -- 172.21.15.67:0/51517755 shutdown_connections 2023-10-14T10:36:13.812 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.808+0000 7fd3a2ca5700 1 --2- 172.21.15.67:0/51517755 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fd38006af00 0x7fd38006d3c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:13.812 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.808+0000 7fd3a2ca5700 1 --2- 172.21.15.67:0/51517755 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fd39c0afbe0 0x7fd39c0b1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:13.813 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.808+0000 7fd3a2ca5700 1 --2- 172.21.15.67:0/51517755 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd39c06af50 0x7fd39c0c6c20 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:13.813 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.808+0000 7fd3a2ca5700 1 --2- 172.21.15.67:0/51517755 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd39c0af800 0x7fd39c0b15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:13.813 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.808+0000 7fd3a2ca5700 1 -- 172.21.15.67:0/51517755 >> 172.21.15.67:0/51517755 conn(0x7fd39c099a30 msgr2=0x7fd39c06e9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:13.813 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.808+0000 7fd3a2ca5700 1 -- 172.21.15.67:0/51517755 shutdown_connections 2023-10-14T10:36:13.813 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.808+0000 7fd3a2ca5700 1 -- 172.21.15.67:0/51517755 wait complete. 2023-10-14T10:36:13.822 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-10-14T10:36:13.822 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-14T10:36:13.949 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.947+0000 7f53c9b71700 1 Processor -- start 2023-10-14T10:36:13.949 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.947+0000 7f53c9b71700 1 -- start start 2023-10-14T10:36:13.951 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.947+0000 7f53c9b71700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f53c40cf8a0 0x7f53c40d1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:13.952 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.947+0000 7f53c9b71700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53c415ee90 con 0x7f53c415eab0 2023-10-14T10:36:13.952 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.948+0000 7f53c9b71700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53c4158370 con 0x7f53c40d1bc0 2023-10-14T10:36:13.952 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.948+0000 7f53c9b71700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53c40d23b0 con 0x7f53c40cf8a0 2023-10-14T10:36:13.952 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.948+0000 7f53c2ffd700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f53c40cf8a0 0x7f53c40d1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:13.952 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.948+0000 7f53c37fe700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f53c415eab0 0x7f53c40cf190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:52124/0 (socket says 172.21.15.67:52124) 2023-10-14T10:36:13.953 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.948+0000 7f53c37fe700 1 -- 172.21.15.67:0/1405698511 learned_addr learned my addr 172.21.15.67:0/1405698511 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:36:13.953 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.948+0000 7f53c2ffd700 1 -- 172.21.15.67:0/1405698511 >> v1:172.21.15.72:6789/0 conn(0x7f53c40d1bc0 legacy=0x7f53c40cbc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:13.953 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.948+0000 7f53c2ffd700 1 -- 172.21.15.67:0/1405698511 >> v1:172.21.15.67:6789/0 conn(0x7f53c415eab0 legacy=0x7f53c40cf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:13.953 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.948+0000 7f53c27fc700 1 -- 172.21.15.67:0/1405698511 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2106808992 0 0) 0x7f53c415ee90 con 0x7f53c415eab0 2023-10-14T10:36:13.954 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.948+0000 7f53c2ffd700 1 -- 172.21.15.67:0/1405698511 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53c40cda20 con 0x7f53c40cf8a0 2023-10-14T10:36:13.954 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.948+0000 7f53c2ffd700 1 --2- 172.21.15.67:0/1405698511 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f53c40cf8a0 0x7f53c40d1680 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f53b8009840 tx=0x7f53b8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=326853eaf51a39ef server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:13.954 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.948+0000 7f53c27fc700 1 -- 172.21.15.67:0/1405698511 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b8004230 con 0x7f53c40cf8a0 2023-10-14T10:36:13.954 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.948+0000 7f53c27fc700 1 -- 172.21.15.67:0/1405698511 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53b8004390 con 0x7f53c40cf8a0 2023-10-14T10:36:13.954 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.948+0000 7f53c27fc700 1 -- 172.21.15.67:0/1405698511 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b80036a0 con 0x7f53c40cf8a0 2023-10-14T10:36:13.955 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.949+0000 7f53c9b71700 1 -- 172.21.15.67:0/1405698511 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f53c40cf8a0 msgr2=0x7f53c40d1680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:13.955 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.949+0000 7f53c9b71700 1 --2- 172.21.15.67:0/1405698511 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f53c40cf8a0 0x7f53c40d1680 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f53b8009840 tx=0x7f53b8009b50 comp rx=0 tx=0).stop 2023-10-14T10:36:13.955 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.949+0000 7f53c9b71700 1 -- 172.21.15.67:0/1405698511 shutdown_connections 2023-10-14T10:36:13.955 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.949+0000 7f53c9b71700 1 --2- 172.21.15.67:0/1405698511 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f53c40cf8a0 0x7f53c40d1680 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:13.955 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.949+0000 7f53c9b71700 1 -- 172.21.15.67:0/1405698511 >> 172.21.15.67:0/1405698511 conn(0x7f53c40665a0 msgr2=0x7f53c4066980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:13.956 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.949+0000 7f53c9b71700 1 -- 172.21.15.67:0/1405698511 shutdown_connections 2023-10-14T10:36:13.956 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.949+0000 7f53c9b71700 1 -- 172.21.15.67:0/1405698511 wait complete. 2023-10-14T10:36:13.959 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.950+0000 7f53c9b71700 1 Processor -- start 2023-10-14T10:36:13.959 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.950+0000 7f53c9b71700 1 -- start start 2023-10-14T10:36:13.959 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.951+0000 7f53c9b71700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f53c40cf8a0 0x7f53c4155d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:13.959 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.951+0000 7f53c9b71700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f53c40d1bc0 0x7f53c41550e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:13.960 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.951+0000 7f53c9b71700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f53c415eab0 0x7f53c40b2450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:13.960 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.951+0000 7f53c9b71700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53c40b48b0 con 0x7f53c40d1bc0 2023-10-14T10:36:13.960 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.951+0000 7f53c9b71700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53c4155620 con 0x7f53c40cf8a0 2023-10-14T10:36:13.960 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.951+0000 7f53c9b71700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53c40b6300 con 0x7f53c415eab0 2023-10-14T10:36:13.960 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.951+0000 7f53c2ffd700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f53c40d1bc0 0x7f53c41550e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:13.960 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.951+0000 7f53c2ffd700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f53c40d1bc0 0x7f53c41550e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44364/0 (socket says 172.21.15.67:44364) 2023-10-14T10:36:13.961 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.951+0000 7f53c2ffd700 1 -- 172.21.15.67:0/3382114 learned_addr learned my addr 172.21.15.67:0/3382114 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:13.961 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.951+0000 7f53c37fe700 1 --2- 172.21.15.67:0/3382114 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f53c40cf8a0 0x7f53c4155d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:13.961 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.951+0000 7f53c3fff700 1 --2- 172.21.15.67:0/3382114 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f53c415eab0 0x7f53c40b2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:13.961 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.951+0000 7f53c2ffd700 1 -- 172.21.15.67:0/3382114 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f53c415eab0 msgr2=0x7f53c40b2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:13.961 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.951+0000 7f53c2ffd700 1 --2- 172.21.15.67:0/3382114 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f53c415eab0 0x7f53c40b2450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:13.962 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.951+0000 7f53c2ffd700 1 -- 172.21.15.67:0/3382114 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f53c40cf8a0 msgr2=0x7f53c4155d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:13.962 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.951+0000 7f53c2ffd700 1 --2- 172.21.15.67:0/3382114 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f53c40cf8a0 0x7f53c4155d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:13.962 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.952+0000 7f53c2ffd700 1 -- 172.21.15.67:0/3382114 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53b80094a0 con 0x7f53c40d1bc0 2023-10-14T10:36:13.962 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.952+0000 7f53c3fff700 1 --2- 172.21.15.67:0/3382114 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f53c415eab0 0x7f53c40b2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-14T10:36:13.962 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.952+0000 7f53c37fe700 1 --2- 172.21.15.67:0/3382114 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f53c40cf8a0 0x7f53c4155d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-14T10:36:13.963 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.952+0000 7f53c2ffd700 1 --2- 172.21.15.67:0/3382114 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f53c40d1bc0 0x7f53c41550e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f53b80090b0 tx=0x7f53b80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:13.963 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.952+0000 7f53abfff700 1 -- 172.21.15.67:0/3382114 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b8011630 con 0x7f53c40d1bc0 2023-10-14T10:36:13.963 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.952+0000 7f53abfff700 1 -- 172.21.15.67:0/3382114 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53b8011790 con 0x7f53c40d1bc0 2023-10-14T10:36:13.963 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.952+0000 7f53abfff700 1 -- 172.21.15.67:0/3382114 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b8011b40 con 0x7f53c40d1bc0 2023-10-14T10:36:13.963 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.952+0000 7f53c9b71700 1 -- 172.21.15.67:0/3382114 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53c40b65e0 con 0x7f53c40d1bc0 2023-10-14T10:36:13.964 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.952+0000 7f53c9b71700 1 -- 172.21.15.67:0/3382114 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53c40b4540 con 0x7f53c40d1bc0 2023-10-14T10:36:13.964 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.953+0000 7f53c9b71700 1 -- 172.21.15.67:0/3382114 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53c40b7540 con 0x7f53c40d1bc0 2023-10-14T10:36:13.964 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.954+0000 7f53abfff700 1 -- 172.21.15.67:0/3382114 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f53b8011ca0 con 0x7f53c40d1bc0 2023-10-14T10:36:13.964 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.954+0000 7f53abfff700 1 --2- 172.21.15.67:0/3382114 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f53b006af00 0x7f53b006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:13.964 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.954+0000 7f53abfff700 1 -- 172.21.15.67:0/3382114 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f53b808b2e0 con 0x7f53c40d1bc0 2023-10-14T10:36:13.965 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.954+0000 7f53c37fe700 1 --2- 172.21.15.67:0/3382114 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f53b006af00 0x7f53b006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:13.965 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.955+0000 7f53c37fe700 1 --2- 172.21.15.67:0/3382114 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f53b006af00 0x7f53b006d3c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f53b4000f60 tx=0x7f53b4007cc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:13.965 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:13.958+0000 7f53abfff700 1 -- 172.21.15.67:0/3382114 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f53b8056500 con 0x7f53c40d1bc0 2023-10-14T10:36:14.164 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.162+0000 7f53c9b71700 1 -- 172.21.15.67:0/3382114 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f53c40b7540 con 0x7f53c40d1bc0 2023-10-14T10:36:14.165 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.163+0000 7f53abfff700 1 -- 172.21.15.67:0/3382114 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16457 (secure 0 0 0) 0x7f53b8059b20 con 0x7f53c40d1bc0 2023-10-14T10:36:14.165 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-14T10:36:14.166 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":22,"fsid":"37f9ab46-3082-4b71-8b24-08d39897b008","created":"2023-10-14T10:35:46.067554+0000","modified":"2023-10-14T10:36:04.837296+0000","last_up_change":"2023-10-14T10:35:56.668142+0000","last_in_change":"2023-10-14T10:35:48.551490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-14T10:35:57.809566+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-14T10:36:01.667146+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-14T10:36:02.219700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2b59a07e-11ef-440c-b766-9f64ac885838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6801","nonce":1287255499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6803","nonce":1287255499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6807","nonce":1287255499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6805","nonce":1287255499}]},"public_addr":"172.21.15.67:6801/1287255499","cluster_addr":"172.21.15.67:6803/1287255499","heartbeat_back_addr":"172.21.15.67:6807/1287255499","heartbeat_front_addr":"172.21.15.67:6805/1287255499","state":["exists","up"]},{"osd":1,"uuid":"6f813bd6-fa0b-487d-a204-431eb1276a60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6818","nonce":2435538787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6822","nonce":2435538787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6830","nonce":2435538787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6826","nonce":2435538787}]},"public_addr":"172.21.15.67:6818/2435538787","cluster_addr":"172.21.15.67:6822/2435538787","heartbeat_back_addr":"172.21.15.67:6830/2435538787","heartbeat_front_addr":"172.21.15.67:6826/2435538787","state":["exists","up"]},{"osd":2,"uuid":"6af42c6a-1b59-4669-9c04-b6876788446d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6809","nonce":2827491791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6811","nonce":2827491791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6815","nonce":2827491791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6813","nonce":2827491791}]},"public_addr":"172.21.15.67:6809/2827491791","cluster_addr":"172.21.15.67:6811/2827491791","heartbeat_back_addr":"172.21.15.67:6815/2827491791","heartbeat_front_addr":"172.21.15.67:6813/2827491791","state":["exists","up"]},{"osd":3,"uuid":"1d889d76-a646-499d-bfff-ed5bae5434ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6819","nonce":1848001176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6821","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6823","nonce":1848001176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6831","nonce":1848001176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6825","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6827","nonce":1848001176}]},"public_addr":"172.21.15.67:6819/1848001176","cluster_addr":"172.21.15.67:6823/1848001176","heartbeat_back_addr":"172.21.15.67:6831/1848001176","heartbeat_front_addr":"172.21.15.67:6827/1848001176","state":["exists","up"]},{"osd":4,"uuid":"c3271dff-b437-4fdd-b13b-6e49a35473af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6804","nonce":1891230785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6808","nonce":1891230785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6815","nonce":1891230785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6811","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6813","nonce":1891230785}]},"public_addr":"172.21.15.72:6804/1891230785","cluster_addr":"172.21.15.72:6808/1891230785","heartbeat_back_addr":"172.21.15.72:6815/1891230785","heartbeat_front_addr":"172.21.15.72:6813/1891230785","state":["exists","up"]},{"osd":5,"uuid":"0337bb18-4b68-4eff-b528-a667275e6926","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6801","nonce":569686991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6803","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6805","nonce":569686991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6812","nonce":569686991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6807","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6809","nonce":569686991}]},"public_addr":"172.21.15.72:6801/569686991","cluster_addr":"172.21.15.72:6805/569686991","heartbeat_back_addr":"172.21.15.72:6812/569686991","heartbeat_front_addr":"172.21.15.72:6809/569686991","state":["exists","up"]},{"osd":6,"uuid":"8e92d9cf-0c8c-46eb-89a7-db2023ef1509","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6817","nonce":3767685934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6819","nonce":3767685934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6823","nonce":3767685934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6821","nonce":3767685934}]},"public_addr":"172.21.15.72:6817/3767685934","cluster_addr":"172.21.15.72:6819/3767685934","heartbeat_back_addr":"172.21.15.72:6823/3767685934","heartbeat_front_addr":"172.21.15.72:6821/3767685934","state":["exists","up"]},{"osd":7,"uuid":"3fd6aa7d-50a9-45c9-9561-5902a044a119","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6825","nonce":3567465926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6827","nonce":3567465926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6831","nonce":3567465926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6829","nonce":3567465926}]},"public_addr":"172.21.15.72:6825/3567465926","cluster_addr":"172.21.15.72:6827/3567465926","heartbeat_back_addr":"172.21.15.72:6831/3567465926","heartbeat_front_addr":"172.21.15.72:6829/3567465926","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.511466+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.795107+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.858674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.728895+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.794842+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.750064+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.732122+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.765185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-14T10:36:14.168 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.165+0000 7f53c9b71700 1 -- 172.21.15.67:0/3382114 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f53b006af00 msgr2=0x7f53b006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:14.168 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.165+0000 7f53c9b71700 1 --2- 172.21.15.67:0/3382114 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f53b006af00 0x7f53b006d3c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f53b4000f60 tx=0x7f53b4007cc0 comp rx=0 tx=0).stop 2023-10-14T10:36:14.168 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.165+0000 7f53c9b71700 1 -- 172.21.15.67:0/3382114 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f53c40d1bc0 msgr2=0x7f53c41550e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:14.168 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.165+0000 7f53c9b71700 1 --2- 172.21.15.67:0/3382114 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f53c40d1bc0 0x7f53c41550e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f53b80090b0 tx=0x7f53b80047c0 comp rx=0 tx=0).stop 2023-10-14T10:36:14.169 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.165+0000 7f53c9b71700 1 -- 172.21.15.67:0/3382114 shutdown_connections 2023-10-14T10:36:14.169 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.165+0000 7f53c9b71700 1 --2- 172.21.15.67:0/3382114 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f53b006af00 0x7f53b006d3c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:14.169 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.165+0000 7f53c9b71700 1 --2- 172.21.15.67:0/3382114 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f53c415eab0 0x7f53c40b2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:14.169 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.165+0000 7f53c9b71700 1 --2- 172.21.15.67:0/3382114 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f53c40cf8a0 0x7f53c4155d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:14.170 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.165+0000 7f53c9b71700 1 --2- 172.21.15.67:0/3382114 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f53c40d1bc0 0x7f53c41550e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:14.170 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.165+0000 7f53c9b71700 1 -- 172.21.15.67:0/3382114 >> 172.21.15.67:0/3382114 conn(0x7f53c40665a0 msgr2=0x7f53c40b15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:14.170 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.165+0000 7f53c9b71700 1 -- 172.21.15.67:0/3382114 shutdown_connections 2023-10-14T10:36:14.170 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.165+0000 7f53c9b71700 1 -- 172.21.15.67:0/3382114 wait complete. 2023-10-14T10:36:14.180 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-14T10:35:57.809566+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-14T10:36:01.667146+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-14T10:36:02.219700+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-14T10:36:14.181 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-14T10:36:14.308 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.306+0000 7f29044b5700 1 Processor -- start 2023-10-14T10:36:14.308 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.306+0000 7f29044b5700 1 -- start start 2023-10-14T10:36:14.310 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.307+0000 7f29044b5700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f28fc06eb30 0x7f28fc15e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:14.311 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.307+0000 7f29044b5700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f28fc0b54f0 0x7f28fc15ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:14.311 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.307+0000 7f29044b5700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28fc0b5c70 con 0x7f28fc06e750 2023-10-14T10:36:14.311 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.307+0000 7f29044b5700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28fc0b5990 con 0x7f28fc0b54f0 2023-10-14T10:36:14.311 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.307+0000 7f29044b5700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28fc15b2b0 con 0x7f28fc06eb30 2023-10-14T10:36:14.311 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.307+0000 7f2902a52700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f28fc0b54f0 0x7f28fc15ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:14.312 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.307+0000 7f2902a52700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f28fc0b54f0 0x7f28fc15ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44374/0 (socket says 172.21.15.67:44374) 2023-10-14T10:36:14.312 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.307+0000 7f2902a52700 1 -- 172.21.15.67:0/2886818240 learned_addr learned my addr 172.21.15.67:0/2886818240 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:14.312 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.307+0000 7f2901a50700 1 --2- 172.21.15.67:0/2886818240 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f28fc06eb30 0x7f28fc15e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:14.312 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.307+0000 7f2902a52700 1 -- 172.21.15.67:0/2886818240 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f28fc06eb30 msgr2=0x7f28fc15e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:14.312 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.307+0000 7f2902a52700 1 --2- 172.21.15.67:0/2886818240 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f28fc06eb30 0x7f28fc15e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:14.313 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.307+0000 7f2902a52700 1 -- 172.21.15.67:0/2886818240 >> v1:172.21.15.72:6789/0 conn(0x7f28fc06e750 legacy=0x7f28fc0ceec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:14.313 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.307+0000 7f2902a52700 1 -- 172.21.15.67:0/2886818240 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28fc15b590 con 0x7f28fc0b54f0 2023-10-14T10:36:14.313 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.307+0000 7f2902a52700 1 --2- 172.21.15.67:0/2886818240 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f28fc0b54f0 0x7f28fc15ec40 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f28f4009e50 tx=0x7f28f400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ba494b4f5ad6264 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:14.313 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.307+0000 7f290124f700 1 -- 172.21.15.67:0/2886818240 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f400e070 con 0x7f28fc0b54f0 2023-10-14T10:36:14.313 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.308+0000 7f290124f700 1 -- 172.21.15.67:0/2886818240 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28f4003a20 con 0x7f28fc0b54f0 2023-10-14T10:36:14.314 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.308+0000 7f290124f700 1 -- 172.21.15.67:0/2886818240 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f4003dd0 con 0x7f28fc0b54f0 2023-10-14T10:36:14.314 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.308+0000 7f29044b5700 1 -- 172.21.15.67:0/2886818240 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f28fc0b54f0 msgr2=0x7f28fc15ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:14.314 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.308+0000 7f29044b5700 1 --2- 172.21.15.67:0/2886818240 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f28fc0b54f0 0x7f28fc15ec40 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f28f4009e50 tx=0x7f28f400b330 comp rx=0 tx=0).stop 2023-10-14T10:36:14.314 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.308+0000 7f29044b5700 1 -- 172.21.15.67:0/2886818240 shutdown_connections 2023-10-14T10:36:14.314 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.308+0000 7f29044b5700 1 --2- 172.21.15.67:0/2886818240 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f28fc0b54f0 0x7f28fc15ec40 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:14.315 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.308+0000 7f29044b5700 1 --2- 172.21.15.67:0/2886818240 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f28fc06eb30 0x7f28fc15e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:14.315 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.308+0000 7f29044b5700 1 -- 172.21.15.67:0/2886818240 >> 172.21.15.67:0/2886818240 conn(0x7f28fc0cc6f0 msgr2=0x7f28fc0ccb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:14.315 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.308+0000 7f29044b5700 1 -- 172.21.15.67:0/2886818240 shutdown_connections 2023-10-14T10:36:14.315 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.308+0000 7f29044b5700 1 -- 172.21.15.67:0/2886818240 wait complete. 2023-10-14T10:36:14.318 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.309+0000 7f29044b5700 1 Processor -- start 2023-10-14T10:36:14.318 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.309+0000 7f29044b5700 1 -- start start 2023-10-14T10:36:14.318 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.310+0000 7f29044b5700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f28fc06e750 0x7f28fc0c6550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:14.318 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.310+0000 7f29044b5700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f28fc06eb30 0x7f28fc0a8660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:14.318 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.310+0000 7f29044b5700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f28fc0b54f0 0x7f28fc146c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:14.319 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.310+0000 7f29044b5700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28fc1471d0 con 0x7f28fc06e750 2023-10-14T10:36:14.319 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.310+0000 7f29044b5700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28fc147470 con 0x7f28fc0b54f0 2023-10-14T10:36:14.319 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.310+0000 7f29044b5700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28fc147750 con 0x7f28fc06eb30 2023-10-14T10:36:14.319 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.310+0000 7f2902251700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f28fc06e750 0x7f28fc0c6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:14.319 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.310+0000 7f2902251700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f28fc06e750 0x7f28fc0c6550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44376/0 (socket says 172.21.15.67:44376) 2023-10-14T10:36:14.320 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.310+0000 7f2902251700 1 -- 172.21.15.67:0/2237764324 learned_addr learned my addr 172.21.15.67:0/2237764324 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:14.320 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.310+0000 7f2902251700 1 -- 172.21.15.67:0/2237764324 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f28fc06eb30 msgr2=0x7f28fc0a8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:14.320 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.310+0000 7f2902a52700 1 --2- 172.21.15.67:0/2237764324 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f28fc0b54f0 0x7f28fc146c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:14.320 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.310+0000 7f2902251700 1 --2- 172.21.15.67:0/2237764324 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f28fc06eb30 0x7f28fc0a8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:14.320 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.310+0000 7f2902251700 1 -- 172.21.15.67:0/2237764324 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f28fc0b54f0 msgr2=0x7f28fc146c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:14.321 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.310+0000 7f2902251700 1 --2- 172.21.15.67:0/2237764324 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f28fc0b54f0 0x7f28fc146c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:14.321 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.311+0000 7f2902251700 1 -- 172.21.15.67:0/2237764324 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28f4009b30 con 0x7f28fc06e750 2023-10-14T10:36:14.321 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.311+0000 7f2902251700 1 --2- 172.21.15.67:0/2237764324 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f28fc06e750 0x7f28fc0c6550 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f28f800c700 tx=0x7f28f800cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:14.321 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.311+0000 7f28f2ffd700 1 -- 172.21.15.67:0/2237764324 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f8011790 con 0x7f28fc06e750 2023-10-14T10:36:14.321 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.311+0000 7f28f2ffd700 1 -- 172.21.15.67:0/2237764324 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28f80118f0 con 0x7f28fc06e750 2023-10-14T10:36:14.322 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.311+0000 7f29044b5700 1 -- 172.21.15.67:0/2237764324 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28fc147a30 con 0x7f28fc06e750 2023-10-14T10:36:14.322 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.311+0000 7f28f2ffd700 1 -- 172.21.15.67:0/2237764324 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f8011ca0 con 0x7f28fc06e750 2023-10-14T10:36:14.322 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.311+0000 7f29044b5700 1 -- 172.21.15.67:0/2237764324 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28fc147ff0 con 0x7f28fc06e750 2023-10-14T10:36:14.322 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.312+0000 7f29044b5700 1 -- 172.21.15.67:0/2237764324 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28fc067d20 con 0x7f28fc06e750 2023-10-14T10:36:14.322 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.312+0000 7f28f2ffd700 1 -- 172.21.15.67:0/2237764324 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f28f8005080 con 0x7f28fc06e750 2023-10-14T10:36:14.322 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.313+0000 7f28f2ffd700 1 --2- 172.21.15.67:0/2237764324 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f28e006af00 0x7f28e006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:14.323 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.313+0000 7f28f2ffd700 1 -- 172.21.15.67:0/2237764324 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f28f8089020 con 0x7f28fc06e750 2023-10-14T10:36:14.323 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.313+0000 7f2901a50700 1 --2- 172.21.15.67:0/2237764324 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f28e006af00 0x7f28e006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:14.323 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.313+0000 7f2901a50700 1 --2- 172.21.15.67:0/2237764324 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f28e006af00 0x7f28e006d3c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f28fc158520 tx=0x7f28ec008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:14.324 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.317+0000 7f28f2ffd700 1 -- 172.21.15.67:0/2237764324 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f28f8054160 con 0x7f28fc06e750 2023-10-14T10:36:14.521 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.519+0000 7f29044b5700 1 -- 172.21.15.67:0/2237764324 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f28fc06b950 con 0x7f28fc06e750 2023-10-14T10:36:14.522 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.520+0000 7f28f2ffd700 1 -- 172.21.15.67:0/2237764324 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f28f8057780 con 0x7f28fc06e750 2023-10-14T10:36:14.522 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2023-10-14T10:36:14.524 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.522+0000 7f29044b5700 1 -- 172.21.15.67:0/2237764324 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f28e006af00 msgr2=0x7f28e006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:14.524 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.522+0000 7f29044b5700 1 --2- 172.21.15.67:0/2237764324 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f28e006af00 0x7f28e006d3c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f28fc158520 tx=0x7f28ec008040 comp rx=0 tx=0).stop 2023-10-14T10:36:14.525 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.522+0000 7f29044b5700 1 -- 172.21.15.67:0/2237764324 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f28fc06e750 msgr2=0x7f28fc0c6550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:14.525 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.522+0000 7f29044b5700 1 --2- 172.21.15.67:0/2237764324 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f28fc06e750 0x7f28fc0c6550 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f28f800c700 tx=0x7f28f800cac0 comp rx=0 tx=0).stop 2023-10-14T10:36:14.525 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.522+0000 7f29044b5700 1 -- 172.21.15.67:0/2237764324 shutdown_connections 2023-10-14T10:36:14.525 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.522+0000 7f29044b5700 1 --2- 172.21.15.67:0/2237764324 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f28e006af00 0x7f28e006d3c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:14.525 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.522+0000 7f29044b5700 1 --2- 172.21.15.67:0/2237764324 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f28fc0b54f0 0x7f28fc146c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:14.526 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.522+0000 7f29044b5700 1 --2- 172.21.15.67:0/2237764324 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f28fc06e750 0x7f28fc0c6550 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:14.526 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.522+0000 7f29044b5700 1 --2- 172.21.15.67:0/2237764324 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f28fc06eb30 0x7f28fc0a8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:14.526 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.522+0000 7f29044b5700 1 -- 172.21.15.67:0/2237764324 >> 172.21.15.67:0/2237764324 conn(0x7f28fc0cc6f0 msgr2=0x7f28fc0ceec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:14.526 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.522+0000 7f29044b5700 1 -- 172.21.15.67:0/2237764324 shutdown_connections 2023-10-14T10:36:14.526 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.522+0000 7f29044b5700 1 -- 172.21.15.67:0/2237764324 wait complete. 2023-10-14T10:36:14.537 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-14T10:36:14.665 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.663+0000 7fda46237700 1 Processor -- start 2023-10-14T10:36:14.665 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.663+0000 7fda46237700 1 -- start start 2023-10-14T10:36:14.668 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.664+0000 7fda46237700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fda4015b740 0x7fda400d1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:14.668 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.664+0000 7fda46237700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fda400ce6e0 0x7fda400cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:14.668 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.664+0000 7fda46237700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda400cf190 con 0x7fda400d0c70 2023-10-14T10:36:14.668 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.664+0000 7fda46237700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda4015bb20 con 0x7fda4015b740 2023-10-14T10:36:14.669 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.664+0000 7fda46237700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda400cbd40 con 0x7fda400ce6e0 2023-10-14T10:36:14.669 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.664+0000 7fda3f7fe700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fda4015b740 0x7fda400d1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:14.669 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.664+0000 7fda3f7fe700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fda4015b740 0x7fda400d1cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44384/0 (socket says 172.21.15.67:44384) 2023-10-14T10:36:14.669 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.664+0000 7fda3f7fe700 1 -- 172.21.15.67:0/3906022347 learned_addr learned my addr 172.21.15.67:0/3906022347 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:14.670 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.664+0000 7fda3ffff700 1 --2- 172.21.15.67:0/3906022347 >> v2:172.21.15.72:3300/0 conn(0x7fda400ce6e0 0x7fda400cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:14.670 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.664+0000 7fda3f7fe700 1 -- 172.21.15.67:0/3906022347 >> v2:172.21.15.72:3300/0 conn(0x7fda400ce6e0 msgr2=0x7fda400cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:14.670 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.664+0000 7fda3f7fe700 1 --2- 172.21.15.67:0/3906022347 >> v2:172.21.15.72:3300/0 conn(0x7fda400ce6e0 0x7fda400cf690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:14.670 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.664+0000 7fda3f7fe700 1 -- 172.21.15.67:0/3906022347 >> v1:172.21.15.72:6789/0 conn(0x7fda400d0c70 legacy=0x7fda400cdfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:14.671 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.664+0000 7fda3f7fe700 1 -- 172.21.15.67:0/3906022347 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda400cc020 con 0x7fda4015b740 2023-10-14T10:36:14.671 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.664+0000 7fda3f7fe700 1 --2- 172.21.15.67:0/3906022347 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fda4015b740 0x7fda400d1cd0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fda400c8620 tx=0x7fda28009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5143879d0bebe456 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:14.671 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.664+0000 7fda3e7fc700 1 -- 172.21.15.67:0/3906022347 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2800b870 con 0x7fda4015b740 2023-10-14T10:36:14.671 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.665+0000 7fda3e7fc700 1 -- 172.21.15.67:0/3906022347 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda2800b9d0 con 0x7fda4015b740 2023-10-14T10:36:14.671 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.665+0000 7fda3e7fc700 1 -- 172.21.15.67:0/3906022347 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2800bd60 con 0x7fda4015b740 2023-10-14T10:36:14.672 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.665+0000 7fda46237700 1 -- 172.21.15.67:0/3906022347 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fda4015b740 msgr2=0x7fda400d1cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:14.672 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.665+0000 7fda46237700 1 --2- 172.21.15.67:0/3906022347 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fda4015b740 0x7fda400d1cd0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fda400c8620 tx=0x7fda28009bd0 comp rx=0 tx=0).stop 2023-10-14T10:36:14.672 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.665+0000 7fda46237700 1 -- 172.21.15.67:0/3906022347 shutdown_connections 2023-10-14T10:36:14.672 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.665+0000 7fda46237700 1 --2- 172.21.15.67:0/3906022347 >> v2:172.21.15.72:3300/0 conn(0x7fda400ce6e0 0x7fda400cf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:14.672 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.665+0000 7fda46237700 1 --2- 172.21.15.67:0/3906022347 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fda4015b740 0x7fda400d1cd0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:14.673 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.665+0000 7fda46237700 1 -- 172.21.15.67:0/3906022347 >> 172.21.15.67:0/3906022347 conn(0x7fda400665a0 msgr2=0x7fda40066980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:14.673 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.665+0000 7fda46237700 1 -- 172.21.15.67:0/3906022347 shutdown_connections 2023-10-14T10:36:14.673 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.665+0000 7fda46237700 1 -- 172.21.15.67:0/3906022347 wait complete. 2023-10-14T10:36:14.676 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.666+0000 7fda46237700 1 Processor -- start 2023-10-14T10:36:14.676 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.667+0000 7fda46237700 1 -- start start 2023-10-14T10:36:14.676 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.667+0000 7fda46237700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fda400ce6e0 0x7fda401f26c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:14.676 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.667+0000 7fda46237700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fda400d0c70 0x7fda400b6500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:14.676 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.667+0000 7fda46237700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fda4015b740 0x7fda400b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:14.677 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.667+0000 7fda46237700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda40151f40 con 0x7fda400ce6e0 2023-10-14T10:36:14.677 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.667+0000 7fda46237700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda4006d170 con 0x7fda400d0c70 2023-10-14T10:36:14.677 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.667+0000 7fda46237700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda4006d600 con 0x7fda4015b740 2023-10-14T10:36:14.677 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.667+0000 7fda3f7fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fda400ce6e0 0x7fda401f26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:14.677 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.667+0000 7fda3f7fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fda400ce6e0 0x7fda401f26c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44390/0 (socket says 172.21.15.67:44390) 2023-10-14T10:36:14.678 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.667+0000 7fda3f7fe700 1 -- 172.21.15.67:0/1587245788 learned_addr learned my addr 172.21.15.67:0/1587245788 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:14.678 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.668+0000 7fda3effd700 1 --2- 172.21.15.67:0/1587245788 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fda400d0c70 0x7fda400b6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:14.678 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.668+0000 7fda3ffff700 1 --2- 172.21.15.67:0/1587245788 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fda4015b740 0x7fda400b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:14.678 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.668+0000 7fda3f7fe700 1 -- 172.21.15.67:0/1587245788 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fda4015b740 msgr2=0x7fda400b6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:14.679 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.668+0000 7fda3f7fe700 1 --2- 172.21.15.67:0/1587245788 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fda4015b740 0x7fda400b6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:14.679 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.668+0000 7fda3f7fe700 1 -- 172.21.15.67:0/1587245788 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fda400d0c70 msgr2=0x7fda400b6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:14.679 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.668+0000 7fda3f7fe700 1 --2- 172.21.15.67:0/1587245788 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fda400d0c70 0x7fda400b6500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:14.679 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.668+0000 7fda3f7fe700 1 -- 172.21.15.67:0/1587245788 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda28009550 con 0x7fda400ce6e0 2023-10-14T10:36:14.679 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.668+0000 7fda3f7fe700 1 --2- 172.21.15.67:0/1587245788 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fda400ce6e0 0x7fda401f26c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fda28000c00 tx=0x7fda28004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:14.680 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.668+0000 7fda37fff700 1 -- 172.21.15.67:0/1587245788 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda28004540 con 0x7fda400ce6e0 2023-10-14T10:36:14.680 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.668+0000 7fda37fff700 1 -- 172.21.15.67:0/1587245788 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda280036a0 con 0x7fda400ce6e0 2023-10-14T10:36:14.680 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.668+0000 7fda46237700 1 -- 172.21.15.67:0/1587245788 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda4006c560 con 0x7fda400ce6e0 2023-10-14T10:36:14.680 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.668+0000 7fda37fff700 1 -- 172.21.15.67:0/1587245788 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda28003a30 con 0x7fda400ce6e0 2023-10-14T10:36:14.680 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.668+0000 7fda46237700 1 -- 172.21.15.67:0/1587245788 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda4006cb20 con 0x7fda400ce6e0 2023-10-14T10:36:14.681 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.669+0000 7fda46237700 1 -- 172.21.15.67:0/1587245788 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda40067d20 con 0x7fda400ce6e0 2023-10-14T10:36:14.681 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.669+0000 7fda37fff700 1 -- 172.21.15.67:0/1587245788 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fda28021070 con 0x7fda400ce6e0 2023-10-14T10:36:14.681 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.670+0000 7fda37fff700 1 --2- 172.21.15.67:0/1587245788 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fda2006af00 0x7fda2006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:14.681 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.670+0000 7fda37fff700 1 -- 172.21.15.67:0/1587245788 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fda2808ac90 con 0x7fda400ce6e0 2023-10-14T10:36:14.681 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.670+0000 7fda3effd700 1 --2- 172.21.15.67:0/1587245788 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fda2006af00 0x7fda2006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:14.682 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.670+0000 7fda3effd700 1 --2- 172.21.15.67:0/1587245788 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fda2006af00 0x7fda2006d3c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fda40151b40 tx=0x7fda30009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:14.682 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.674+0000 7fda37fff700 1 -- 172.21.15.67:0/1587245788 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fda28055ef0 con 0x7fda400ce6e0 2023-10-14T10:36:14.876 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.874+0000 7fda46237700 1 -- 172.21.15.67:0/1587245788 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fda4006dc80 con 0x7fda400ce6e0 2023-10-14T10:36:14.876 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.874+0000 7fda37fff700 1 -- 172.21.15.67:0/1587245788 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fda280150a0 con 0x7fda400ce6e0 2023-10-14T10:36:14.877 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2023-10-14T10:36:14.878 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.876+0000 7fda46237700 1 -- 172.21.15.67:0/1587245788 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fda2006af00 msgr2=0x7fda2006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:14.879 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.876+0000 7fda46237700 1 --2- 172.21.15.67:0/1587245788 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fda2006af00 0x7fda2006d3c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fda40151b40 tx=0x7fda30009040 comp rx=0 tx=0).stop 2023-10-14T10:36:14.879 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.876+0000 7fda46237700 1 -- 172.21.15.67:0/1587245788 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fda400ce6e0 msgr2=0x7fda401f26c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:14.879 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.876+0000 7fda46237700 1 --2- 172.21.15.67:0/1587245788 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fda400ce6e0 0x7fda401f26c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fda28000c00 tx=0x7fda28004750 comp rx=0 tx=0).stop 2023-10-14T10:36:14.879 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.876+0000 7fda46237700 1 -- 172.21.15.67:0/1587245788 shutdown_connections 2023-10-14T10:36:14.879 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.876+0000 7fda46237700 1 --2- 172.21.15.67:0/1587245788 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fda2006af00 0x7fda2006d3c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:14.880 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.876+0000 7fda46237700 1 --2- 172.21.15.67:0/1587245788 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fda4015b740 0x7fda400b6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:14.880 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.876+0000 7fda46237700 1 --2- 172.21.15.67:0/1587245788 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fda400ce6e0 0x7fda401f26c0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:14.880 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.876+0000 7fda46237700 1 --2- 172.21.15.67:0/1587245788 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fda400d0c70 0x7fda400b6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:14.880 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.876+0000 7fda46237700 1 -- 172.21.15.67:0/1587245788 >> 172.21.15.67:0/1587245788 conn(0x7fda400665a0 msgr2=0x7fda400c5ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:14.880 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.876+0000 7fda46237700 1 -- 172.21.15.67:0/1587245788 shutdown_connections 2023-10-14T10:36:14.881 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:14.876+0000 7fda46237700 1 -- 172.21.15.67:0/1587245788 wait complete. 2023-10-14T10:36:14.891 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-14T10:36:15.019 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.017+0000 7f12e2755700 1 Processor -- start 2023-10-14T10:36:15.019 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.017+0000 7f12e2755700 1 -- start start 2023-10-14T10:36:15.022 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.017+0000 7f12e2755700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f12dc15ab10 0x7f12dc15f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:15.022 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.017+0000 7f12e2755700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f12dc0b1f50 0x7f12dc0cf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:15.023 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.017+0000 7f12e2755700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f12dc0b23d0 0x7f12dc0cfc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:15.023 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.017+0000 7f12e2755700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12dc15e7a0 con 0x7f12dc15ab10 2023-10-14T10:36:15.023 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.017+0000 7f12e2755700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12dc15f730 con 0x7f12dc0b23d0 2023-10-14T10:36:15.023 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.017+0000 7f12e2755700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12dc0b29a0 con 0x7f12dc0b1f50 2023-10-14T10:36:15.023 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.018+0000 7f12dbfff700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f12dc15ab10 0x7f12dc15f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:15.023 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.018+0000 7f12dbfff700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f12dc15ab10 0x7f12dc15f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44398/0 (socket says 172.21.15.67:44398) 2023-10-14T10:36:15.024 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.018+0000 7f12dbfff700 1 -- 172.21.15.67:0/930689833 learned_addr learned my addr 172.21.15.67:0/930689833 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:15.024 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.018+0000 7f12db7fe700 1 --2- 172.21.15.67:0/930689833 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f12dc0b1f50 0x7f12dc0cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:15.024 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.018+0000 7f12e0cf2700 1 --2- 172.21.15.67:0/930689833 >> v2:172.21.15.72:3300/0 conn(0x7f12dc0b23d0 0x7f12dc0cfc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:15.024 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.018+0000 7f12dbfff700 1 -- 172.21.15.67:0/930689833 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f12dc0b1f50 msgr2=0x7f12dc0cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:15.025 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.018+0000 7f12dbfff700 1 --2- 172.21.15.67:0/930689833 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f12dc0b1f50 0x7f12dc0cf720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:15.025 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.018+0000 7f12dbfff700 1 -- 172.21.15.67:0/930689833 >> v2:172.21.15.72:3300/0 conn(0x7f12dc0b23d0 msgr2=0x7f12dc0cfc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:15.025 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.018+0000 7f12dbfff700 1 --2- 172.21.15.67:0/930689833 >> v2:172.21.15.72:3300/0 conn(0x7f12dc0b23d0 0x7f12dc0cfc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:15.025 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.018+0000 7f12dbfff700 1 -- 172.21.15.67:0/930689833 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12dc0b2c80 con 0x7f12dc15ab10 2023-10-14T10:36:15.025 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.018+0000 7f12e0cf2700 1 --2- 172.21.15.67:0/930689833 >> v2:172.21.15.72:3300/0 conn(0x7f12dc0b23d0 0x7f12dc0cfc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-14T10:36:15.025 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.018+0000 7f12dbfff700 1 --2- 172.21.15.67:0/930689833 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12dc15ab10 0x7f12dc15f1f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f12dc072ef0 tx=0x7f12c4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d145d1f2b8a12ebf server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:15.026 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.018+0000 7f12daffd700 1 -- 172.21.15.67:0/930689833 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c400b870 con 0x7f12dc15ab10 2023-10-14T10:36:15.026 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.018+0000 7f12daffd700 1 -- 172.21.15.67:0/930689833 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12c400b9d0 con 0x7f12dc15ab10 2023-10-14T10:36:15.026 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.018+0000 7f12daffd700 1 -- 172.21.15.67:0/930689833 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c400bd60 con 0x7f12dc15ab10 2023-10-14T10:36:15.026 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.019+0000 7f12e2755700 1 -- 172.21.15.67:0/930689833 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12dc15ab10 msgr2=0x7f12dc15f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:15.026 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.019+0000 7f12e2755700 1 --2- 172.21.15.67:0/930689833 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12dc15ab10 0x7f12dc15f1f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f12dc072ef0 tx=0x7f12c4009bd0 comp rx=0 tx=0).stop 2023-10-14T10:36:15.027 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.019+0000 7f12e2755700 1 -- 172.21.15.67:0/930689833 shutdown_connections 2023-10-14T10:36:15.027 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.019+0000 7f12e2755700 1 --2- 172.21.15.67:0/930689833 >> v2:172.21.15.72:3300/0 conn(0x7f12dc0b23d0 0x7f12dc0cfc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:15.027 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.019+0000 7f12e2755700 1 --2- 172.21.15.67:0/930689833 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12dc15ab10 0x7f12dc15f1f0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:15.027 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.019+0000 7f12e2755700 1 --2- 172.21.15.67:0/930689833 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f12dc0b1f50 0x7f12dc0cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:15.027 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.019+0000 7f12e2755700 1 -- 172.21.15.67:0/930689833 >> 172.21.15.67:0/930689833 conn(0x7f12dc06dae0 msgr2=0x7f12dc06def0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:15.028 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.019+0000 7f12e2755700 1 -- 172.21.15.67:0/930689833 shutdown_connections 2023-10-14T10:36:15.028 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.019+0000 7f12e2755700 1 -- 172.21.15.67:0/930689833 wait complete. 2023-10-14T10:36:15.028 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.020+0000 7f12e2755700 1 Processor -- start 2023-10-14T10:36:15.028 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.020+0000 7f12e2755700 1 -- start start 2023-10-14T10:36:15.031 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.021+0000 7f12e2755700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f12dc0b1f50 0x7f12dc1f7a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:15.031 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.021+0000 7f12e2755700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12dc0b23d0 0x7f12dc0b4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:15.032 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.021+0000 7f12e2755700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f12dc15ab10 0x7f12dc1f0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:15.032 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.021+0000 7f12e2755700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12dc1f39a0 con 0x7f12dc0b23d0 2023-10-14T10:36:15.032 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.021+0000 7f12e2755700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12dc1f44c0 con 0x7f12dc0b1f50 2023-10-14T10:36:15.032 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.021+0000 7f12e2755700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12dc0b5250 con 0x7f12dc15ab10 2023-10-14T10:36:15.032 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.021+0000 7f12db7fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12dc0b23d0 0x7f12dc0b4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:15.033 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.021+0000 7f12db7fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12dc0b23d0 0x7f12dc0b4d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44406/0 (socket says 172.21.15.67:44406) 2023-10-14T10:36:15.033 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.021+0000 7f12db7fe700 1 -- 172.21.15.67:0/654418755 learned_addr learned my addr 172.21.15.67:0/654418755 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:15.033 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.021+0000 7f12dbfff700 1 --2- 172.21.15.67:0/654418755 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f12dc0b1f50 0x7f12dc1f7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:15.033 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.021+0000 7f12e0cf2700 1 --2- 172.21.15.67:0/654418755 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f12dc15ab10 0x7f12dc1f0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:15.033 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.021+0000 7f12db7fe700 1 -- 172.21.15.67:0/654418755 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f12dc15ab10 msgr2=0x7f12dc1f0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:15.034 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.021+0000 7f12db7fe700 1 --2- 172.21.15.67:0/654418755 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f12dc15ab10 0x7f12dc1f0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:15.034 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.021+0000 7f12db7fe700 1 -- 172.21.15.67:0/654418755 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f12dc0b1f50 msgr2=0x7f12dc1f7a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:15.034 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.021+0000 7f12db7fe700 1 --2- 172.21.15.67:0/654418755 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f12dc0b1f50 0x7f12dc1f7a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:15.034 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.021+0000 7f12db7fe700 1 -- 172.21.15.67:0/654418755 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12c4009550 con 0x7f12dc0b23d0 2023-10-14T10:36:15.034 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.022+0000 7f12db7fe700 1 --2- 172.21.15.67:0/654418755 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12dc0b23d0 0x7f12dc0b4d10 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f12cc00d800 tx=0x7f12cc00dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:15.035 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.022+0000 7f12d8ff9700 1 -- 172.21.15.67:0/654418755 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12cc0125f0 con 0x7f12dc0b23d0 2023-10-14T10:36:15.035 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.022+0000 7f12d8ff9700 1 -- 172.21.15.67:0/654418755 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12cc012750 con 0x7f12dc0b23d0 2023-10-14T10:36:15.035 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.022+0000 7f12d8ff9700 1 -- 172.21.15.67:0/654418755 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12cc012ae0 con 0x7f12dc0b23d0 2023-10-14T10:36:15.035 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.022+0000 7f12e2755700 1 -- 172.21.15.67:0/654418755 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12dc1f3b90 con 0x7f12dc0b23d0 2023-10-14T10:36:15.037 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.022+0000 7f12e2755700 1 -- 172.21.15.67:0/654418755 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12dc1f4150 con 0x7f12dc0b23d0 2023-10-14T10:36:15.037 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.023+0000 7f12e2755700 1 -- 172.21.15.67:0/654418755 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12dc067d20 con 0x7f12dc0b23d0 2023-10-14T10:36:15.037 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.023+0000 7f12d8ff9700 1 -- 172.21.15.67:0/654418755 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f12cc012c40 con 0x7f12dc0b23d0 2023-10-14T10:36:15.037 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.024+0000 7f12d8ff9700 1 --2- 172.21.15.67:0/654418755 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f12bc06af00 0x7f12bc06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:15.038 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.024+0000 7f12d8ff9700 1 -- 172.21.15.67:0/654418755 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f12cc0898f0 con 0x7f12dc0b23d0 2023-10-14T10:36:15.038 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.024+0000 7f12dbfff700 1 --2- 172.21.15.67:0/654418755 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f12bc06af00 0x7f12bc06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:15.038 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.024+0000 7f12dbfff700 1 --2- 172.21.15.67:0/654418755 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f12bc06af00 0x7f12bc06d3c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f12c4014040 tx=0x7f12c4008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:15.038 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.028+0000 7f12d8ff9700 1 -- 172.21.15.67:0/654418755 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f12cc0549a0 con 0x7f12dc0b23d0 2023-10-14T10:36:15.228 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.226+0000 7f12e2755700 1 -- 172.21.15.67:0/654418755 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f12dc1f4850 con 0x7f12dc0b23d0 2023-10-14T10:36:15.229 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.227+0000 7f12d8ff9700 1 -- 172.21.15.67:0/654418755 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f12cc057fc0 con 0x7f12dc0b23d0 2023-10-14T10:36:15.229 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2023-10-14T10:36:15.231 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.229+0000 7f12e2755700 1 -- 172.21.15.67:0/654418755 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f12bc06af00 msgr2=0x7f12bc06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:15.232 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.229+0000 7f12e2755700 1 --2- 172.21.15.67:0/654418755 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f12bc06af00 0x7f12bc06d3c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f12c4014040 tx=0x7f12c4008030 comp rx=0 tx=0).stop 2023-10-14T10:36:15.232 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.229+0000 7f12e2755700 1 -- 172.21.15.67:0/654418755 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12dc0b23d0 msgr2=0x7f12dc0b4d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:15.232 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.229+0000 7f12e2755700 1 --2- 172.21.15.67:0/654418755 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12dc0b23d0 0x7f12dc0b4d10 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f12cc00d800 tx=0x7f12cc00dbc0 comp rx=0 tx=0).stop 2023-10-14T10:36:15.232 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.229+0000 7f12e2755700 1 -- 172.21.15.67:0/654418755 shutdown_connections 2023-10-14T10:36:15.232 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.229+0000 7f12e2755700 1 --2- 172.21.15.67:0/654418755 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f12bc06af00 0x7f12bc06d3c0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:15.233 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.229+0000 7f12e2755700 1 --2- 172.21.15.67:0/654418755 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f12dc15ab10 0x7f12dc1f0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:15.233 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.229+0000 7f12e2755700 1 --2- 172.21.15.67:0/654418755 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f12dc0b1f50 0x7f12dc1f7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:15.233 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.229+0000 7f12e2755700 1 --2- 172.21.15.67:0/654418755 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12dc0b23d0 0x7f12dc0b4d10 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:15.233 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.229+0000 7f12e2755700 1 -- 172.21.15.67:0/654418755 >> 172.21.15.67:0/654418755 conn(0x7f12dc06dae0 msgr2=0x7f12dc06dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:15.233 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.229+0000 7f12e2755700 1 -- 172.21.15.67:0/654418755 shutdown_connections 2023-10-14T10:36:15.233 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.229+0000 7f12e2755700 1 -- 172.21.15.67:0/654418755 wait complete. 2023-10-14T10:36:15.247 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-14T10:36:15.378 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.376+0000 7f8a142e5700 1 Processor -- start 2023-10-14T10:36:15.378 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.376+0000 7f8a142e5700 1 -- start start 2023-10-14T10:36:15.381 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.377+0000 7f8a142e5700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f8a0c15ab10 0x7f8a0c15f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:15.381 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.377+0000 7f8a142e5700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f8a0c0b1f50 0x7f8a0c0cf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:15.381 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.377+0000 7f8a142e5700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f8a0c0b23d0 0x7f8a0c0cfc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:15.381 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.377+0000 7f8a142e5700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a0c15e7a0 con 0x7f8a0c0b23d0 2023-10-14T10:36:15.382 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.377+0000 7f8a142e5700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a0c15f730 con 0x7f8a0c15ab10 2023-10-14T10:36:15.382 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.377+0000 7f8a142e5700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0c0b29a0 con 0x7f8a0c0b1f50 2023-10-14T10:36:15.382 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.377+0000 7f8a12882700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f8a0c0b23d0 0x7f8a0c0cfc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:15.382 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.377+0000 7f8a12882700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f8a0c0b23d0 0x7f8a0c0cfc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44416/0 (socket says 172.21.15.67:44416) 2023-10-14T10:36:15.382 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.378+0000 7f8a12882700 1 -- 172.21.15.67:0/4139338397 learned_addr learned my addr 172.21.15.67:0/4139338397 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:15.383 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.378+0000 7f8a11880700 1 --2- 172.21.15.67:0/4139338397 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f8a0c0b1f50 0x7f8a0c0cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:15.383 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.378+0000 7f8a12081700 1 --2- 172.21.15.67:0/4139338397 >> v2:172.21.15.72:3300/0 conn(0x7f8a0c15ab10 0x7f8a0c15f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:15.383 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.378+0000 7f8a12882700 1 -- 172.21.15.67:0/4139338397 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f8a0c0b1f50 msgr2=0x7f8a0c0cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:15.383 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.378+0000 7f8a12882700 1 --2- 172.21.15.67:0/4139338397 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f8a0c0b1f50 0x7f8a0c0cf720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:15.383 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.378+0000 7f8a12882700 1 -- 172.21.15.67:0/4139338397 >> v2:172.21.15.72:3300/0 conn(0x7f8a0c15ab10 msgr2=0x7f8a0c15f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:15.384 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.378+0000 7f8a12882700 1 --2- 172.21.15.67:0/4139338397 >> v2:172.21.15.72:3300/0 conn(0x7f8a0c15ab10 0x7f8a0c15f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:15.384 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.378+0000 7f8a12882700 1 -- 172.21.15.67:0/4139338397 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a0c0b2c80 con 0x7f8a0c0b23d0 2023-10-14T10:36:15.384 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.378+0000 7f8a12882700 1 --2- 172.21.15.67:0/4139338397 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a0c0b23d0 0x7f8a0c0cfc60 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c0d0430 tx=0x7f8a0400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=567df49b7496ef1 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:15.384 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.378+0000 7f8a1107f700 1 -- 172.21.15.67:0/4139338397 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a0400e070 con 0x7f8a0c0b23d0 2023-10-14T10:36:15.384 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.378+0000 7f8a1107f700 1 -- 172.21.15.67:0/4139338397 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a0400b910 con 0x7f8a0c0b23d0 2023-10-14T10:36:15.385 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.378+0000 7f8a1107f700 1 -- 172.21.15.67:0/4139338397 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a0400bcc0 con 0x7f8a0c0b23d0 2023-10-14T10:36:15.385 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.379+0000 7f8a142e5700 1 -- 172.21.15.67:0/4139338397 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a0c0b23d0 msgr2=0x7f8a0c0cfc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:15.385 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.379+0000 7f8a142e5700 1 --2- 172.21.15.67:0/4139338397 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a0c0b23d0 0x7f8a0c0cfc60 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c0d0430 tx=0x7f8a0400b330 comp rx=0 tx=0).stop 2023-10-14T10:36:15.385 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.379+0000 7f8a142e5700 1 -- 172.21.15.67:0/4139338397 shutdown_connections 2023-10-14T10:36:15.385 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.379+0000 7f8a142e5700 1 --2- 172.21.15.67:0/4139338397 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a0c0b23d0 0x7f8a0c0cfc60 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:15.385 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.379+0000 7f8a142e5700 1 --2- 172.21.15.67:0/4139338397 >> v2:172.21.15.72:3300/0 conn(0x7f8a0c15ab10 0x7f8a0c15f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:15.386 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.379+0000 7f8a142e5700 1 --2- 172.21.15.67:0/4139338397 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f8a0c0b1f50 0x7f8a0c0cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:15.386 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.379+0000 7f8a142e5700 1 -- 172.21.15.67:0/4139338397 >> 172.21.15.67:0/4139338397 conn(0x7f8a0c06dae0 msgr2=0x7f8a0c06def0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:15.386 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.379+0000 7f8a142e5700 1 -- 172.21.15.67:0/4139338397 shutdown_connections 2023-10-14T10:36:15.386 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.379+0000 7f8a142e5700 1 -- 172.21.15.67:0/4139338397 wait complete. 2023-10-14T10:36:15.389 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.380+0000 7f8a142e5700 1 Processor -- start 2023-10-14T10:36:15.389 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.380+0000 7f8a142e5700 1 -- start start 2023-10-14T10:36:15.389 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.381+0000 7f8a142e5700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a0c0b1f50 0x7f8a0c0c3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:15.389 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.381+0000 7f8a142e5700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f8a0c0b23d0 0x7f8a0c06ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:15.390 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.381+0000 7f8a142e5700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8a0c15ab10 0x7f8a0c06b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:15.390 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.381+0000 7f8a142e5700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0c150ef0 con 0x7f8a0c0b1f50 2023-10-14T10:36:15.390 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.381+0000 7f8a142e5700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0c0c9860 con 0x7f8a0c15ab10 2023-10-14T10:36:15.390 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.381+0000 7f8a142e5700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0c0c9b00 con 0x7f8a0c0b23d0 2023-10-14T10:36:15.390 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.381+0000 7f8a12081700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a0c0b1f50 0x7f8a0c0c3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:15.391 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.381+0000 7f8a12081700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a0c0b1f50 0x7f8a0c0c3990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44418/0 (socket says 172.21.15.67:44418) 2023-10-14T10:36:15.391 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.381+0000 7f8a12081700 1 -- 172.21.15.67:0/1729535083 learned_addr learned my addr 172.21.15.67:0/1729535083 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:15.391 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.382+0000 7f8a11880700 1 --2- 172.21.15.67:0/1729535083 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f8a0c0b23d0 0x7f8a0c06ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:15.391 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.382+0000 7f8a12882700 1 --2- 172.21.15.67:0/1729535083 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8a0c15ab10 0x7f8a0c06b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:15.391 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.382+0000 7f8a12081700 1 -- 172.21.15.67:0/1729535083 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f8a0c0b23d0 msgr2=0x7f8a0c06ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:15.392 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.382+0000 7f8a12081700 1 --2- 172.21.15.67:0/1729535083 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f8a0c0b23d0 0x7f8a0c06ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:15.392 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.382+0000 7f8a12081700 1 -- 172.21.15.67:0/1729535083 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8a0c15ab10 msgr2=0x7f8a0c06b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:15.392 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.382+0000 7f8a12081700 1 --2- 172.21.15.67:0/1729535083 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8a0c15ab10 0x7f8a0c06b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:15.392 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.382+0000 7f8a12081700 1 -- 172.21.15.67:0/1729535083 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a04009b30 con 0x7f8a0c0b1f50 2023-10-14T10:36:15.392 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.382+0000 7f8a12081700 1 --2- 172.21.15.67:0/1729535083 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a0c0b1f50 0x7f8a0c0c3990 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f89fc00e870 tx=0x7f89fc00ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:15.392 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.382+0000 7f8a02ffd700 1 -- 172.21.15.67:0/1729535083 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89fc009590 con 0x7f8a0c0b1f50 2023-10-14T10:36:15.393 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.382+0000 7f8a02ffd700 1 -- 172.21.15.67:0/1729535083 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89fc00ee70 con 0x7f8a0c0b1f50 2023-10-14T10:36:15.393 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.382+0000 7f8a142e5700 1 -- 172.21.15.67:0/1729535083 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a0c0c9de0 con 0x7f8a0c0b1f50 2023-10-14T10:36:15.393 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.382+0000 7f8a02ffd700 1 -- 172.21.15.67:0/1729535083 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89fc00ca50 con 0x7f8a0c0b1f50 2023-10-14T10:36:15.393 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.382+0000 7f8a142e5700 1 -- 172.21.15.67:0/1729535083 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a0c0ca3a0 con 0x7f8a0c0b1f50 2023-10-14T10:36:15.393 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.383+0000 7f8a142e5700 1 -- 172.21.15.67:0/1729535083 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a0c0ca560 con 0x7f8a0c0b1f50 2023-10-14T10:36:15.394 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.383+0000 7f8a02ffd700 1 -- 172.21.15.67:0/1729535083 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f89fc00cbb0 con 0x7f8a0c0b1f50 2023-10-14T10:36:15.394 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.384+0000 7f8a02ffd700 1 --2- 172.21.15.67:0/1729535083 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f89f006af00 0x7f89f006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:15.394 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.384+0000 7f8a02ffd700 1 -- 172.21.15.67:0/1729535083 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f89fc089550 con 0x7f8a0c0b1f50 2023-10-14T10:36:15.394 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.384+0000 7f8a11880700 1 --2- 172.21.15.67:0/1729535083 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f89f006af00 0x7f89f006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:15.394 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.384+0000 7f8a11880700 1 --2- 172.21.15.67:0/1729535083 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f89f006af00 0x7f89f006d3c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c0b1390 tx=0x7f8a08008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:15.395 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.388+0000 7f8a02ffd700 1 -- 172.21.15.67:0/1729535083 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f89fc054600 con 0x7f8a0c0b1f50 2023-10-14T10:36:15.637 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.635+0000 7f8a142e5700 1 -- 172.21.15.67:0/1729535083 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8a0c067d20 con 0x7f8a0c0b1f50 2023-10-14T10:36:15.638 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.636+0000 7f8a02ffd700 1 -- 172.21.15.67:0/1729535083 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4901 (secure 0 0 0) 0x7f89fc00ce60 con 0x7f8a0c0b1f50 2023-10-14T10:36:15.638 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-14T10:36:15.638 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4217,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6836/838298018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":838298018},{"type":"v1","addr":"172.21.15.72:6836","nonce":838298018}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4255,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/1687192726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6833","nonce":1687192726},{"type":"v1","addr":"172.21.15.67:6835","nonce":1687192726}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6834/3936231765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":3936231765},{"type":"v1","addr":"172.21.15.67:6834","nonce":3936231765}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-14T10:36:03.247976+0000","modified":"2023-10-14T10:36:06.283381+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4267},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4267":{"gid":4267,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.72:6837/757500960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6835","nonce":757500960},{"type":"v1","addr":"172.21.15.72:6837","nonce":757500960}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-14T10:36:15.640 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.638+0000 7f8a142e5700 1 -- 172.21.15.67:0/1729535083 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f89f006af00 msgr2=0x7f89f006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:15.640 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.638+0000 7f8a142e5700 1 --2- 172.21.15.67:0/1729535083 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f89f006af00 0x7f89f006d3c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c0b1390 tx=0x7f8a08008040 comp rx=0 tx=0).stop 2023-10-14T10:36:15.641 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.638+0000 7f8a142e5700 1 -- 172.21.15.67:0/1729535083 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a0c0b1f50 msgr2=0x7f8a0c0c3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:15.641 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.638+0000 7f8a142e5700 1 --2- 172.21.15.67:0/1729535083 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a0c0b1f50 0x7f8a0c0c3990 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f89fc00e870 tx=0x7f89fc00ec30 comp rx=0 tx=0).stop 2023-10-14T10:36:15.641 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.638+0000 7f8a142e5700 1 -- 172.21.15.67:0/1729535083 shutdown_connections 2023-10-14T10:36:15.641 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.638+0000 7f8a142e5700 1 --2- 172.21.15.67:0/1729535083 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f89f006af00 0x7f89f006d3c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:15.641 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.638+0000 7f8a142e5700 1 --2- 172.21.15.67:0/1729535083 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8a0c15ab10 0x7f8a0c06b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:15.642 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.638+0000 7f8a142e5700 1 --2- 172.21.15.67:0/1729535083 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a0c0b1f50 0x7f8a0c0c3990 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:15.642 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.638+0000 7f8a142e5700 1 --2- 172.21.15.67:0/1729535083 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f8a0c0b23d0 0x7f8a0c06ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:15.642 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.638+0000 7f8a142e5700 1 -- 172.21.15.67:0/1729535083 >> 172.21.15.67:0/1729535083 conn(0x7f8a0c06dae0 msgr2=0x7f8a0c0b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:15.642 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.638+0000 7f8a142e5700 1 -- 172.21.15.67:0/1729535083 shutdown_connections 2023-10-14T10:36:15.642 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.638+0000 7f8a142e5700 1 -- 172.21.15.67:0/1729535083 wait complete. 2023-10-14T10:36:15.643 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 8 2023-10-14T10:36:15.655 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-14T10:36:15.780 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.778+0000 7f4f10aea700 1 Processor -- start 2023-10-14T10:36:15.781 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.778+0000 7f4f10aea700 1 -- start start 2023-10-14T10:36:15.784 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.779+0000 7f4f10aea700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f4f0c15b740 0x7f4f0c0d1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:15.784 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.779+0000 7f4f10aea700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4f0c0ce6e0 0x7f4f0c0cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:15.784 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.779+0000 7f4f10aea700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f0c0cf190 con 0x7f4f0c0d0c70 2023-10-14T10:36:15.785 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.779+0000 7f4f10aea700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f0c15bb20 con 0x7f4f0c0ce6e0 2023-10-14T10:36:15.785 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.779+0000 7f4f10aea700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f0c0cbd40 con 0x7f4f0c15b740 2023-10-14T10:36:15.785 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.779+0000 7f4f0ad9d700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4f0c0ce6e0 0x7f4f0c0cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:15.785 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.779+0000 7f4f0ad9d700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4f0c0ce6e0 0x7f4f0c0cf690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44430/0 (socket says 172.21.15.67:44430) 2023-10-14T10:36:15.785 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.779+0000 7f4f0ad9d700 1 -- 172.21.15.67:0/3175913212 learned_addr learned my addr 172.21.15.67:0/3175913212 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:15.786 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.780+0000 7f4f0a59c700 1 --2- 172.21.15.67:0/3175913212 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f4f0c15b740 0x7f4f0c0d1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:15.786 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.780+0000 7f4f0ad9d700 1 -- 172.21.15.67:0/3175913212 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f4f0c15b740 msgr2=0x7f4f0c0d1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:15.786 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.780+0000 7f4f0ad9d700 1 --2- 172.21.15.67:0/3175913212 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f4f0c15b740 0x7f4f0c0d1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:15.786 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.780+0000 7f4f0ad9d700 1 -- 172.21.15.67:0/3175913212 >> v1:172.21.15.72:6789/0 conn(0x7f4f0c0d0c70 legacy=0x7f4f0c0cdfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:15.787 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.780+0000 7f4f0ad9d700 1 -- 172.21.15.67:0/3175913212 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f0c0cc020 con 0x7f4f0c0ce6e0 2023-10-14T10:36:15.787 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.780+0000 7f4f0a59c700 1 --2- 172.21.15.67:0/3175913212 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f4f0c15b740 0x7f4f0c0d1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-14T10:36:15.787 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.780+0000 7f4f0ad9d700 1 --2- 172.21.15.67:0/3175913212 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4f0c0ce6e0 0x7f4f0c0cf690 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f4f04009e50 tx=0x7f4f0400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3f144b659eaba8e server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:15.787 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.780+0000 7f4f0959a700 1 -- 172.21.15.67:0/3175913212 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f0400e070 con 0x7f4f0c0ce6e0 2023-10-14T10:36:15.788 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.780+0000 7f4f0959a700 1 -- 172.21.15.67:0/3175913212 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f0400b9c0 con 0x7f4f0c0ce6e0 2023-10-14T10:36:15.788 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.780+0000 7f4f0959a700 1 -- 172.21.15.67:0/3175913212 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f0400bd70 con 0x7f4f0c0ce6e0 2023-10-14T10:36:15.788 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.781+0000 7f4f10aea700 1 -- 172.21.15.67:0/3175913212 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4f0c0ce6e0 msgr2=0x7f4f0c0cf690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:15.788 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.781+0000 7f4f10aea700 1 --2- 172.21.15.67:0/3175913212 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4f0c0ce6e0 0x7f4f0c0cf690 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f4f04009e50 tx=0x7f4f0400b330 comp rx=0 tx=0).stop 2023-10-14T10:36:15.789 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.781+0000 7f4f10aea700 1 -- 172.21.15.67:0/3175913212 shutdown_connections 2023-10-14T10:36:15.789 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.781+0000 7f4f10aea700 1 --2- 172.21.15.67:0/3175913212 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4f0c0ce6e0 0x7f4f0c0cf690 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:15.789 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.781+0000 7f4f10aea700 1 --2- 172.21.15.67:0/3175913212 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f4f0c15b740 0x7f4f0c0d1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:15.789 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.781+0000 7f4f10aea700 1 -- 172.21.15.67:0/3175913212 >> 172.21.15.67:0/3175913212 conn(0x7f4f0c0665a0 msgr2=0x7f4f0c066980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:15.789 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.781+0000 7f4f10aea700 1 -- 172.21.15.67:0/3175913212 shutdown_connections 2023-10-14T10:36:15.790 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.781+0000 7f4f10aea700 1 -- 172.21.15.67:0/3175913212 wait complete. 2023-10-14T10:36:15.790 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.782+0000 7f4f10aea700 1 Processor -- start 2023-10-14T10:36:15.790 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.782+0000 7f4f10aea700 1 -- start start 2023-10-14T10:36:15.793 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.783+0000 7f4f10aea700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4f0c0ce6e0 0x7f4f0c158e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:15.794 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.783+0000 7f4f10aea700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f4f0c0d0c70 0x7f4f0c0aa8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:15.794 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.783+0000 7f4f10aea700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4f0c15b740 0x7f4f0c0a87e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:15.794 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.783+0000 7f4f10aea700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f0c0a92a0 con 0x7f4f0c15b740 2023-10-14T10:36:15.794 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.783+0000 7f4f10aea700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f0c0c9100 con 0x7f4f0c0ce6e0 2023-10-14T10:36:15.794 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.783+0000 7f4f10aea700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f0c0c93e0 con 0x7f4f0c0d0c70 2023-10-14T10:36:15.795 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.783+0000 7f4f0ad9d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4f0c15b740 0x7f4f0c0a87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:15.795 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.783+0000 7f4f0ad9d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4f0c15b740 0x7f4f0c0a87e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44434/0 (socket says 172.21.15.67:44434) 2023-10-14T10:36:15.795 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.783+0000 7f4f0ad9d700 1 -- 172.21.15.67:0/1180916082 learned_addr learned my addr 172.21.15.67:0/1180916082 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:15.795 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.783+0000 7f4f0a59c700 1 --2- 172.21.15.67:0/1180916082 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4f0c0ce6e0 0x7f4f0c158e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:15.796 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.783+0000 7f4f09d9b700 1 --2- 172.21.15.67:0/1180916082 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f4f0c0d0c70 0x7f4f0c0aa8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:15.796 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.783+0000 7f4f0ad9d700 1 -- 172.21.15.67:0/1180916082 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f4f0c0d0c70 msgr2=0x7f4f0c0aa8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:15.796 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.783+0000 7f4f0ad9d700 1 --2- 172.21.15.67:0/1180916082 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f4f0c0d0c70 0x7f4f0c0aa8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:15.796 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.783+0000 7f4f0ad9d700 1 -- 172.21.15.67:0/1180916082 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4f0c0ce6e0 msgr2=0x7f4f0c158e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:15.796 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.783+0000 7f4f0ad9d700 1 --2- 172.21.15.67:0/1180916082 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4f0c0ce6e0 0x7f4f0c158e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:15.797 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.783+0000 7f4f0ad9d700 1 -- 172.21.15.67:0/1180916082 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f04009b30 con 0x7f4f0c15b740 2023-10-14T10:36:15.797 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.784+0000 7f4f0ad9d700 1 --2- 172.21.15.67:0/1180916082 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4f0c15b740 0x7f4f0c0a87e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4f04012000 tx=0x7f4f04004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:15.797 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.784+0000 7f4efaffd700 1 -- 172.21.15.67:0/1180916082 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f04004800 con 0x7f4f0c15b740 2023-10-14T10:36:15.797 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.784+0000 7f4efaffd700 1 -- 172.21.15.67:0/1180916082 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f04004960 con 0x7f4f0c15b740 2023-10-14T10:36:15.798 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.784+0000 7f4f10aea700 1 -- 172.21.15.67:0/1180916082 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f0c0c96c0 con 0x7f4f0c15b740 2023-10-14T10:36:15.798 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.784+0000 7f4efaffd700 1 -- 172.21.15.67:0/1180916082 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f0401c680 con 0x7f4f0c15b740 2023-10-14T10:36:15.798 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.784+0000 7f4f10aea700 1 -- 172.21.15.67:0/1180916082 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f0c159380 con 0x7f4f0c15b740 2023-10-14T10:36:15.798 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.785+0000 7f4f10aea700 1 -- 172.21.15.67:0/1180916082 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f0c0c9b80 con 0x7f4f0c15b740 2023-10-14T10:36:15.799 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.785+0000 7f4efaffd700 1 -- 172.21.15.67:0/1180916082 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f4f0401c7e0 con 0x7f4f0c15b740 2023-10-14T10:36:15.799 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.785+0000 7f4efaffd700 1 --2- 172.21.15.67:0/1180916082 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f4eec06af30 0x7f4eec06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:15.799 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.786+0000 7f4efaffd700 1 -- 172.21.15.67:0/1180916082 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4f0408adb0 con 0x7f4f0c15b740 2023-10-14T10:36:15.799 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.786+0000 7f4f0a59c700 1 --2- 172.21.15.67:0/1180916082 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f4eec06af30 0x7f4eec06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:15.800 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.786+0000 7f4f0a59c700 1 --2- 172.21.15.67:0/1180916082 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f4eec06af30 0x7f4eec06d3f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c066f80 tx=0x7f4f00008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:15.800 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.790+0000 7f4efaffd700 1 -- 172.21.15.67:0/1180916082 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4f04055ef0 con 0x7f4f0c15b740 2023-10-14T10:36:15.994 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.992+0000 7f4f10aea700 1 -- 172.21.15.67:0/1180916082 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4f0c067d20 con 0x7f4f0c15b740 2023-10-14T10:36:15.995 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.993+0000 7f4efaffd700 1 -- 172.21.15.67:0/1180916082 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16457 (secure 0 0 0) 0x7f4f04059510 con 0x7f4f0c15b740 2023-10-14T10:36:15.996 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-14T10:36:15.996 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":22,"fsid":"37f9ab46-3082-4b71-8b24-08d39897b008","created":"2023-10-14T10:35:46.067554+0000","modified":"2023-10-14T10:36:04.837296+0000","last_up_change":"2023-10-14T10:35:56.668142+0000","last_in_change":"2023-10-14T10:35:48.551490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-14T10:35:57.809566+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-14T10:36:01.667146+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-14T10:36:02.219700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2b59a07e-11ef-440c-b766-9f64ac885838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6801","nonce":1287255499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6803","nonce":1287255499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6807","nonce":1287255499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6805","nonce":1287255499}]},"public_addr":"172.21.15.67:6801/1287255499","cluster_addr":"172.21.15.67:6803/1287255499","heartbeat_back_addr":"172.21.15.67:6807/1287255499","heartbeat_front_addr":"172.21.15.67:6805/1287255499","state":["exists","up"]},{"osd":1,"uuid":"6f813bd6-fa0b-487d-a204-431eb1276a60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6818","nonce":2435538787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6822","nonce":2435538787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6830","nonce":2435538787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6826","nonce":2435538787}]},"public_addr":"172.21.15.67:6818/2435538787","cluster_addr":"172.21.15.67:6822/2435538787","heartbeat_back_addr":"172.21.15.67:6830/2435538787","heartbeat_front_addr":"172.21.15.67:6826/2435538787","state":["exists","up"]},{"osd":2,"uuid":"6af42c6a-1b59-4669-9c04-b6876788446d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6809","nonce":2827491791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6811","nonce":2827491791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6815","nonce":2827491791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6813","nonce":2827491791}]},"public_addr":"172.21.15.67:6809/2827491791","cluster_addr":"172.21.15.67:6811/2827491791","heartbeat_back_addr":"172.21.15.67:6815/2827491791","heartbeat_front_addr":"172.21.15.67:6813/2827491791","state":["exists","up"]},{"osd":3,"uuid":"1d889d76-a646-499d-bfff-ed5bae5434ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6819","nonce":1848001176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6821","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6823","nonce":1848001176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6831","nonce":1848001176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6825","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6827","nonce":1848001176}]},"public_addr":"172.21.15.67:6819/1848001176","cluster_addr":"172.21.15.67:6823/1848001176","heartbeat_back_addr":"172.21.15.67:6831/1848001176","heartbeat_front_addr":"172.21.15.67:6827/1848001176","state":["exists","up"]},{"osd":4,"uuid":"c3271dff-b437-4fdd-b13b-6e49a35473af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6804","nonce":1891230785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6808","nonce":1891230785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6815","nonce":1891230785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6811","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6813","nonce":1891230785}]},"public_addr":"172.21.15.72:6804/1891230785","cluster_addr":"172.21.15.72:6808/1891230785","heartbeat_back_addr":"172.21.15.72:6815/1891230785","heartbeat_front_addr":"172.21.15.72:6813/1891230785","state":["exists","up"]},{"osd":5,"uuid":"0337bb18-4b68-4eff-b528-a667275e6926","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6801","nonce":569686991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6803","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6805","nonce":569686991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6812","nonce":569686991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6807","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6809","nonce":569686991}]},"public_addr":"172.21.15.72:6801/569686991","cluster_addr":"172.21.15.72:6805/569686991","heartbeat_back_addr":"172.21.15.72:6812/569686991","heartbeat_front_addr":"172.21.15.72:6809/569686991","state":["exists","up"]},{"osd":6,"uuid":"8e92d9cf-0c8c-46eb-89a7-db2023ef1509","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6817","nonce":3767685934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6819","nonce":3767685934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6823","nonce":3767685934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6821","nonce":3767685934}]},"public_addr":"172.21.15.72:6817/3767685934","cluster_addr":"172.21.15.72:6819/3767685934","heartbeat_back_addr":"172.21.15.72:6823/3767685934","heartbeat_front_addr":"172.21.15.72:6821/3767685934","state":["exists","up"]},{"osd":7,"uuid":"3fd6aa7d-50a9-45c9-9561-5902a044a119","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6825","nonce":3567465926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6827","nonce":3567465926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6831","nonce":3567465926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6829","nonce":3567465926}]},"public_addr":"172.21.15.72:6825/3567465926","cluster_addr":"172.21.15.72:6827/3567465926","heartbeat_back_addr":"172.21.15.72:6831/3567465926","heartbeat_front_addr":"172.21.15.72:6829/3567465926","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.511466+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.795107+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.858674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.728895+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.794842+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.750064+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.732122+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.765185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-14T10:36:15.998 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.996+0000 7f4f10aea700 1 -- 172.21.15.67:0/1180916082 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f4eec06af30 msgr2=0x7f4eec06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:15.998 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.996+0000 7f4f10aea700 1 --2- 172.21.15.67:0/1180916082 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f4eec06af30 0x7f4eec06d3f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c066f80 tx=0x7f4f00008040 comp rx=0 tx=0).stop 2023-10-14T10:36:15.998 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.996+0000 7f4f10aea700 1 -- 172.21.15.67:0/1180916082 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4f0c15b740 msgr2=0x7f4f0c0a87e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:15.999 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.996+0000 7f4f10aea700 1 --2- 172.21.15.67:0/1180916082 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4f0c15b740 0x7f4f0c0a87e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4f04012000 tx=0x7f4f04004ad0 comp rx=0 tx=0).stop 2023-10-14T10:36:15.999 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.996+0000 7f4f10aea700 1 -- 172.21.15.67:0/1180916082 shutdown_connections 2023-10-14T10:36:15.999 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.996+0000 7f4f10aea700 1 --2- 172.21.15.67:0/1180916082 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f4eec06af30 0x7f4eec06d3f0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:15.999 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.996+0000 7f4f10aea700 1 --2- 172.21.15.67:0/1180916082 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4f0c15b740 0x7f4f0c0a87e0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:15.999 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.996+0000 7f4f10aea700 1 --2- 172.21.15.67:0/1180916082 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4f0c0ce6e0 0x7f4f0c158e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:16.000 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.996+0000 7f4f10aea700 1 --2- 172.21.15.67:0/1180916082 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f4f0c0d0c70 0x7f4f0c0aa8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:16.000 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.996+0000 7f4f10aea700 1 -- 172.21.15.67:0/1180916082 >> 172.21.15.67:0/1180916082 conn(0x7f4f0c0665a0 msgr2=0x7f4f0c0c6fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:16.000 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.996+0000 7f4f10aea700 1 -- 172.21.15.67:0/1180916082 shutdown_connections 2023-10-14T10:36:16.000 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:15.996+0000 7f4f10aea700 1 -- 172.21.15.67:0/1180916082 wait complete. 2023-10-14T10:36:16.010 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-14T10:35:57.809566+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-14T10:36:01.667146+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-14T10:36:02.219700+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-14T10:36:16.011 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-14T10:36:16.139 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.137+0000 7efd34d2e700 1 Processor -- start 2023-10-14T10:36:16.139 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.137+0000 7efd34d2e700 1 -- start start 2023-10-14T10:36:16.142 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.138+0000 7efd34d2e700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7efd300cd950 0x7efd3015e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:16.142 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.138+0000 7efd34d2e700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7efd300cede0 0x7efd3015f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:16.142 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.138+0000 7efd34d2e700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd300cf260 con 0x7efd3006a840 2023-10-14T10:36:16.143 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.138+0000 7efd34d2e700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd301619a0 con 0x7efd300cede0 2023-10-14T10:36:16.143 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.138+0000 7efd34d2e700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd300b2d60 con 0x7efd300cd950 2023-10-14T10:36:16.143 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.138+0000 7efd2dd9b700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7efd300cede0 0x7efd3015f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:16.143 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.138+0000 7efd2dd9b700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7efd300cede0 0x7efd3015f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44442/0 (socket says 172.21.15.67:44442) 2023-10-14T10:36:16.143 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.138+0000 7efd2dd9b700 1 -- 172.21.15.67:0/2642410266 learned_addr learned my addr 172.21.15.67:0/2642410266 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:16.143 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.138+0000 7efd2e59c700 1 --2- 172.21.15.67:0/2642410266 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7efd300cd950 0x7efd3015e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:16.144 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.138+0000 7efd2dd9b700 1 -- 172.21.15.67:0/2642410266 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7efd300cd950 msgr2=0x7efd3015e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:16.144 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.138+0000 7efd2dd9b700 1 --2- 172.21.15.67:0/2642410266 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7efd300cd950 0x7efd3015e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:16.144 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.138+0000 7efd2dd9b700 1 -- 172.21.15.67:0/2642410266 >> v1:172.21.15.72:6789/0 conn(0x7efd3006a840 legacy=0x7efd30161df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:16.144 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.138+0000 7efd2dd9b700 1 -- 172.21.15.67:0/2642410266 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd300cdd30 con 0x7efd300cede0 2023-10-14T10:36:16.144 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.138+0000 7efd2dd9b700 1 --2- 172.21.15.67:0/2642410266 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd300cede0 0x7efd3015f570 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7efd24006fd0 tx=0x7efd24009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8d64f007862d205 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:16.145 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.139+0000 7efd2d59a700 1 -- 172.21.15.67:0/2642410266 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd24005e30 con 0x7efd300cede0 2023-10-14T10:36:16.145 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.139+0000 7efd2d59a700 1 -- 172.21.15.67:0/2642410266 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd24004030 con 0x7efd300cede0 2023-10-14T10:36:16.145 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.139+0000 7efd2d59a700 1 -- 172.21.15.67:0/2642410266 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd240043e0 con 0x7efd300cede0 2023-10-14T10:36:16.145 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.139+0000 7efd34d2e700 1 -- 172.21.15.67:0/2642410266 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd300cede0 msgr2=0x7efd3015f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:16.145 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.139+0000 7efd34d2e700 1 --2- 172.21.15.67:0/2642410266 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd300cede0 0x7efd3015f570 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7efd24006fd0 tx=0x7efd24009bd0 comp rx=0 tx=0).stop 2023-10-14T10:36:16.146 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.139+0000 7efd34d2e700 1 -- 172.21.15.67:0/2642410266 shutdown_connections 2023-10-14T10:36:16.146 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.139+0000 7efd34d2e700 1 --2- 172.21.15.67:0/2642410266 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7efd300cd950 0x7efd3015e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:16.146 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.139+0000 7efd34d2e700 1 --2- 172.21.15.67:0/2642410266 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd300cede0 0x7efd3015f570 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:16.146 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.139+0000 7efd34d2e700 1 -- 172.21.15.67:0/2642410266 >> 172.21.15.67:0/2642410266 conn(0x7efd3006dae0 msgr2=0x7efd3006def0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:16.146 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.139+0000 7efd34d2e700 1 -- 172.21.15.67:0/2642410266 shutdown_connections 2023-10-14T10:36:16.147 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.139+0000 7efd34d2e700 1 -- 172.21.15.67:0/2642410266 wait complete. 2023-10-14T10:36:16.150 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.141+0000 7efd34d2e700 1 Processor -- start 2023-10-14T10:36:16.150 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.141+0000 7efd34d2e700 1 -- start start 2023-10-14T10:36:16.150 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.141+0000 7efd34d2e700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd3006a840 0x7efd300af360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:16.150 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.141+0000 7efd34d2e700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7efd300cd950 0x7efd300b5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:16.151 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.141+0000 7efd34d2e700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efd300cede0 0x7efd300c6870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:16.151 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.141+0000 7efd34d2e700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd301f78e0 con 0x7efd3006a840 2023-10-14T10:36:16.151 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.142+0000 7efd34d2e700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd300b6060 con 0x7efd300cede0 2023-10-14T10:36:16.151 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.142+0000 7efd34d2e700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd300aeed0 con 0x7efd300cd950 2023-10-14T10:36:16.151 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.142+0000 7efd2e59c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd3006a840 0x7efd300af360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:16.152 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.142+0000 7efd2e59c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd3006a840 0x7efd300af360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44446/0 (socket says 172.21.15.67:44446) 2023-10-14T10:36:16.152 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.142+0000 7efd2e59c700 1 -- 172.21.15.67:0/4082759009 learned_addr learned my addr 172.21.15.67:0/4082759009 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:16.152 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.142+0000 7efd2dd9b700 1 --2- 172.21.15.67:0/4082759009 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7efd300cd950 0x7efd300b5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:16.152 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.142+0000 7efd2ed9d700 1 --2- 172.21.15.67:0/4082759009 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efd300cede0 0x7efd300c6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:16.152 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.142+0000 7efd2e59c700 1 -- 172.21.15.67:0/4082759009 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7efd300cd950 msgr2=0x7efd300b5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:16.153 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.142+0000 7efd2e59c700 1 --2- 172.21.15.67:0/4082759009 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7efd300cd950 0x7efd300b5b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:16.153 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.142+0000 7efd2e59c700 1 -- 172.21.15.67:0/4082759009 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efd300cede0 msgr2=0x7efd300c6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:16.153 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.142+0000 7efd2e59c700 1 --2- 172.21.15.67:0/4082759009 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efd300cede0 0x7efd300c6870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:16.153 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.142+0000 7efd2e59c700 1 -- 172.21.15.67:0/4082759009 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd24009500 con 0x7efd3006a840 2023-10-14T10:36:16.153 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.142+0000 7efd2dd9b700 1 --2- 172.21.15.67:0/4082759009 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7efd300cd950 0x7efd300b5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-14T10:36:16.154 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.142+0000 7efd2ed9d700 1 --2- 172.21.15.67:0/4082759009 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efd300cede0 0x7efd300c6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-14T10:36:16.154 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.142+0000 7efd2e59c700 1 --2- 172.21.15.67:0/4082759009 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd3006a840 0x7efd300af360 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7efd2000e9b0 tx=0x7efd2000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:16.154 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.142+0000 7efd1effd700 1 -- 172.21.15.67:0/4082759009 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd20009710 con 0x7efd3006a840 2023-10-14T10:36:16.154 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.142+0000 7efd1effd700 1 -- 172.21.15.67:0/4082759009 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd2000c850 con 0x7efd3006a840 2023-10-14T10:36:16.155 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.142+0000 7efd34d2e700 1 -- 172.21.15.67:0/4082759009 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd300ae2c0 con 0x7efd3006a840 2023-10-14T10:36:16.155 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.142+0000 7efd1effd700 1 -- 172.21.15.67:0/4082759009 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd2000cbc0 con 0x7efd3006a840 2023-10-14T10:36:16.155 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.142+0000 7efd34d2e700 1 -- 172.21.15.67:0/4082759009 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd300ae880 con 0x7efd3006a840 2023-10-14T10:36:16.155 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.143+0000 7efd34d2e700 1 -- 172.21.15.67:0/4082759009 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd30067d20 con 0x7efd3006a840 2023-10-14T10:36:16.155 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.144+0000 7efd1effd700 1 -- 172.21.15.67:0/4082759009 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7efd2000cd20 con 0x7efd3006a840 2023-10-14T10:36:16.155 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.144+0000 7efd1effd700 1 --2- 172.21.15.67:0/4082759009 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7efd1006af00 0x7efd1006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:16.156 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.144+0000 7efd1effd700 1 -- 172.21.15.67:0/4082759009 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efd200896d0 con 0x7efd3006a840 2023-10-14T10:36:16.156 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.144+0000 7efd2dd9b700 1 --2- 172.21.15.67:0/4082759009 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7efd1006af00 0x7efd1006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:16.156 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.145+0000 7efd2dd9b700 1 --2- 172.21.15.67:0/4082759009 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7efd1006af00 0x7efd1006d3c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7efd24006fd0 tx=0x7efd24008070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:16.157 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.149+0000 7efd1effd700 1 -- 172.21.15.67:0/4082759009 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7efd20054780 con 0x7efd3006a840 2023-10-14T10:36:16.349 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.347+0000 7efd34d2e700 1 -- 172.21.15.67:0/4082759009 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efd300afbb0 con 0x7efd3006a840 2023-10-14T10:36:16.350 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.348+0000 7efd1effd700 1 -- 172.21.15.67:0/4082759009 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7efd20057da0 con 0x7efd3006a840 2023-10-14T10:36:16.350 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2023-10-14T10:36:16.352 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.350+0000 7efd34d2e700 1 -- 172.21.15.67:0/4082759009 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7efd1006af00 msgr2=0x7efd1006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:16.352 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.350+0000 7efd34d2e700 1 --2- 172.21.15.67:0/4082759009 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7efd1006af00 0x7efd1006d3c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7efd24006fd0 tx=0x7efd24008070 comp rx=0 tx=0).stop 2023-10-14T10:36:16.353 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.350+0000 7efd34d2e700 1 -- 172.21.15.67:0/4082759009 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd3006a840 msgr2=0x7efd300af360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:16.353 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.350+0000 7efd34d2e700 1 --2- 172.21.15.67:0/4082759009 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd3006a840 0x7efd300af360 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7efd2000e9b0 tx=0x7efd2000ed70 comp rx=0 tx=0).stop 2023-10-14T10:36:16.353 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.350+0000 7efd34d2e700 1 -- 172.21.15.67:0/4082759009 shutdown_connections 2023-10-14T10:36:16.353 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.350+0000 7efd34d2e700 1 --2- 172.21.15.67:0/4082759009 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7efd1006af00 0x7efd1006d3c0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:16.353 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.350+0000 7efd34d2e700 1 --2- 172.21.15.67:0/4082759009 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efd300cede0 0x7efd300c6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:16.354 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.350+0000 7efd34d2e700 1 --2- 172.21.15.67:0/4082759009 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd3006a840 0x7efd300af360 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:16.354 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.350+0000 7efd34d2e700 1 --2- 172.21.15.67:0/4082759009 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7efd300cd950 0x7efd300b5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:16.354 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.350+0000 7efd34d2e700 1 -- 172.21.15.67:0/4082759009 >> 172.21.15.67:0/4082759009 conn(0x7efd3006dae0 msgr2=0x7efd300b05d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:16.354 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.350+0000 7efd34d2e700 1 -- 172.21.15.67:0/4082759009 shutdown_connections 2023-10-14T10:36:16.354 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.350+0000 7efd34d2e700 1 -- 172.21.15.67:0/4082759009 wait complete. 2023-10-14T10:36:16.364 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-14T10:36:16.493 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.491+0000 7fbacca99700 1 Processor -- start 2023-10-14T10:36:16.493 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.491+0000 7fbacca99700 1 -- start start 2023-10-14T10:36:16.496 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.492+0000 7fbacca99700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fbac815f130 0x7fbac80cede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:16.496 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.492+0000 7fbacca99700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fbac806af50 0x7fbac80b3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:16.496 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.492+0000 7fbacca99700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbac80b3530 con 0x7fbac80b28b0 2023-10-14T10:36:16.496 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.492+0000 7fbacca99700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbac80cde00 con 0x7fbac815f130 2023-10-14T10:36:16.497 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.492+0000 7fbacca99700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbac80cd180 con 0x7fbac806af50 2023-10-14T10:36:16.497 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.492+0000 7fbac659c700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fbac815f130 0x7fbac80cede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:16.497 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.492+0000 7fbac5d9b700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fbac80b28b0 0x7fbac806a840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:52222/0 (socket says 172.21.15.67:52222) 2023-10-14T10:36:16.497 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.492+0000 7fbac5d9b700 1 -- 172.21.15.67:0/163905367 learned_addr learned my addr 172.21.15.67:0/163905367 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:36:16.497 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.492+0000 7fbac6d9d700 1 --2- 172.21.15.67:0/163905367 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fbac806af50 0x7fbac80b3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:16.498 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.493+0000 7fbac659c700 1 -- 172.21.15.67:0/163905367 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fbac806af50 msgr2=0x7fbac80b3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:16.498 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.493+0000 7fbac659c700 1 --2- 172.21.15.67:0/163905367 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fbac806af50 0x7fbac80b3960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:16.498 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.493+0000 7fbac659c700 1 -- 172.21.15.67:0/163905367 >> v1:172.21.15.67:6789/0 conn(0x7fbac80b28b0 legacy=0x7fbac806a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:16.498 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.493+0000 7fbac559a700 1 -- 172.21.15.67:0/163905367 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3694808745 0 0) 0x7fbac80b3530 con 0x7fbac80b28b0 2023-10-14T10:36:16.499 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.493+0000 7fbac659c700 1 -- 172.21.15.67:0/163905367 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbac80cd460 con 0x7fbac815f130 2023-10-14T10:36:16.499 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.493+0000 7fbac659c700 1 --2- 172.21.15.67:0/163905367 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbac815f130 0x7fbac80cede0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fbabc009840 tx=0x7fbabc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed607610fa50998a server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:16.499 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.493+0000 7fbac559a700 1 -- 172.21.15.67:0/163905367 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbabc011830 con 0x7fbac815f130 2023-10-14T10:36:16.499 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.493+0000 7fbac559a700 1 -- 172.21.15.67:0/163905367 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbabc011990 con 0x7fbac815f130 2023-10-14T10:36:16.499 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.493+0000 7fbac559a700 1 -- 172.21.15.67:0/163905367 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbabc011d20 con 0x7fbac815f130 2023-10-14T10:36:16.500 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.493+0000 7fbacca99700 1 -- 172.21.15.67:0/163905367 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbac815f130 msgr2=0x7fbac80cede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:16.500 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.493+0000 7fbacca99700 1 --2- 172.21.15.67:0/163905367 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbac815f130 0x7fbac80cede0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fbabc009840 tx=0x7fbabc009b50 comp rx=0 tx=0).stop 2023-10-14T10:36:16.500 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.493+0000 7fbacca99700 1 -- 172.21.15.67:0/163905367 shutdown_connections 2023-10-14T10:36:16.500 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.493+0000 7fbacca99700 1 --2- 172.21.15.67:0/163905367 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fbac806af50 0x7fbac80b3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:16.500 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.493+0000 7fbacca99700 1 --2- 172.21.15.67:0/163905367 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbac815f130 0x7fbac80cede0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:16.501 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.493+0000 7fbacca99700 1 -- 172.21.15.67:0/163905367 >> 172.21.15.67:0/163905367 conn(0x7fbac806dae0 msgr2=0x7fbac806def0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:16.501 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.494+0000 7fbacca99700 1 -- 172.21.15.67:0/163905367 shutdown_connections 2023-10-14T10:36:16.501 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.494+0000 7fbacca99700 1 -- 172.21.15.67:0/163905367 wait complete. 2023-10-14T10:36:16.503 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.495+0000 7fbacca99700 1 Processor -- start 2023-10-14T10:36:16.504 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.495+0000 7fbacca99700 1 -- start start 2023-10-14T10:36:16.504 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.496+0000 7fbacca99700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbac806af50 0x7fbac80c8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:16.504 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.496+0000 7fbacca99700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fbac80b28b0 0x7fbac80c8560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:16.504 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.496+0000 7fbacca99700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbac815f130 0x7fbac80c9200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:16.504 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.496+0000 7fbacca99700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbac80c9070 con 0x7fbac806af50 2023-10-14T10:36:16.505 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.496+0000 7fbacca99700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbac80c8d90 con 0x7fbac815f130 2023-10-14T10:36:16.505 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.496+0000 7fbacca99700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbac80c41a0 con 0x7fbac80b28b0 2023-10-14T10:36:16.505 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.496+0000 7fbac659c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbac806af50 0x7fbac80c8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:16.505 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.496+0000 7fbac659c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbac806af50 0x7fbac80c8020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44460/0 (socket says 172.21.15.67:44460) 2023-10-14T10:36:16.505 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.496+0000 7fbac659c700 1 -- 172.21.15.67:0/1321912203 learned_addr learned my addr 172.21.15.67:0/1321912203 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:16.506 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.497+0000 7fbac659c700 1 -- 172.21.15.67:0/1321912203 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fbac80b28b0 msgr2=0x7fbac80c8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:16.506 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.497+0000 7fbac659c700 1 --2- 172.21.15.67:0/1321912203 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fbac80b28b0 0x7fbac80c8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:16.506 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.497+0000 7fbac659c700 1 -- 172.21.15.67:0/1321912203 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbac815f130 msgr2=0x7fbac80c9200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:16.506 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.497+0000 7fbac659c700 1 --2- 172.21.15.67:0/1321912203 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbac815f130 0x7fbac80c9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:16.506 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.497+0000 7fbac659c700 1 -- 172.21.15.67:0/1321912203 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbabc0094a0 con 0x7fbac806af50 2023-10-14T10:36:16.507 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.497+0000 7fbac659c700 1 --2- 172.21.15.67:0/1321912203 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbac806af50 0x7fbac80c8020 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fbabc0090b0 tx=0x7fbabc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:16.507 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.497+0000 7fbab6ffd700 1 -- 172.21.15.67:0/1321912203 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbabc0038a0 con 0x7fbac806af50 2023-10-14T10:36:16.507 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.497+0000 7fbab6ffd700 1 -- 172.21.15.67:0/1321912203 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbabc01f070 con 0x7fbac806af50 2023-10-14T10:36:16.507 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.497+0000 7fbab6ffd700 1 -- 172.21.15.67:0/1321912203 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbabc003c20 con 0x7fbac806af50 2023-10-14T10:36:16.507 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.497+0000 7fbacca99700 1 -- 172.21.15.67:0/1321912203 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbac80c4480 con 0x7fbac806af50 2023-10-14T10:36:16.508 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.497+0000 7fbacca99700 1 -- 172.21.15.67:0/1321912203 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbac80c8aa0 con 0x7fbac806af50 2023-10-14T10:36:16.508 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.498+0000 7fbacca99700 1 -- 172.21.15.67:0/1321912203 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbac80c49a0 con 0x7fbac806af50 2023-10-14T10:36:16.508 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.498+0000 7fbab6ffd700 1 -- 172.21.15.67:0/1321912203 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fbabc01a430 con 0x7fbac806af50 2023-10-14T10:36:16.508 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.499+0000 7fbab6ffd700 1 --2- 172.21.15.67:0/1321912203 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fbaa806af00 0x7fbaa806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:16.508 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.499+0000 7fbab6ffd700 1 -- 172.21.15.67:0/1321912203 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbabc021070 con 0x7fbac806af50 2023-10-14T10:36:16.509 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.499+0000 7fbac5d9b700 1 --2- 172.21.15.67:0/1321912203 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fbaa806af00 0x7fbaa806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:16.509 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.500+0000 7fbac5d9b700 1 --2- 172.21.15.67:0/1321912203 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fbaa806af00 0x7fbaa806d3c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fbab80010e0 tx=0x7fbab8007850 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:16.509 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.503+0000 7fbab6ffd700 1 -- 172.21.15.67:0/1321912203 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbac80c49a0 con 0x7fbac806af50 2023-10-14T10:36:16.703 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.701+0000 7fbacca99700 1 -- 172.21.15.67:0/1321912203 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbac80c49a0 con 0x7fbac806af50 2023-10-14T10:36:16.703 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.701+0000 7fbab6ffd700 1 -- 172.21.15.67:0/1321912203 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbabc0560f0 con 0x7fbac806af50 2023-10-14T10:36:16.704 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2023-10-14T10:36:16.706 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.703+0000 7fbacca99700 1 -- 172.21.15.67:0/1321912203 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fbaa806af00 msgr2=0x7fbaa806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:16.706 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.703+0000 7fbacca99700 1 --2- 172.21.15.67:0/1321912203 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fbaa806af00 0x7fbaa806d3c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fbab80010e0 tx=0x7fbab8007850 comp rx=0 tx=0).stop 2023-10-14T10:36:16.706 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.703+0000 7fbacca99700 1 -- 172.21.15.67:0/1321912203 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbac806af50 msgr2=0x7fbac80c8020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:16.706 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.703+0000 7fbacca99700 1 --2- 172.21.15.67:0/1321912203 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbac806af50 0x7fbac80c8020 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fbabc0090b0 tx=0x7fbabc004750 comp rx=0 tx=0).stop 2023-10-14T10:36:16.707 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.703+0000 7fbacca99700 1 -- 172.21.15.67:0/1321912203 shutdown_connections 2023-10-14T10:36:16.707 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.703+0000 7fbacca99700 1 --2- 172.21.15.67:0/1321912203 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fbaa806af00 0x7fbaa806d3c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:16.707 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.703+0000 7fbacca99700 1 --2- 172.21.15.67:0/1321912203 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbac815f130 0x7fbac80c9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:16.707 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.703+0000 7fbacca99700 1 --2- 172.21.15.67:0/1321912203 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbac806af50 0x7fbac80c8020 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:16.707 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.703+0000 7fbacca99700 1 --2- 172.21.15.67:0/1321912203 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fbac80b28b0 0x7fbac80c8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:16.708 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.703+0000 7fbacca99700 1 -- 172.21.15.67:0/1321912203 >> 172.21.15.67:0/1321912203 conn(0x7fbac806dae0 msgr2=0x7fbac80b5e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:16.708 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.704+0000 7fbacca99700 1 -- 172.21.15.67:0/1321912203 shutdown_connections 2023-10-14T10:36:16.708 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.704+0000 7fbacca99700 1 -- 172.21.15.67:0/1321912203 wait complete. 2023-10-14T10:36:16.718 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-14T10:36:16.847 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.844+0000 7f28f71f8700 1 Processor -- start 2023-10-14T10:36:16.847 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.845+0000 7f28f71f8700 1 -- start start 2023-10-14T10:36:16.849 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.845+0000 7f28f71f8700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f28f015e030 0x7f28f00cb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:16.849 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.845+0000 7f28f71f8700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28f006adb0 con 0x7f28f006a840 2023-10-14T10:36:16.850 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.845+0000 7f28f71f8700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28f006b090 con 0x7f28f00ce280 2023-10-14T10:36:16.850 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.845+0000 7f28f71f8700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28f006b370 con 0x7f28f015e030 2023-10-14T10:36:16.850 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.845+0000 7f28f4f94700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f28f015e030 0x7f28f00cb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:16.850 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.846+0000 7f28effff700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f28f006a840 0x7f28f00cdb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:52236/0 (socket says 172.21.15.67:52236) 2023-10-14T10:36:16.851 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.846+0000 7f28effff700 1 -- 172.21.15.67:0/3497581777 learned_addr learned my addr 172.21.15.67:0/3497581777 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:36:16.851 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.846+0000 7f28f4f94700 1 -- 172.21.15.67:0/3497581777 >> v1:172.21.15.72:6789/0 conn(0x7f28f00ce280 legacy=0x7f28f006d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:16.851 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.846+0000 7f28f4f94700 1 -- 172.21.15.67:0/3497581777 >> v1:172.21.15.67:6789/0 conn(0x7f28f006a840 legacy=0x7f28f00cdb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:16.851 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.846+0000 7f28f4f94700 1 -- 172.21.15.67:0/3497581777 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28f006b650 con 0x7f28f015e030 2023-10-14T10:36:16.851 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.846+0000 7f28f4f94700 1 --2- 172.21.15.67:0/3497581777 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f28f015e030 0x7f28f00cb3a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f28e4009840 tx=0x7f28e4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97dc354b94ef71a3 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:16.852 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.846+0000 7f28ef7fe700 1 -- 172.21.15.67:0/3497581777 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e4004230 con 0x7f28f015e030 2023-10-14T10:36:16.852 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.846+0000 7f28ef7fe700 1 -- 172.21.15.67:0/3497581777 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28e4004390 con 0x7f28f015e030 2023-10-14T10:36:16.852 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.846+0000 7f28ef7fe700 1 -- 172.21.15.67:0/3497581777 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e40036a0 con 0x7f28f015e030 2023-10-14T10:36:16.852 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.847+0000 7f28f71f8700 1 -- 172.21.15.67:0/3497581777 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f28f015e030 msgr2=0x7f28f00cb3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:16.852 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.847+0000 7f28f71f8700 1 --2- 172.21.15.67:0/3497581777 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f28f015e030 0x7f28f00cb3a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f28e4009840 tx=0x7f28e4009b50 comp rx=0 tx=0).stop 2023-10-14T10:36:16.853 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.847+0000 7f28f71f8700 1 -- 172.21.15.67:0/3497581777 shutdown_connections 2023-10-14T10:36:16.853 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.847+0000 7f28f71f8700 1 --2- 172.21.15.67:0/3497581777 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f28f015e030 0x7f28f00cb3a0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:16.853 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.847+0000 7f28f71f8700 1 -- 172.21.15.67:0/3497581777 >> 172.21.15.67:0/3497581777 conn(0x7f28f00cc6f0 msgr2=0x7f28f00ccb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:16.853 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.847+0000 7f28f71f8700 1 -- 172.21.15.67:0/3497581777 shutdown_connections 2023-10-14T10:36:16.854 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.847+0000 7f28f71f8700 1 -- 172.21.15.67:0/3497581777 wait complete. 2023-10-14T10:36:16.856 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.848+0000 7f28f71f8700 1 Processor -- start 2023-10-14T10:36:16.857 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.848+0000 7f28f71f8700 1 -- start start 2023-10-14T10:36:16.857 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.849+0000 7f28f71f8700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f28f006a840 0x7f28f0160f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:16.857 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.849+0000 7f28f71f8700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f28f00ce280 0x7f28f00c9110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:16.857 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.849+0000 7f28f71f8700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f28f015e030 0x7f28f015f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:16.857 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.849+0000 7f28f71f8700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f00c9650 con 0x7f28f00ce280 2023-10-14T10:36:16.858 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.849+0000 7f28f71f8700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f00c6d40 con 0x7f28f006a840 2023-10-14T10:36:16.858 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.849+0000 7f28f71f8700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28f015fd40 con 0x7f28f015e030 2023-10-14T10:36:16.858 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.849+0000 7f28effff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f28f00ce280 0x7f28f00c9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:16.858 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.849+0000 7f28effff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f28f00ce280 0x7f28f00c9110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44476/0 (socket says 172.21.15.67:44476) 2023-10-14T10:36:16.858 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.849+0000 7f28effff700 1 -- 172.21.15.67:0/2581067709 learned_addr learned my addr 172.21.15.67:0/2581067709 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:16.859 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.849+0000 7f28f5795700 1 --2- 172.21.15.67:0/2581067709 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f28f015e030 0x7f28f015f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:16.859 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.849+0000 7f28f4f94700 1 --2- 172.21.15.67:0/2581067709 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f28f006a840 0x7f28f0160f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:16.859 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.849+0000 7f28effff700 1 -- 172.21.15.67:0/2581067709 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f28f015e030 msgr2=0x7f28f015f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:16.859 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.849+0000 7f28effff700 1 --2- 172.21.15.67:0/2581067709 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f28f015e030 0x7f28f015f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:16.860 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.849+0000 7f28effff700 1 -- 172.21.15.67:0/2581067709 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f28f006a840 msgr2=0x7f28f0160f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:16.860 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.849+0000 7f28effff700 1 --2- 172.21.15.67:0/2581067709 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f28f006a840 0x7f28f0160f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:16.860 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.849+0000 7f28effff700 1 -- 172.21.15.67:0/2581067709 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28e40094a0 con 0x7f28f00ce280 2023-10-14T10:36:16.860 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.849+0000 7f28effff700 1 --2- 172.21.15.67:0/2581067709 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f28f00ce280 0x7f28f00c9110 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f28dc00c780 tx=0x7f28dc00ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:16.860 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.850+0000 7f28ed7fa700 1 -- 172.21.15.67:0/2581067709 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28dc004c80 con 0x7f28f00ce280 2023-10-14T10:36:16.861 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.850+0000 7f28ed7fa700 1 -- 172.21.15.67:0/2581067709 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28dc004de0 con 0x7f28f00ce280 2023-10-14T10:36:16.861 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.850+0000 7f28f71f8700 1 -- 172.21.15.67:0/2581067709 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28f00c9860 con 0x7f28f00ce280 2023-10-14T10:36:16.861 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.850+0000 7f28ed7fa700 1 -- 172.21.15.67:0/2581067709 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28dc0072f0 con 0x7f28f00ce280 2023-10-14T10:36:16.861 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.850+0000 7f28f71f8700 1 -- 172.21.15.67:0/2581067709 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28f00c7270 con 0x7f28f00ce280 2023-10-14T10:36:16.861 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.850+0000 7f28f71f8700 1 -- 172.21.15.67:0/2581067709 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28f0067d20 con 0x7f28f00ce280 2023-10-14T10:36:16.862 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.851+0000 7f28ed7fa700 1 -- 172.21.15.67:0/2581067709 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f28dc007450 con 0x7f28f00ce280 2023-10-14T10:36:16.862 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.852+0000 7f28ed7fa700 1 --2- 172.21.15.67:0/2581067709 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f28d406af00 0x7f28d406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:16.862 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.852+0000 7f28ed7fa700 1 -- 172.21.15.67:0/2581067709 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f28dc0892b0 con 0x7f28f00ce280 2023-10-14T10:36:16.862 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.852+0000 7f28f4f94700 1 --2- 172.21.15.67:0/2581067709 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f28d406af00 0x7f28d406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:16.862 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.852+0000 7f28f4f94700 1 --2- 172.21.15.67:0/2581067709 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f28d406af00 0x7f28d406d3c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f28e40090b0 tx=0x7f28e4008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:16.863 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:16.856+0000 7f28ed7fa700 1 -- 172.21.15.67:0/2581067709 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f28dc054360 con 0x7f28f00ce280 2023-10-14T10:36:17.061 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.059+0000 7f28f71f8700 1 -- 172.21.15.67:0/2581067709 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f28f01614c0 con 0x7f28f00ce280 2023-10-14T10:36:17.061 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.059+0000 7f28ed7fa700 1 -- 172.21.15.67:0/2581067709 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f28dc057980 con 0x7f28f00ce280 2023-10-14T10:36:17.062 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2023-10-14T10:36:17.064 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.061+0000 7f28f71f8700 1 -- 172.21.15.67:0/2581067709 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f28d406af00 msgr2=0x7f28d406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:17.064 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.061+0000 7f28f71f8700 1 --2- 172.21.15.67:0/2581067709 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f28d406af00 0x7f28d406d3c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f28e40090b0 tx=0x7f28e4008030 comp rx=0 tx=0).stop 2023-10-14T10:36:17.064 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.061+0000 7f28f71f8700 1 -- 172.21.15.67:0/2581067709 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f28f00ce280 msgr2=0x7f28f00c9110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:17.064 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.061+0000 7f28f71f8700 1 --2- 172.21.15.67:0/2581067709 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f28f00ce280 0x7f28f00c9110 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f28dc00c780 tx=0x7f28dc00ca90 comp rx=0 tx=0).stop 2023-10-14T10:36:17.064 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.061+0000 7f28f71f8700 1 -- 172.21.15.67:0/2581067709 shutdown_connections 2023-10-14T10:36:17.065 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.061+0000 7f28f71f8700 1 --2- 172.21.15.67:0/2581067709 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f28d406af00 0x7f28d406d3c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:17.065 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.061+0000 7f28f71f8700 1 --2- 172.21.15.67:0/2581067709 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f28f015e030 0x7f28f015f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:17.065 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.061+0000 7f28f71f8700 1 --2- 172.21.15.67:0/2581067709 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f28f006a840 0x7f28f0160f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:17.065 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.061+0000 7f28f71f8700 1 --2- 172.21.15.67:0/2581067709 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f28f00ce280 0x7f28f00c9110 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:17.065 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.061+0000 7f28f71f8700 1 -- 172.21.15.67:0/2581067709 >> 172.21.15.67:0/2581067709 conn(0x7f28f00cc6f0 msgr2=0x7f28f00ae8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:17.066 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.061+0000 7f28f71f8700 1 -- 172.21.15.67:0/2581067709 shutdown_connections 2023-10-14T10:36:17.066 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.061+0000 7f28f71f8700 1 -- 172.21.15.67:0/2581067709 wait complete. 2023-10-14T10:36:17.075 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-14T10:36:17.200 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.197+0000 7f83c0f89700 1 Processor -- start 2023-10-14T10:36:17.200 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.198+0000 7f83c0f89700 1 -- start start 2023-10-14T10:36:17.203 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.198+0000 7f83c0f89700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f83bc15f1f0 0x7f83bc0cf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:17.203 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.198+0000 7f83c0f89700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83bc15e7a0 con 0x7f83bc0cfc60 2023-10-14T10:36:17.203 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.198+0000 7f83c0f89700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83bc0d0090 con 0x7f83bc15ab10 2023-10-14T10:36:17.203 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.198+0000 7f83c0f89700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83bc0b29a0 con 0x7f83bc15f1f0 2023-10-14T10:36:17.204 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.199+0000 7f83bad9d700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f83bc0cfc60 0x7f83bc0cbf40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:52254/0 (socket says 172.21.15.67:52254) 2023-10-14T10:36:17.204 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.199+0000 7f83bad9d700 1 -- 172.21.15.67:0/373859204 learned_addr learned my addr 172.21.15.67:0/373859204 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:36:17.204 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.199+0000 7f83b9d9b700 1 --2- 172.21.15.67:0/373859204 >> v2:172.21.15.72:3300/0 conn(0x7f83bc15f1f0 0x7f83bc0cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:17.205 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.199+0000 7f83b959a700 1 -- 172.21.15.67:0/373859204 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1225341325 0 0) 0x7f83bc15e7a0 con 0x7f83bc0cfc60 2023-10-14T10:36:17.205 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.199+0000 7f83b959a700 1 -- 172.21.15.67:0/373859204 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83a40035b0 con 0x7f83bc0cfc60 2023-10-14T10:36:17.205 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.199+0000 7f83b9d9b700 1 -- 172.21.15.67:0/373859204 >> v1:172.21.15.72:6789/0 conn(0x7f83bc15ab10 legacy=0x7f83bc0b1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:17.205 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.199+0000 7f83b9d9b700 1 -- 172.21.15.67:0/373859204 >> v1:172.21.15.67:6789/0 conn(0x7f83bc0cfc60 legacy=0x7f83bc0cbf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:17.206 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.199+0000 7f83b9d9b700 1 -- 172.21.15.67:0/373859204 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83bc0b2c80 con 0x7f83bc15f1f0 2023-10-14T10:36:17.206 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.199+0000 7f83b9d9b700 1 --2- 172.21.15.67:0/373859204 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f83bc15f1f0 0x7f83bc0cf720 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f83ac009820 tx=0x7f83ac009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5386019d37916c4d server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:17.206 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.199+0000 7f83b959a700 1 -- 172.21.15.67:0/373859204 <== mon.1 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83ac005e30 con 0x7f83bc15f1f0 2023-10-14T10:36:17.206 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.199+0000 7f83b959a700 1 -- 172.21.15.67:0/373859204 <== mon.1 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83ac006030 con 0x7f83bc15f1f0 2023-10-14T10:36:17.207 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.199+0000 7f83b959a700 1 -- 172.21.15.67:0/373859204 <== mon.1 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83ac0063c0 con 0x7f83bc15f1f0 2023-10-14T10:36:17.207 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.200+0000 7f83c0f89700 1 -- 172.21.15.67:0/373859204 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f83bc15f1f0 msgr2=0x7f83bc0cf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:17.207 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.200+0000 7f83c0f89700 1 --2- 172.21.15.67:0/373859204 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f83bc15f1f0 0x7f83bc0cf720 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f83ac009820 tx=0x7f83ac009b30 comp rx=0 tx=0).stop 2023-10-14T10:36:17.207 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.200+0000 7f83c0f89700 1 -- 172.21.15.67:0/373859204 shutdown_connections 2023-10-14T10:36:17.208 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.200+0000 7f83c0f89700 1 --2- 172.21.15.67:0/373859204 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f83bc15f1f0 0x7f83bc0cf720 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:17.208 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.200+0000 7f83c0f89700 1 -- 172.21.15.67:0/373859204 >> 172.21.15.67:0/373859204 conn(0x7f83bc06dae0 msgr2=0x7f83bc06def0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:17.208 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.200+0000 7f83c0f89700 1 -- 172.21.15.67:0/373859204 shutdown_connections 2023-10-14T10:36:17.208 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.200+0000 7f83c0f89700 1 -- 172.21.15.67:0/373859204 wait complete. 2023-10-14T10:36:17.212 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.201+0000 7f83c0f89700 1 Processor -- start 2023-10-14T10:36:17.212 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.201+0000 7f83c0f89700 1 -- start start 2023-10-14T10:36:17.212 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.202+0000 7f83c0f89700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83bc0cfc60 0x7f83bc157d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:17.212 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.202+0000 7f83c0f89700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f83bc15ab10 0x7f83bc0c64e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:17.213 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.202+0000 7f83c0f89700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f83bc15f1f0 0x7f83bc0c6a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:17.213 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.202+0000 7f83c0f89700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83bc0c94f0 con 0x7f83bc0cfc60 2023-10-14T10:36:17.213 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.202+0000 7f83c0f89700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83bc0c6f60 con 0x7f83bc15f1f0 2023-10-14T10:36:17.213 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.202+0000 7f83c0f89700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83bc0c4ef0 con 0x7f83bc15ab10 2023-10-14T10:36:17.214 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.202+0000 7f83ba59c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83bc0cfc60 0x7f83bc157d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:17.214 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.202+0000 7f83ba59c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83bc0cfc60 0x7f83bc157d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44488/0 (socket says 172.21.15.67:44488) 2023-10-14T10:36:17.214 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.202+0000 7f83ba59c700 1 -- 172.21.15.67:0/1632875103 learned_addr learned my addr 172.21.15.67:0/1632875103 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:17.214 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.202+0000 7f83bad9d700 1 --2- 172.21.15.67:0/1632875103 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f83bc15f1f0 0x7f83bc0c6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:17.215 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.202+0000 7f83b9d9b700 1 --2- 172.21.15.67:0/1632875103 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f83bc15ab10 0x7f83bc0c64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:17.215 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.203+0000 7f83ba59c700 1 -- 172.21.15.67:0/1632875103 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f83bc15ab10 msgr2=0x7f83bc0c64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:17.215 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.203+0000 7f83ba59c700 1 --2- 172.21.15.67:0/1632875103 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f83bc15ab10 0x7f83bc0c64e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:17.215 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.203+0000 7f83ba59c700 1 -- 172.21.15.67:0/1632875103 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f83bc15f1f0 msgr2=0x7f83bc0c6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:17.216 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.203+0000 7f83ba59c700 1 --2- 172.21.15.67:0/1632875103 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f83bc15f1f0 0x7f83bc0c6a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:17.216 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.203+0000 7f83ba59c700 1 -- 172.21.15.67:0/1632875103 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83ac009430 con 0x7f83bc0cfc60 2023-10-14T10:36:17.216 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.203+0000 7f83ba59c700 1 --2- 172.21.15.67:0/1632875103 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83bc0cfc60 0x7f83bc157d90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f83b000c480 tx=0x7f83b000c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:17.216 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.203+0000 7f83aaffd700 1 -- 172.21.15.67:0/1632875103 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b00052d0 con 0x7f83bc0cfc60 2023-10-14T10:36:17.216 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.203+0000 7f83aaffd700 1 -- 172.21.15.67:0/1632875103 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83b0005430 con 0x7f83bc0cfc60 2023-10-14T10:36:17.217 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.203+0000 7f83c0f89700 1 -- 172.21.15.67:0/1632875103 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83bc0c8bc0 con 0x7f83bc0cfc60 2023-10-14T10:36:17.217 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.203+0000 7f83aaffd700 1 -- 172.21.15.67:0/1632875103 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b00038c0 con 0x7f83bc0cfc60 2023-10-14T10:36:17.217 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.203+0000 7f83c0f89700 1 -- 172.21.15.67:0/1632875103 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83bc0c9180 con 0x7f83bc0cfc60 2023-10-14T10:36:17.217 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.204+0000 7f83c0f89700 1 -- 172.21.15.67:0/1632875103 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83bc0afc10 con 0x7f83bc0cfc60 2023-10-14T10:36:17.217 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.204+0000 7f83aaffd700 1 -- 172.21.15.67:0/1632875103 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f83b0003a20 con 0x7f83bc0cfc60 2023-10-14T10:36:17.218 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.205+0000 7f83aaffd700 1 --2- 172.21.15.67:0/1632875103 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f839c06af00 0x7f839c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:17.218 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.205+0000 7f83aaffd700 1 -- 172.21.15.67:0/1632875103 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f83b0088b10 con 0x7f83bc0cfc60 2023-10-14T10:36:17.218 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.205+0000 7f83b9d9b700 1 --2- 172.21.15.67:0/1632875103 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f839c06af00 0x7f839c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:17.218 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.205+0000 7f83b9d9b700 1 --2- 172.21.15.67:0/1632875103 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f839c06af00 0x7f839c06d3c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f83ac013040 tx=0x7f83ac008070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:17.219 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.209+0000 7f83aaffd700 1 -- 172.21.15.67:0/1632875103 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f83b0053b20 con 0x7f83bc0cfc60 2023-10-14T10:36:17.457 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.455+0000 7f83c0f89700 1 -- 172.21.15.67:0/1632875103 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f83bc15eca0 con 0x7f83bc0cfc60 2023-10-14T10:36:17.457 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.455+0000 7f83aaffd700 1 -- 172.21.15.67:0/1632875103 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4901 (secure 0 0 0) 0x7f83b0017090 con 0x7f83bc0cfc60 2023-10-14T10:36:17.458 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-14T10:36:17.458 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4217,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6836/838298018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":838298018},{"type":"v1","addr":"172.21.15.72:6836","nonce":838298018}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4255,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/1687192726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6833","nonce":1687192726},{"type":"v1","addr":"172.21.15.67:6835","nonce":1687192726}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6834/3936231765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":3936231765},{"type":"v1","addr":"172.21.15.67:6834","nonce":3936231765}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-14T10:36:03.247976+0000","modified":"2023-10-14T10:36:06.283381+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4267},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4267":{"gid":4267,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.72:6837/757500960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6835","nonce":757500960},{"type":"v1","addr":"172.21.15.72:6837","nonce":757500960}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-14T10:36:17.460 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.457+0000 7f83c0f89700 1 -- 172.21.15.67:0/1632875103 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f839c06af00 msgr2=0x7f839c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:17.460 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.457+0000 7f83c0f89700 1 --2- 172.21.15.67:0/1632875103 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f839c06af00 0x7f839c06d3c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f83ac013040 tx=0x7f83ac008070 comp rx=0 tx=0).stop 2023-10-14T10:36:17.460 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.457+0000 7f83c0f89700 1 -- 172.21.15.67:0/1632875103 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83bc0cfc60 msgr2=0x7f83bc157d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:17.461 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.457+0000 7f83c0f89700 1 --2- 172.21.15.67:0/1632875103 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83bc0cfc60 0x7f83bc157d90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f83b000c480 tx=0x7f83b000c840 comp rx=0 tx=0).stop 2023-10-14T10:36:17.461 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.458+0000 7f83c0f89700 1 -- 172.21.15.67:0/1632875103 shutdown_connections 2023-10-14T10:36:17.461 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.458+0000 7f83c0f89700 1 --2- 172.21.15.67:0/1632875103 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f839c06af00 0x7f839c06d3c0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:17.461 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.458+0000 7f83c0f89700 1 --2- 172.21.15.67:0/1632875103 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f83bc15f1f0 0x7f83bc0c6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:17.461 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.458+0000 7f83c0f89700 1 --2- 172.21.15.67:0/1632875103 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83bc0cfc60 0x7f83bc157d90 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:17.462 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.458+0000 7f83c0f89700 1 --2- 172.21.15.67:0/1632875103 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f83bc15ab10 0x7f83bc0c64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:17.462 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.458+0000 7f83c0f89700 1 -- 172.21.15.67:0/1632875103 >> 172.21.15.67:0/1632875103 conn(0x7f83bc06dae0 msgr2=0x7f83bc06b0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:17.462 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.458+0000 7f83c0f89700 1 -- 172.21.15.67:0/1632875103 shutdown_connections 2023-10-14T10:36:17.462 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.458+0000 7f83c0f89700 1 -- 172.21.15.67:0/1632875103 wait complete. 2023-10-14T10:36:17.463 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 8 2023-10-14T10:36:17.472 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-14T10:36:17.596 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.594+0000 7fb87dbd5700 1 Processor -- start 2023-10-14T10:36:17.596 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.594+0000 7fb87dbd5700 1 -- start start 2023-10-14T10:36:17.599 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.594+0000 7fb87dbd5700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fb87815bb50 0x7fb87806e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:17.599 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.594+0000 7fb87dbd5700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb87815bf30 con 0x7fb87806ed50 2023-10-14T10:36:17.599 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.594+0000 7fb87dbd5700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb87806d500 con 0x7fb8780ce280 2023-10-14T10:36:17.600 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.594+0000 7fb87dbd5700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb87806f1d0 con 0x7fb87815bb50 2023-10-14T10:36:17.600 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.595+0000 7fb876ffd700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fb87806ed50 0x7fb8780cdb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:52264/0 (socket says 172.21.15.67:52264) 2023-10-14T10:36:17.600 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.595+0000 7fb876ffd700 1 -- 172.21.15.67:0/3904480598 learned_addr learned my addr 172.21.15.67:0/3904480598 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:36:17.600 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.595+0000 7fb8777fe700 1 --2- 172.21.15.67:0/3904480598 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fb87815bb50 0x7fb87806e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:17.601 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.595+0000 7fb8767fc700 1 -- 172.21.15.67:0/3904480598 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4171487674 0 0) 0x7fb87815bf30 con 0x7fb87806ed50 2023-10-14T10:36:17.601 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.595+0000 7fb8767fc700 1 -- 172.21.15.67:0/3904480598 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb8600035b0 con 0x7fb87806ed50 2023-10-14T10:36:17.601 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.595+0000 7fb8767fc700 1 -- 172.21.15.67:0/3904480598 <== mon.1 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3083986530 0 0) 0x7fb87806d500 con 0x7fb8780ce280 2023-10-14T10:36:17.601 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.595+0000 7fb8767fc700 1 -- 172.21.15.67:0/3904480598 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb87815bf30 con 0x7fb8780ce280 2023-10-14T10:36:17.601 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.595+0000 7fb8777fe700 1 -- 172.21.15.67:0/3904480598 >> v1:172.21.15.72:6789/0 conn(0x7fb8780ce280 legacy=0x7fb87806a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:17.602 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.595+0000 7fb8777fe700 1 -- 172.21.15.67:0/3904480598 >> v1:172.21.15.67:6789/0 conn(0x7fb87806ed50 legacy=0x7fb8780cdb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:17.602 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.595+0000 7fb8777fe700 1 -- 172.21.15.67:0/3904480598 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb87806d7e0 con 0x7fb87815bb50 2023-10-14T10:36:17.602 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.595+0000 7fb8777fe700 1 --2- 172.21.15.67:0/3904480598 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fb87815bb50 0x7fb87806e810 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb868009870 tx=0x7fb868009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=936d683b19f8b4d2 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:17.602 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.595+0000 7fb8767fc700 1 -- 172.21.15.67:0/3904480598 <== mon.2 v2:172.21.15.72:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb868011830 con 0x7fb87815bb50 2023-10-14T10:36:17.602 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.595+0000 7fb8767fc700 1 -- 172.21.15.67:0/3904480598 <== mon.2 v2:172.21.15.72:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb868011990 con 0x7fb87815bb50 2023-10-14T10:36:17.603 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.595+0000 7fb8767fc700 1 -- 172.21.15.67:0/3904480598 <== mon.2 v2:172.21.15.72:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb868011d20 con 0x7fb87815bb50 2023-10-14T10:36:17.603 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.596+0000 7fb87dbd5700 1 -- 172.21.15.67:0/3904480598 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fb87815bb50 msgr2=0x7fb87806e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:17.603 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.596+0000 7fb87dbd5700 1 --2- 172.21.15.67:0/3904480598 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fb87815bb50 0x7fb87806e810 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb868009870 tx=0x7fb868009b80 comp rx=0 tx=0).stop 2023-10-14T10:36:17.604 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.596+0000 7fb87dbd5700 1 -- 172.21.15.67:0/3904480598 shutdown_connections 2023-10-14T10:36:17.604 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.596+0000 7fb87dbd5700 1 --2- 172.21.15.67:0/3904480598 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fb87815bb50 0x7fb87806e810 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:17.604 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.596+0000 7fb87dbd5700 1 -- 172.21.15.67:0/3904480598 >> 172.21.15.67:0/3904480598 conn(0x7fb8780cc6f0 msgr2=0x7fb8780ccb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:17.604 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.596+0000 7fb87dbd5700 1 -- 172.21.15.67:0/3904480598 shutdown_connections 2023-10-14T10:36:17.604 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.596+0000 7fb87dbd5700 1 -- 172.21.15.67:0/3904480598 wait complete. 2023-10-14T10:36:17.604 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.597+0000 7fb87dbd5700 1 Processor -- start 2023-10-14T10:36:17.607 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.597+0000 7fb87dbd5700 1 -- start start 2023-10-14T10:36:17.607 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.598+0000 7fb87dbd5700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb87806ed50 0x7fb8781f1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:17.608 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.598+0000 7fb87dbd5700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fb8780ce280 0x7fb8780aef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:17.608 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.598+0000 7fb87dbd5700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb87815bb50 0x7fb87815de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:17.608 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.598+0000 7fb87dbd5700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8780c7dc0 con 0x7fb87806ed50 2023-10-14T10:36:17.608 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.598+0000 7fb87dbd5700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8780c8a40 con 0x7fb87815bb50 2023-10-14T10:36:17.608 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.598+0000 7fb87dbd5700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8781f2230 con 0x7fb8780ce280 2023-10-14T10:36:17.609 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.598+0000 7fb8777fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb87806ed50 0x7fb8781f1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:17.609 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.598+0000 7fb8777fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb87806ed50 0x7fb8781f1cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44504/0 (socket says 172.21.15.67:44504) 2023-10-14T10:36:17.609 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.598+0000 7fb8777fe700 1 -- 172.21.15.67:0/4005990819 learned_addr learned my addr 172.21.15.67:0/4005990819 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:17.609 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.598+0000 7fb876ffd700 1 --2- 172.21.15.67:0/4005990819 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fb8780ce280 0x7fb8780aef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:17.609 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.598+0000 7fb877fff700 1 --2- 172.21.15.67:0/4005990819 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb87815bb50 0x7fb87815de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:17.610 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.599+0000 7fb8777fe700 1 -- 172.21.15.67:0/4005990819 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fb8780ce280 msgr2=0x7fb8780aef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:17.610 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.599+0000 7fb8777fe700 1 --2- 172.21.15.67:0/4005990819 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fb8780ce280 0x7fb8780aef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:17.610 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.599+0000 7fb8777fe700 1 -- 172.21.15.67:0/4005990819 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb87815bb50 msgr2=0x7fb87815de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:17.610 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.599+0000 7fb8777fe700 1 --2- 172.21.15.67:0/4005990819 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb87815bb50 0x7fb87815de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:17.610 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.599+0000 7fb8777fe700 1 -- 172.21.15.67:0/4005990819 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8680094a0 con 0x7fb87806ed50 2023-10-14T10:36:17.610 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.599+0000 7fb877fff700 1 --2- 172.21.15.67:0/4005990819 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb87815bb50 0x7fb87815de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-14T10:36:17.611 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.599+0000 7fb8777fe700 1 --2- 172.21.15.67:0/4005990819 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb87806ed50 0x7fb8781f1cf0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb868009bc0 tx=0x7fb868004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:17.611 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.599+0000 7fb85ffff700 1 -- 172.21.15.67:0/4005990819 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb86801a9f0 con 0x7fb87806ed50 2023-10-14T10:36:17.611 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.599+0000 7fb85ffff700 1 -- 172.21.15.67:0/4005990819 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb86801ab50 con 0x7fb87806ed50 2023-10-14T10:36:17.611 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.599+0000 7fb85ffff700 1 -- 172.21.15.67:0/4005990819 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8680243f0 con 0x7fb87806ed50 2023-10-14T10:36:17.611 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.599+0000 7fb87dbd5700 1 -- 172.21.15.67:0/4005990819 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8780c7240 con 0x7fb87806ed50 2023-10-14T10:36:17.612 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.599+0000 7fb87dbd5700 1 -- 172.21.15.67:0/4005990819 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8780c7800 con 0x7fb87806ed50 2023-10-14T10:36:17.612 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.600+0000 7fb87dbd5700 1 -- 172.21.15.67:0/4005990819 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb87806dd80 con 0x7fb87806ed50 2023-10-14T10:36:17.612 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.601+0000 7fb85ffff700 1 -- 172.21.15.67:0/4005990819 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fb86802d450 con 0x7fb87806ed50 2023-10-14T10:36:17.612 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.601+0000 7fb85ffff700 1 --2- 172.21.15.67:0/4005990819 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fb86406af00 0x7fb86406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:17.613 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.601+0000 7fb85ffff700 1 -- 172.21.15.67:0/4005990819 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb86808a8e0 con 0x7fb87806ed50 2023-10-14T10:36:17.613 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.601+0000 7fb876ffd700 1 --2- 172.21.15.67:0/4005990819 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fb86406af00 0x7fb86406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:17.613 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.602+0000 7fb876ffd700 1 --2- 172.21.15.67:0/4005990819 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fb86406af00 0x7fb86406d3c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb87815e960 tx=0x7fb86c007e00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:17.613 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.605+0000 7fb85ffff700 1 -- 172.21.15.67:0/4005990819 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb87806dd80 con 0x7fb87806ed50 2023-10-14T10:36:17.808 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.806+0000 7fb87dbd5700 1 -- 172.21.15.67:0/4005990819 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb87806dd80 con 0x7fb87806ed50 2023-10-14T10:36:17.809 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.807+0000 7fb85ffff700 1 -- 172.21.15.67:0/4005990819 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16457 (secure 0 0 0) 0x7fb87806dd80 con 0x7fb87806ed50 2023-10-14T10:36:17.809 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-14T10:36:17.810 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":22,"fsid":"37f9ab46-3082-4b71-8b24-08d39897b008","created":"2023-10-14T10:35:46.067554+0000","modified":"2023-10-14T10:36:04.837296+0000","last_up_change":"2023-10-14T10:35:56.668142+0000","last_in_change":"2023-10-14T10:35:48.551490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-14T10:35:57.809566+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-14T10:36:01.667146+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-14T10:36:02.219700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2b59a07e-11ef-440c-b766-9f64ac885838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6801","nonce":1287255499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6803","nonce":1287255499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6807","nonce":1287255499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6805","nonce":1287255499}]},"public_addr":"172.21.15.67:6801/1287255499","cluster_addr":"172.21.15.67:6803/1287255499","heartbeat_back_addr":"172.21.15.67:6807/1287255499","heartbeat_front_addr":"172.21.15.67:6805/1287255499","state":["exists","up"]},{"osd":1,"uuid":"6f813bd6-fa0b-487d-a204-431eb1276a60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6818","nonce":2435538787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6822","nonce":2435538787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6830","nonce":2435538787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6826","nonce":2435538787}]},"public_addr":"172.21.15.67:6818/2435538787","cluster_addr":"172.21.15.67:6822/2435538787","heartbeat_back_addr":"172.21.15.67:6830/2435538787","heartbeat_front_addr":"172.21.15.67:6826/2435538787","state":["exists","up"]},{"osd":2,"uuid":"6af42c6a-1b59-4669-9c04-b6876788446d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6809","nonce":2827491791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6811","nonce":2827491791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6815","nonce":2827491791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6813","nonce":2827491791}]},"public_addr":"172.21.15.67:6809/2827491791","cluster_addr":"172.21.15.67:6811/2827491791","heartbeat_back_addr":"172.21.15.67:6815/2827491791","heartbeat_front_addr":"172.21.15.67:6813/2827491791","state":["exists","up"]},{"osd":3,"uuid":"1d889d76-a646-499d-bfff-ed5bae5434ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6819","nonce":1848001176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6821","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6823","nonce":1848001176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6831","nonce":1848001176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6825","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6827","nonce":1848001176}]},"public_addr":"172.21.15.67:6819/1848001176","cluster_addr":"172.21.15.67:6823/1848001176","heartbeat_back_addr":"172.21.15.67:6831/1848001176","heartbeat_front_addr":"172.21.15.67:6827/1848001176","state":["exists","up"]},{"osd":4,"uuid":"c3271dff-b437-4fdd-b13b-6e49a35473af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6804","nonce":1891230785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6808","nonce":1891230785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6815","nonce":1891230785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6811","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6813","nonce":1891230785}]},"public_addr":"172.21.15.72:6804/1891230785","cluster_addr":"172.21.15.72:6808/1891230785","heartbeat_back_addr":"172.21.15.72:6815/1891230785","heartbeat_front_addr":"172.21.15.72:6813/1891230785","state":["exists","up"]},{"osd":5,"uuid":"0337bb18-4b68-4eff-b528-a667275e6926","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6801","nonce":569686991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6803","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6805","nonce":569686991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6812","nonce":569686991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6807","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6809","nonce":569686991}]},"public_addr":"172.21.15.72:6801/569686991","cluster_addr":"172.21.15.72:6805/569686991","heartbeat_back_addr":"172.21.15.72:6812/569686991","heartbeat_front_addr":"172.21.15.72:6809/569686991","state":["exists","up"]},{"osd":6,"uuid":"8e92d9cf-0c8c-46eb-89a7-db2023ef1509","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6817","nonce":3767685934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6819","nonce":3767685934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6823","nonce":3767685934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6821","nonce":3767685934}]},"public_addr":"172.21.15.72:6817/3767685934","cluster_addr":"172.21.15.72:6819/3767685934","heartbeat_back_addr":"172.21.15.72:6823/3767685934","heartbeat_front_addr":"172.21.15.72:6821/3767685934","state":["exists","up"]},{"osd":7,"uuid":"3fd6aa7d-50a9-45c9-9561-5902a044a119","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6825","nonce":3567465926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6827","nonce":3567465926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6831","nonce":3567465926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6829","nonce":3567465926}]},"public_addr":"172.21.15.72:6825/3567465926","cluster_addr":"172.21.15.72:6827/3567465926","heartbeat_back_addr":"172.21.15.72:6831/3567465926","heartbeat_front_addr":"172.21.15.72:6829/3567465926","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.511466+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.795107+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.858674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.728895+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.794842+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.750064+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.732122+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.765185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-14T10:36:17.812 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.809+0000 7fb87dbd5700 1 -- 172.21.15.67:0/4005990819 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fb86406af00 msgr2=0x7fb86406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:17.812 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.809+0000 7fb87dbd5700 1 --2- 172.21.15.67:0/4005990819 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fb86406af00 0x7fb86406d3c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb87815e960 tx=0x7fb86c007e00 comp rx=0 tx=0).stop 2023-10-14T10:36:17.812 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.809+0000 7fb87dbd5700 1 -- 172.21.15.67:0/4005990819 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb87806ed50 msgr2=0x7fb8781f1cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:17.812 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.809+0000 7fb87dbd5700 1 --2- 172.21.15.67:0/4005990819 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb87806ed50 0x7fb8781f1cf0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb868009bc0 tx=0x7fb868004750 comp rx=0 tx=0).stop 2023-10-14T10:36:17.813 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.809+0000 7fb87dbd5700 1 -- 172.21.15.67:0/4005990819 shutdown_connections 2023-10-14T10:36:17.813 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.809+0000 7fb87dbd5700 1 --2- 172.21.15.67:0/4005990819 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fb86406af00 0x7fb86406d3c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:17.813 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.809+0000 7fb87dbd5700 1 --2- 172.21.15.67:0/4005990819 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb87815bb50 0x7fb87815de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:17.813 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.809+0000 7fb87dbd5700 1 --2- 172.21.15.67:0/4005990819 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb87806ed50 0x7fb8781f1cf0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:17.813 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.809+0000 7fb87dbd5700 1 --2- 172.21.15.67:0/4005990819 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fb8780ce280 0x7fb8780aef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:17.814 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.809+0000 7fb87dbd5700 1 -- 172.21.15.67:0/4005990819 >> 172.21.15.67:0/4005990819 conn(0x7fb8780cc6f0 msgr2=0x7fb8780b1020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:17.814 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.810+0000 7fb87dbd5700 1 -- 172.21.15.67:0/4005990819 shutdown_connections 2023-10-14T10:36:17.814 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.810+0000 7fb87dbd5700 1 -- 172.21.15.67:0/4005990819 wait complete. 2023-10-14T10:36:17.824 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-14T10:36:17.950 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.948+0000 7efc15ca1700 1 Processor -- start 2023-10-14T10:36:17.951 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.948+0000 7efc15ca1700 1 -- start start 2023-10-14T10:36:17.953 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.949+0000 7efc15ca1700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7efc1015f130 0x7efc100cede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:17.953 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.949+0000 7efc15ca1700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7efc100b28b0 0x7efc1006a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:17.953 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.949+0000 7efc15ca1700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc100b3530 con 0x7efc1006ad80 2023-10-14T10:36:17.954 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.949+0000 7efc15ca1700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc100cde00 con 0x7efc1015f130 2023-10-14T10:36:17.954 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.949+0000 7efc15ca1700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc1006b1d0 con 0x7efc100b28b0 2023-10-14T10:36:17.954 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.949+0000 7efc0f7fe700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7efc1015f130 0x7efc100cede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:17.954 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.949+0000 7efc0f7fe700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7efc1015f130 0x7efc100cede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44510/0 (socket says 172.21.15.67:44510) 2023-10-14T10:36:17.954 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.949+0000 7efc0f7fe700 1 -- 172.21.15.67:0/3353146144 learned_addr learned my addr 172.21.15.67:0/3353146144 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:17.955 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.949+0000 7efc0effd700 1 --2- 172.21.15.67:0/3353146144 >> v2:172.21.15.72:3300/0 conn(0x7efc100b28b0 0x7efc1006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:17.955 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.950+0000 7efc0f7fe700 1 -- 172.21.15.67:0/3353146144 >> v2:172.21.15.72:3300/0 conn(0x7efc100b28b0 msgr2=0x7efc1006a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:17.955 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.950+0000 7efc0f7fe700 1 --2- 172.21.15.67:0/3353146144 >> v2:172.21.15.72:3300/0 conn(0x7efc100b28b0 0x7efc1006a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:17.955 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.950+0000 7efc0f7fe700 1 -- 172.21.15.67:0/3353146144 >> v1:172.21.15.72:6789/0 conn(0x7efc1006ad80 legacy=0x7efc10071af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:17.955 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.950+0000 7efc0f7fe700 1 -- 172.21.15.67:0/3353146144 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc100cd180 con 0x7efc1015f130 2023-10-14T10:36:17.956 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.950+0000 7efc0f7fe700 1 --2- 172.21.15.67:0/3353146144 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc1015f130 0x7efc100cede0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7efc100c3f30 tx=0x7efc00009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=569aff52d976a66d server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:17.956 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.950+0000 7efc0e7fc700 1 -- 172.21.15.67:0/3353146144 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc0000b870 con 0x7efc1015f130 2023-10-14T10:36:17.956 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.950+0000 7efc0e7fc700 1 -- 172.21.15.67:0/3353146144 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc0000b9d0 con 0x7efc1015f130 2023-10-14T10:36:17.956 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.950+0000 7efc0e7fc700 1 -- 172.21.15.67:0/3353146144 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc0000bd60 con 0x7efc1015f130 2023-10-14T10:36:17.956 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.950+0000 7efc15ca1700 1 -- 172.21.15.67:0/3353146144 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc1015f130 msgr2=0x7efc100cede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:17.957 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.950+0000 7efc15ca1700 1 --2- 172.21.15.67:0/3353146144 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc1015f130 0x7efc100cede0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7efc100c3f30 tx=0x7efc00009bd0 comp rx=0 tx=0).stop 2023-10-14T10:36:17.957 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.950+0000 7efc15ca1700 1 -- 172.21.15.67:0/3353146144 shutdown_connections 2023-10-14T10:36:17.957 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.950+0000 7efc15ca1700 1 --2- 172.21.15.67:0/3353146144 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc1015f130 0x7efc100cede0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:17.957 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.950+0000 7efc15ca1700 1 --2- 172.21.15.67:0/3353146144 >> v2:172.21.15.72:3300/0 conn(0x7efc100b28b0 0x7efc1006a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:17.957 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.950+0000 7efc15ca1700 1 -- 172.21.15.67:0/3353146144 >> 172.21.15.67:0/3353146144 conn(0x7efc1006dae0 msgr2=0x7efc1006def0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:17.957 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.951+0000 7efc15ca1700 1 -- 172.21.15.67:0/3353146144 shutdown_connections 2023-10-14T10:36:17.958 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.951+0000 7efc15ca1700 1 -- 172.21.15.67:0/3353146144 wait complete. 2023-10-14T10:36:17.960 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.952+0000 7efc15ca1700 1 Processor -- start 2023-10-14T10:36:17.960 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.952+0000 7efc15ca1700 1 -- start start 2023-10-14T10:36:17.961 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.953+0000 7efc15ca1700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7efc1006ad80 0x7efc101498b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:17.961 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.953+0000 7efc15ca1700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efc100b28b0 0x7efc10153fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:17.961 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.953+0000 7efc15ca1700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc1015f130 0x7efc101f49c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:17.961 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.953+0000 7efc15ca1700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc100c4000 con 0x7efc1015f130 2023-10-14T10:36:17.961 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.953+0000 7efc15ca1700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc100c4580 con 0x7efc100b28b0 2023-10-14T10:36:17.962 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.953+0000 7efc15ca1700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc10149df0 con 0x7efc1006ad80 2023-10-14T10:36:17.962 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.953+0000 7efc0ffff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc1015f130 0x7efc101f49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:17.962 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.953+0000 7efc0ffff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc1015f130 0x7efc101f49c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44520/0 (socket says 172.21.15.67:44520) 2023-10-14T10:36:17.962 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.953+0000 7efc0ffff700 1 -- 172.21.15.67:0/2609012309 learned_addr learned my addr 172.21.15.67:0/2609012309 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:17.963 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.954+0000 7efc0effd700 1 --2- 172.21.15.67:0/2609012309 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efc100b28b0 0x7efc10153fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:17.963 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.954+0000 7efc0f7fe700 1 --2- 172.21.15.67:0/2609012309 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7efc1006ad80 0x7efc101498b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:17.963 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.954+0000 7efc0ffff700 1 -- 172.21.15.67:0/2609012309 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7efc1006ad80 msgr2=0x7efc101498b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:17.963 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.954+0000 7efc0ffff700 1 --2- 172.21.15.67:0/2609012309 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7efc1006ad80 0x7efc101498b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:17.963 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.954+0000 7efc0ffff700 1 -- 172.21.15.67:0/2609012309 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efc100b28b0 msgr2=0x7efc10153fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:17.964 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.954+0000 7efc0ffff700 1 --2- 172.21.15.67:0/2609012309 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efc100b28b0 0x7efc10153fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:17.964 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.954+0000 7efc0ffff700 1 -- 172.21.15.67:0/2609012309 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc00009550 con 0x7efc1015f130 2023-10-14T10:36:17.964 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.954+0000 7efc0effd700 1 --2- 172.21.15.67:0/2609012309 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efc100b28b0 0x7efc10153fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-14T10:36:17.964 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.954+0000 7efc0ffff700 1 --2- 172.21.15.67:0/2609012309 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc1015f130 0x7efc101f49c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7efc0800cb50 tx=0x7efc0800cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:17.964 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.954+0000 7efc07fff700 1 -- 172.21.15.67:0/2609012309 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc08003c00 con 0x7efc1015f130 2023-10-14T10:36:17.965 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.954+0000 7efc07fff700 1 -- 172.21.15.67:0/2609012309 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc08003d60 con 0x7efc1015f130 2023-10-14T10:36:17.965 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.954+0000 7efc07fff700 1 -- 172.21.15.67:0/2609012309 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc0800ab70 con 0x7efc1015f130 2023-10-14T10:36:17.965 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.954+0000 7efc15ca1700 1 -- 172.21.15.67:0/2609012309 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc101f5460 con 0x7efc1015f130 2023-10-14T10:36:17.965 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.954+0000 7efc15ca1700 1 -- 172.21.15.67:0/2609012309 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc100c4210 con 0x7efc1015f130 2023-10-14T10:36:17.965 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.955+0000 7efc15ca1700 1 -- 172.21.15.67:0/2609012309 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc100c46f0 con 0x7efc1015f130 2023-10-14T10:36:17.966 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.955+0000 7efc07fff700 1 -- 172.21.15.67:0/2609012309 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7efc0800acd0 con 0x7efc1015f130 2023-10-14T10:36:17.966 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.956+0000 7efc07fff700 1 --2- 172.21.15.67:0/2609012309 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7efbf006af00 0x7efbf006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:17.966 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.956+0000 7efc07fff700 1 -- 172.21.15.67:0/2609012309 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efc08089630 con 0x7efc1015f130 2023-10-14T10:36:17.966 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.956+0000 7efc0f7fe700 1 --2- 172.21.15.67:0/2609012309 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7efbf006af00 0x7efbf006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:17.966 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.956+0000 7efc0f7fe700 1 --2- 172.21.15.67:0/2609012309 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7efbf006af00 0x7efbf006d3c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7efc00009130 tx=0x7efc00008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:17.967 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:17.960+0000 7efc07fff700 1 -- 172.21.15.67:0/2609012309 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7efc080546e0 con 0x7efc1015f130 2023-10-14T10:36:18.206 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.204+0000 7efc15ca1700 1 -- 172.21.15.67:0/2609012309 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efc10067d20 con 0x7efc1015f130 2023-10-14T10:36:18.207 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.204+0000 7efc07fff700 1 -- 172.21.15.67:0/2609012309 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4901 (secure 0 0 0) 0x7efc08057d00 con 0x7efc1015f130 2023-10-14T10:36:18.207 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-14T10:36:18.207 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4217,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6836/838298018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":838298018},{"type":"v1","addr":"172.21.15.72:6836","nonce":838298018}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4255,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/1687192726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6833","nonce":1687192726},{"type":"v1","addr":"172.21.15.67:6835","nonce":1687192726}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6834/3936231765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":3936231765},{"type":"v1","addr":"172.21.15.67:6834","nonce":3936231765}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-14T10:36:03.247976+0000","modified":"2023-10-14T10:36:06.283381+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4267},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4267":{"gid":4267,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.72:6837/757500960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6835","nonce":757500960},{"type":"v1","addr":"172.21.15.72:6837","nonce":757500960}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-14T10:36:18.209 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.207+0000 7efc15ca1700 1 -- 172.21.15.67:0/2609012309 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7efbf006af00 msgr2=0x7efbf006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:18.209 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.207+0000 7efc15ca1700 1 --2- 172.21.15.67:0/2609012309 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7efbf006af00 0x7efbf006d3c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7efc00009130 tx=0x7efc00008030 comp rx=0 tx=0).stop 2023-10-14T10:36:18.209 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.207+0000 7efc15ca1700 1 -- 172.21.15.67:0/2609012309 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc1015f130 msgr2=0x7efc101f49c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:18.209 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.207+0000 7efc15ca1700 1 --2- 172.21.15.67:0/2609012309 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc1015f130 0x7efc101f49c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7efc0800cb50 tx=0x7efc0800cf10 comp rx=0 tx=0).stop 2023-10-14T10:36:18.210 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.207+0000 7efc15ca1700 1 -- 172.21.15.67:0/2609012309 shutdown_connections 2023-10-14T10:36:18.210 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.207+0000 7efc15ca1700 1 --2- 172.21.15.67:0/2609012309 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7efbf006af00 0x7efbf006d3c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:18.210 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.207+0000 7efc15ca1700 1 --2- 172.21.15.67:0/2609012309 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc1015f130 0x7efc101f49c0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:18.210 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.207+0000 7efc15ca1700 1 --2- 172.21.15.67:0/2609012309 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7efc1006ad80 0x7efc101498b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:18.210 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.207+0000 7efc15ca1700 1 --2- 172.21.15.67:0/2609012309 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efc100b28b0 0x7efc10153fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:18.211 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.207+0000 7efc15ca1700 1 -- 172.21.15.67:0/2609012309 >> 172.21.15.67:0/2609012309 conn(0x7efc1006dae0 msgr2=0x7efc1015b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:18.211 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.207+0000 7efc15ca1700 1 -- 172.21.15.67:0/2609012309 shutdown_connections 2023-10-14T10:36:18.211 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.207+0000 7efc15ca1700 1 -- 172.21.15.67:0/2609012309 wait complete. 2023-10-14T10:36:18.212 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 8 2023-10-14T10:36:18.623 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-14T10:36:03.247976+0000', 'modified': '2023-10-14T10:36:06.283381+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4267}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4267': {'gid': 4267, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.72:6837/757500960', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.72:6835', 'nonce': 757500960}, {'type': 'v1', 'addr': '172.21.15.72:6837', 'nonce': 757500960}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-14T10:36:18.624 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-14T10:36:18.624 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-14T10:36:18.754 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.752+0000 7f14b7a09700 1 Processor -- start 2023-10-14T10:36:18.755 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.752+0000 7f14b7a09700 1 -- start start 2023-10-14T10:36:18.757 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.753+0000 7f14b7a09700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f14b015dce0 0x7f14b00b03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:18.757 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.753+0000 7f14b7a09700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f14b015e0c0 0x7f14b00cf4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:18.757 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.753+0000 7f14b7a09700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f14b00cfa30 0x7f14b00cfe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:18.758 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.753+0000 7f14b7a09700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14b00d03c0 con 0x7f14b015e0c0 2023-10-14T10:36:18.758 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.753+0000 7f14b7a09700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14b00cbc40 con 0x7f14b00cfa30 2023-10-14T10:36:18.758 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.753+0000 7f14b7a09700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14b006cb70 con 0x7f14b015dce0 2023-10-14T10:36:18.758 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.753+0000 7f14b4fa4700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f14b015e0c0 0x7f14b00cf4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:18.758 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.753+0000 7f14b4fa4700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f14b015e0c0 0x7f14b00cf4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44526/0 (socket says 172.21.15.67:44526) 2023-10-14T10:36:18.759 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.753+0000 7f14b4fa4700 1 -- 172.21.15.67:0/2018150616 learned_addr learned my addr 172.21.15.67:0/2018150616 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:18.759 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.753+0000 7f14b5fa6700 1 --2- 172.21.15.67:0/2018150616 >> v2:172.21.15.72:3300/0 conn(0x7f14b00cfa30 0x7f14b00cfe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:18.759 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.753+0000 7f14b57a5700 1 --2- 172.21.15.67:0/2018150616 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f14b015dce0 0x7f14b00b03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:18.759 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.754+0000 7f14b4fa4700 1 -- 172.21.15.67:0/2018150616 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f14b015dce0 msgr2=0x7f14b00b03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:18.759 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.754+0000 7f14b4fa4700 1 --2- 172.21.15.67:0/2018150616 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f14b015dce0 0x7f14b00b03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:18.760 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.754+0000 7f14b4fa4700 1 -- 172.21.15.67:0/2018150616 >> v2:172.21.15.72:3300/0 conn(0x7f14b00cfa30 msgr2=0x7f14b00cfe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:18.760 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.754+0000 7f14b4fa4700 1 --2- 172.21.15.67:0/2018150616 >> v2:172.21.15.72:3300/0 conn(0x7f14b00cfa30 0x7f14b00cfe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:18.760 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.754+0000 7f14b4fa4700 1 -- 172.21.15.67:0/2018150616 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14b006ce50 con 0x7f14b015e0c0 2023-10-14T10:36:18.760 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.754+0000 7f14b4fa4700 1 --2- 172.21.15.67:0/2018150616 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14b015e0c0 0x7f14b00cf4f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f14b0067c30 tx=0x7f14ac009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b57981f222a12d4a server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:18.760 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.754+0000 7f14a7fff700 1 -- 172.21.15.67:0/2018150616 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14ac011830 con 0x7f14b015e0c0 2023-10-14T10:36:18.760 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.754+0000 7f14a7fff700 1 -- 172.21.15.67:0/2018150616 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14ac011990 con 0x7f14b015e0c0 2023-10-14T10:36:18.761 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.754+0000 7f14a7fff700 1 -- 172.21.15.67:0/2018150616 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14ac011d20 con 0x7f14b015e0c0 2023-10-14T10:36:18.761 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.755+0000 7f14b7a09700 1 -- 172.21.15.67:0/2018150616 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14b015e0c0 msgr2=0x7f14b00cf4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:18.761 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.755+0000 7f14b7a09700 1 --2- 172.21.15.67:0/2018150616 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14b015e0c0 0x7f14b00cf4f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f14b0067c30 tx=0x7f14ac009bd0 comp rx=0 tx=0).stop 2023-10-14T10:36:18.761 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.755+0000 7f14b7a09700 1 -- 172.21.15.67:0/2018150616 shutdown_connections 2023-10-14T10:36:18.761 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.755+0000 7f14b7a09700 1 --2- 172.21.15.67:0/2018150616 >> v2:172.21.15.72:3300/0 conn(0x7f14b00cfa30 0x7f14b00cfe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:18.762 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.755+0000 7f14b7a09700 1 --2- 172.21.15.67:0/2018150616 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f14b015dce0 0x7f14b00b03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:18.762 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.755+0000 7f14b7a09700 1 --2- 172.21.15.67:0/2018150616 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14b015e0c0 0x7f14b00cf4f0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:18.762 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.755+0000 7f14b7a09700 1 -- 172.21.15.67:0/2018150616 >> 172.21.15.67:0/2018150616 conn(0x7f14b00d15f0 msgr2=0x7f14b00d19d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:18.762 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.755+0000 7f14b7a09700 1 -- 172.21.15.67:0/2018150616 shutdown_connections 2023-10-14T10:36:18.762 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.755+0000 7f14b7a09700 1 -- 172.21.15.67:0/2018150616 wait complete. 2023-10-14T10:36:18.767 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.756+0000 7f14b7a09700 1 Processor -- start 2023-10-14T10:36:18.767 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.756+0000 7f14b7a09700 1 -- start start 2023-10-14T10:36:18.768 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.757+0000 7f14b7a09700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f14b00cfa30 0x7f14b00c7d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:18.768 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.757+0000 7f14b7a09700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14b015dce0 0x7f14b00cb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:18.768 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.757+0000 7f14b7a09700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f14b015e0c0 0x7f14b01f6db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:18.768 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.757+0000 7f14b7a09700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14b00c8f40 con 0x7f14b015dce0 2023-10-14T10:36:18.768 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.757+0000 7f14b7a09700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14b01f6920 con 0x7f14b015e0c0 2023-10-14T10:36:18.769 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.757+0000 7f14b7a09700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14b00c9750 con 0x7f14b00cfa30 2023-10-14T10:36:18.769 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.757+0000 7f14b4fa4700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14b015dce0 0x7f14b00cb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:18.769 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.757+0000 7f14b4fa4700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14b015dce0 0x7f14b00cb2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44532/0 (socket says 172.21.15.67:44532) 2023-10-14T10:36:18.769 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.757+0000 7f14b4fa4700 1 -- 172.21.15.67:0/3518902999 learned_addr learned my addr 172.21.15.67:0/3518902999 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:18.769 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.757+0000 7f14b57a5700 1 --2- 172.21.15.67:0/3518902999 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f14b00cfa30 0x7f14b00c7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:18.770 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.757+0000 7f14b5fa6700 1 --2- 172.21.15.67:0/3518902999 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f14b015e0c0 0x7f14b01f6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:18.770 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.757+0000 7f14b4fa4700 1 -- 172.21.15.67:0/3518902999 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f14b00cfa30 msgr2=0x7f14b00c7d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:18.770 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.757+0000 7f14b4fa4700 1 --2- 172.21.15.67:0/3518902999 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f14b00cfa30 0x7f14b00c7d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:18.770 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.757+0000 7f14b4fa4700 1 -- 172.21.15.67:0/3518902999 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f14b015e0c0 msgr2=0x7f14b01f6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:18.770 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.757+0000 7f14b4fa4700 1 --2- 172.21.15.67:0/3518902999 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f14b015e0c0 0x7f14b01f6db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:18.771 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.757+0000 7f14b4fa4700 1 -- 172.21.15.67:0/3518902999 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14ac009550 con 0x7f14b015dce0 2023-10-14T10:36:18.771 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.757+0000 7f14b4fa4700 1 --2- 172.21.15.67:0/3518902999 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14b015dce0 0x7f14b00cb2f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f14ac014040 tx=0x7f14ac004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:18.771 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.757+0000 7f14a5ffb700 1 -- 172.21.15.67:0/3518902999 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14ac004240 con 0x7f14b015dce0 2023-10-14T10:36:18.771 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.757+0000 7f14a5ffb700 1 -- 172.21.15.67:0/3518902999 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14ac01f070 con 0x7f14b015dce0 2023-10-14T10:36:18.771 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.757+0000 7f14b7a09700 1 -- 172.21.15.67:0/3518902999 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14b00c9190 con 0x7f14b015dce0 2023-10-14T10:36:18.772 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.757+0000 7f14a5ffb700 1 -- 172.21.15.67:0/3518902999 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14ac0045e0 con 0x7f14b015dce0 2023-10-14T10:36:18.772 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.758+0000 7f14b7a09700 1 -- 172.21.15.67:0/3518902999 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14b00c8290 con 0x7f14b015dce0 2023-10-14T10:36:18.772 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.759+0000 7f14a5ffb700 1 -- 172.21.15.67:0/3518902999 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f14ac021030 con 0x7f14b015dce0 2023-10-14T10:36:18.772 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.760+0000 7f14a5ffb700 1 --2- 172.21.15.67:0/3518902999 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f149406af00 0x7f149406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:18.772 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.760+0000 7f14a5ffb700 1 -- 172.21.15.67:0/3518902999 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f14ac08a6d0 con 0x7f14b015dce0 2023-10-14T10:36:18.772 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.760+0000 7f14b57a5700 1 --2- 172.21.15.67:0/3518902999 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f149406af00 0x7f149406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:18.773 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.760+0000 7f14b57a5700 1 --2- 172.21.15.67:0/3518902999 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f149406af00 0x7f149406d3c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f14a0009be0 tx=0x7f14a0009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:18.773 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.762+0000 7f14c3bf3b80 1 Processor -- start 2023-10-14T10:36:18.773 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.762+0000 7f14c3bf3b80 1 -- start start 2023-10-14T10:36:18.773 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.763+0000 7f14c3bf3b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f7494c5220 0x55f7494cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:18.774 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.763+0000 7f14c3bf3b80 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x55f7494ca630 0x55f7494ccfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:18.774 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.763+0000 7f14c3bf3b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f7494a0010 0x55f7494a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:18.774 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.763+0000 7f14c3bf3b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f749416c20 con 0x55f7494a0010 2023-10-14T10:36:18.774 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.763+0000 7f14c3bf3b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7493fbc10 con 0x55f7494c5220 2023-10-14T10:36:18.774 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.763+0000 7f14c3bf3b80 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f7493f9530 con 0x55f7494ca630 2023-10-14T10:36:18.775 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.763+0000 7f14b5fa6700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f7494a0010 0x55f7494a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:18.775 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.763+0000 7f14b5fa6700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f7494a0010 0x55f7494a06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44542/0 (socket says 172.21.15.67:44542) 2023-10-14T10:36:18.775 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.763+0000 7f14b5fa6700 1 -- 172.21.15.67:0/1488896499 learned_addr learned my addr 172.21.15.67:0/1488896499 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:18.775 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.763+0000 7f14b4fa4700 1 --2- 172.21.15.67:0/1488896499 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x55f7494ca630 0x55f7494ccfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:18.775 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.763+0000 7f14b57a5700 1 --2- 172.21.15.67:0/1488896499 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f7494c5220 0x55f7494cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:18.776 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.763+0000 7f14b5fa6700 1 -- 172.21.15.67:0/1488896499 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x55f7494ca630 msgr2=0x55f7494ccfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:18.776 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.763+0000 7f14b5fa6700 1 --2- 172.21.15.67:0/1488896499 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x55f7494ca630 0x55f7494ccfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:18.776 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.763+0000 7f14b5fa6700 1 -- 172.21.15.67:0/1488896499 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f7494c5220 msgr2=0x55f7494cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:18.776 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.763+0000 7f14b5fa6700 1 --2- 172.21.15.67:0/1488896499 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f7494c5220 0x55f7494cca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:18.776 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.763+0000 7f14b5fa6700 1 -- 172.21.15.67:0/1488896499 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f7493b23a0 con 0x55f7494a0010 2023-10-14T10:36:18.777 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.763+0000 7f14b5fa6700 1 --2- 172.21.15.67:0/1488896499 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f7494a0010 0x55f7494a06c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f14a8011500 tx=0x7f14a800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f38deac6062cfb32 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:18.777 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.764+0000 7f149affd700 1 -- 172.21.15.67:0/1488896499 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a8010040 con 0x55f7494a0010 2023-10-14T10:36:18.777 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.764+0000 7f149affd700 1 -- 172.21.15.67:0/1488896499 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14a8016650 con 0x55f7494a0010 2023-10-14T10:36:18.777 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.764+0000 7f149affd700 1 -- 172.21.15.67:0/1488896499 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a80169c0 con 0x55f7494a0010 2023-10-14T10:36:18.777 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.764+0000 7f14c3bf3b80 1 -- 172.21.15.67:0/1488896499 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f7494a0010 msgr2=0x55f7494a06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:18.777 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.764+0000 7f14c3bf3b80 1 --2- 172.21.15.67:0/1488896499 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f7494a0010 0x55f7494a06c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f14a8011500 tx=0x7f14a800b040 comp rx=0 tx=0).stop 2023-10-14T10:36:18.778 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.764+0000 7f14c3bf3b80 1 -- 172.21.15.67:0/1488896499 shutdown_connections 2023-10-14T10:36:18.778 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.764+0000 7f14c3bf3b80 1 --2- 172.21.15.67:0/1488896499 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f7494a0010 0x55f7494a06c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:18.778 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.764+0000 7f14c3bf3b80 1 --2- 172.21.15.67:0/1488896499 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f7494c5220 0x55f7494cca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:18.778 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.764+0000 7f14c3bf3b80 1 --2- 172.21.15.67:0/1488896499 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x55f7494ca630 0x55f7494ccfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:18.778 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.764+0000 7f14c3bf3b80 1 -- 172.21.15.67:0/1488896499 >> 172.21.15.67:0/1488896499 conn(0x55f7493cfd80 msgr2=0x55f7494eaff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:18.779 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.764+0000 7f14c3bf3b80 1 -- 172.21.15.67:0/1488896499 shutdown_connections 2023-10-14T10:36:18.779 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.764+0000 7f14c3bf3b80 1 -- 172.21.15.67:0/1488896499 wait complete. 2023-10-14T10:36:18.779 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.765+0000 7f14c3bf3b80 1 -- start start 2023-10-14T10:36:18.779 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.765+0000 7f14c3bf3b80 1 Processor -- start 2023-10-14T10:36:18.783 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.766+0000 7f14c3bf3b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f7494a0010 0x55f7494cc400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:18.783 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.766+0000 7f14c3bf3b80 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x55f7494c5220 0x55f7494c84f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:18.784 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.766+0000 7f14c3bf3b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f7494ca630 0x55f7494c6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:18.784 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.766+0000 7f14c3bf3b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7493f4e20 con 0x55f7494a0010 2023-10-14T10:36:18.784 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.766+0000 7f14c3bf3b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7493916c0 con 0x55f7494ca630 2023-10-14T10:36:18.784 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.766+0000 7f14c3bf3b80 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f749391360 con 0x55f7494c5220 2023-10-14T10:36:18.784 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.767+0000 7f14c3bf3b80 10 client.0 _ll_register_callbacks cb 0x55f7491ee7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-14T10:36:18.785 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.767+0000 7f14b57a5700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f7494a0010 0x55f7494cc400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:18.785 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.767+0000 7f14b57a5700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f7494a0010 0x55f7494cc400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44544/0 (socket says 172.21.15.67:44544) 2023-10-14T10:36:18.785 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.767+0000 7f14b57a5700 1 -- 172.21.15.67:0/662112233 learned_addr learned my addr 172.21.15.67:0/662112233 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:18.785 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.767+0000 7f14b4fa4700 1 --2- 172.21.15.67:0/662112233 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x55f7494c5220 0x55f7494c84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:18.785 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.767+0000 7f14b5fa6700 1 --2- 172.21.15.67:0/662112233 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f7494ca630 0x55f7494c6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:18.786 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.767+0000 7f14b57a5700 1 -- 172.21.15.67:0/662112233 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x55f7494c5220 msgr2=0x55f7494c84f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:18.786 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.767+0000 7f14b57a5700 1 --2- 172.21.15.67:0/662112233 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x55f7494c5220 0x55f7494c84f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:18.786 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.767+0000 7f14b57a5700 1 -- 172.21.15.67:0/662112233 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f7494ca630 msgr2=0x55f7494c6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:18.786 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.767+0000 7f14b57a5700 1 --2- 172.21.15.67:0/662112233 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f7494ca630 0x55f7494c6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:18.786 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.767+0000 7f14b57a5700 1 -- 172.21.15.67:0/662112233 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14a80111b0 con 0x55f7494a0010 2023-10-14T10:36:18.787 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.767+0000 7f14b57a5700 1 --2- 172.21.15.67:0/662112233 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f7494a0010 0x55f7494cc400 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f14a0011ec0 tx=0x7f14a0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5046c2ed6a8cad server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:18.787 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.767+0000 7f148ffff700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2023-10-14T10:36:18.787 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.767+0000 7f148ffff700 1 -- 172.21.15.67:0/662112233 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a001c040 con 0x55f7494a0010 2023-10-14T10:36:18.787 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.767+0000 7f148ffff700 1 -- 172.21.15.67:0/662112233 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f14a0014650 con 0x55f7494a0010 2023-10-14T10:36:18.787 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.767+0000 7f14c3bf3b80 1 -- 172.21.15.67:0/662112233 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f7494955b0 con 0x55f7494a0010 2023-10-14T10:36:18.787 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.767+0000 7f148ffff700 1 -- 172.21.15.67:0/662112233 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f14a0014810 con 0x55f7494a0010 2023-10-14T10:36:18.788 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.767+0000 7f14c3bf3b80 10 client.4656 fetch_fsmap learned FSMap version 8 2023-10-14T10:36:18.788 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.767+0000 7f14c3bf3b80 1 -- 172.21.15.67:0/662112233 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x55f74921fd90 con 0x55f7494a0010 2023-10-14T10:36:18.788 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.767+0000 7f148ffff700 1 -- 172.21.15.67:0/662112233 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f7494955b0 con 0x55f7494a0010 2023-10-14T10:36:18.788 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.768+0000 7f148ffff700 1 -- 172.21.15.67:0/662112233 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 8) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f14a0014970 con 0x55f7494a0010 2023-10-14T10:36:18.788 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.768+0000 7f14c3bf3b80 10 client.4656 fetch_fsmap finished waiting for FSMap version 8 2023-10-14T10:36:18.789 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.768+0000 7f14c3bf3b80 10 client.4656 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-14T10:36:18.789 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.768+0000 7f14c3bf3b80 20 client.4656 populate_metadata read hostname 'smithi067' 2023-10-14T10:36:18.789 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.768+0000 7f14c3bf3b80 1 --2- 172.21.15.67:0/662112233 >> [v2:172.21.15.72:6835/757500960,v1:172.21.15.72:6837/757500960] conn(0x55f7494c6780 0x55f7494fa910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:18.789 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.768+0000 7f14c3bf3b80 4 client.4656 mds_command: new command op to 4267 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-14T10:36:18.789 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.768+0000 7f14c3bf3b80 1 -- 172.21.15.67:0/662112233 --> [v2:172.21.15.72:6835/757500960,v1:172.21.15.72:6837/757500960] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f7493ede00 con 0x55f7494c6780 2023-10-14T10:36:18.790 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.768+0000 7f14b4fa4700 1 --2- 172.21.15.67:0/662112233 >> [v2:172.21.15.72:6835/757500960,v1:172.21.15.72:6837/757500960] conn(0x55f7494c6780 0x55f7494fa910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:18.790 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.768+0000 7f148ffff700 10 client.4656 ms_handle_connect on v2:172.21.15.72:6835/757500960 2023-10-14T10:36:18.790 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.768+0000 7f14b4fa4700 1 --2- 172.21.15.67:0/662112233 >> [v2:172.21.15.72:6835/757500960,v1:172.21.15.72:6837/757500960] conn(0x55f7494c6780 0x55f7494fa910 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c0bc709c24215b9 server_cookie=8bc08c886d94d3da in_seq=0 out_seq=0 2023-10-14T10:36:18.790 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.769+0000 7f148ffff700 1 -- 172.21.15.67:0/662112233 <== mds.0 v2:172.21.15.72:6835/757500960 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f7493ede00 con 0x55f7494c6780 2023-10-14T10:36:18.790 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.769+0000 7f148ffff700 10 client.4656 handle_command_reply: tid=0 2023-10-14T10:36:18.791 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.769+0000 7f14c3bf3b80 1 client.4656 shutdown 2023-10-14T10:36:18.791 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.769+0000 7f14c3bf3b80 1 -- 172.21.15.67:0/662112233 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f7494a0010 msgr2=0x55f7494cc400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:18.791 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.769+0000 7f14c3bf3b80 1 --2- 172.21.15.67:0/662112233 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f7494a0010 0x55f7494cc400 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f14a0011ec0 tx=0x7f14a0004750 comp rx=0 tx=0).stop 2023-10-14T10:36:18.791 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.769+0000 7f14c3bf3b80 1 -- 172.21.15.67:0/662112233 shutdown_connections 2023-10-14T10:36:18.791 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.769+0000 7f14c3bf3b80 1 --2- 172.21.15.67:0/662112233 >> [v2:172.21.15.72:6835/757500960,v1:172.21.15.72:6837/757500960] conn(0x55f7494c6780 0x55f7494fa910 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:18.792 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.769+0000 7f14c3bf3b80 1 --2- 172.21.15.67:0/662112233 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f7494ca630 0x55f7494c6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:18.792 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.769+0000 7f14c3bf3b80 1 --2- 172.21.15.67:0/662112233 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f7494a0010 0x55f7494cc400 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:18.792 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.769+0000 7f14c3bf3b80 1 --2- 172.21.15.67:0/662112233 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x55f7494c5220 0x55f7494c84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:18.792 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.769+0000 7f148ffff700 0 client.4656 ms_handle_reset on v2:172.21.15.72:6835/757500960 2023-10-14T10:36:18.792 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.769+0000 7f14c3bf3b80 1 -- 172.21.15.67:0/662112233 >> 172.21.15.67:0/662112233 conn(0x55f7493cfd80 msgr2=0x55f7494e8450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:18.792 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.769+0000 7f14c3bf3b80 1 -- 172.21.15.67:0/662112233 shutdown_connections 2023-10-14T10:36:18.793 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.769+0000 7f14c3bf3b80 1 -- 172.21.15.67:0/662112233 wait complete. 2023-10-14T10:36:18.793 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.769+0000 7f14c3bf3b80 20 client.4656 trim_cache size 0 max 16384 2023-10-14T10:36:18.798 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.787+0000 7f14c3bf3b80 1 Processor -- start 2023-10-14T10:36:18.799 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.787+0000 7f14c3bf3b80 1 -- start start 2023-10-14T10:36:18.799 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.788+0000 7f14c3bf3b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f7494a0010 0x55f7494cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:18.799 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.788+0000 7f14c3bf3b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f7494c5220 0x55f749500b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:18.799 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.788+0000 7f14c3bf3b80 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x55f7494ca630 0x55f7494b4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:18.799 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.788+0000 7f14c3bf3b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7492758d0 con 0x55f7494a0010 2023-10-14T10:36:18.800 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.788+0000 7f14c3bf3b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f74949e960 con 0x55f7494c5220 2023-10-14T10:36:18.800 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.788+0000 7f14c3bf3b80 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f7494fa220 con 0x55f7494ca630 2023-10-14T10:36:18.800 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.788+0000 7f14b57a5700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f7494a0010 0x55f7494cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:18.800 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.788+0000 7f14b57a5700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f7494a0010 0x55f7494cc210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44552/0 (socket says 172.21.15.67:44552) 2023-10-14T10:36:18.800 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.788+0000 7f14b57a5700 1 -- 172.21.15.67:0/1361563202 learned_addr learned my addr 172.21.15.67:0/1361563202 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:18.801 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.788+0000 7f14b4fa4700 1 --2- 172.21.15.67:0/1361563202 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f7494c5220 0x55f749500b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:18.801 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.788+0000 7f14b57a5700 1 -- 172.21.15.67:0/1361563202 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x55f7494ca630 msgr2=0x55f7494b4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:18.801 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.788+0000 7f14b5fa6700 1 --2- 172.21.15.67:0/1361563202 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x55f7494ca630 0x55f7494b4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:18.801 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.788+0000 7f14b57a5700 1 --2- 172.21.15.67:0/1361563202 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x55f7494ca630 0x55f7494b4750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:18.801 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.788+0000 7f14b57a5700 1 -- 172.21.15.67:0/1361563202 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f7494c5220 msgr2=0x55f749500b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:18.802 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.788+0000 7f14b57a5700 1 --2- 172.21.15.67:0/1361563202 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f7494c5220 0x55f749500b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:18.802 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.788+0000 7f14b57a5700 1 -- 172.21.15.67:0/1361563202 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f749496af0 con 0x55f7494a0010 2023-10-14T10:36:18.802 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.789+0000 7f14b57a5700 1 --2- 172.21.15.67:0/1361563202 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f7494a0010 0x55f7494cc210 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f14a00046b0 tx=0x7f14a0024000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=907909b732f7d3a2 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:18.802 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.789+0000 7f149b7fe700 1 -- 172.21.15.67:0/1361563202 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a0014650 con 0x55f7494a0010 2023-10-14T10:36:18.802 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.789+0000 7f149b7fe700 1 -- 172.21.15.67:0/1361563202 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14a00147b0 con 0x55f7494a0010 2023-10-14T10:36:18.803 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.789+0000 7f149b7fe700 1 -- 172.21.15.67:0/1361563202 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a002c440 con 0x55f7494a0010 2023-10-14T10:36:18.803 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.789+0000 7f14c3bf3b80 1 -- 172.21.15.67:0/1361563202 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f7494a0010 msgr2=0x55f7494cc210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:18.803 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.789+0000 7f14c3bf3b80 1 --2- 172.21.15.67:0/1361563202 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f7494a0010 0x55f7494cc210 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f14a00046b0 tx=0x7f14a0024000 comp rx=0 tx=0).stop 2023-10-14T10:36:18.803 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.789+0000 7f14c3bf3b80 1 -- 172.21.15.67:0/1361563202 shutdown_connections 2023-10-14T10:36:18.803 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.789+0000 7f14c3bf3b80 1 --2- 172.21.15.67:0/1361563202 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x55f7494ca630 0x55f7494b4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:18.804 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.789+0000 7f14c3bf3b80 1 --2- 172.21.15.67:0/1361563202 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f7494a0010 0x55f7494cc210 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:18.804 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.789+0000 7f14c3bf3b80 1 --2- 172.21.15.67:0/1361563202 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f7494c5220 0x55f749500b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:18.804 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.789+0000 7f14c3bf3b80 1 -- 172.21.15.67:0/1361563202 >> 172.21.15.67:0/1361563202 conn(0x55f7493cfd80 msgr2=0x55f7494c99a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:18.804 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.789+0000 7f14c3bf3b80 1 -- 172.21.15.67:0/1361563202 shutdown_connections 2023-10-14T10:36:18.804 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.789+0000 7f14c3bf3b80 1 -- 172.21.15.67:0/1361563202 wait complete. 2023-10-14T10:36:18.804 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.790+0000 7f14c3bf3b80 1 -- start start 2023-10-14T10:36:18.805 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.790+0000 7f14c3bf3b80 1 Processor -- start 2023-10-14T10:36:18.805 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.791+0000 7f14c3bf3b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f7494a0010 0x55f74954fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:18.805 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.791+0000 7f14c3bf3b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f7494c5220 0x55f7494c5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:18.805 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.791+0000 7f14c3bf3b80 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x55f7494ca630 0x55f7494c99a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:18.805 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.791+0000 7f14c3bf3b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7495504e0 con 0x55f7494c5220 2023-10-14T10:36:18.806 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.791+0000 7f14c3bf3b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7494e23e0 con 0x55f7494a0010 2023-10-14T10:36:18.806 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.791+0000 7f14c3bf3b80 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f749497730 con 0x55f7494ca630 2023-10-14T10:36:18.806 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.791+0000 7f14c3bf3b80 10 client.0 _ll_register_callbacks cb 0x7f14a0011860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-14T10:36:18.806 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.791+0000 7f14b4fa4700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f7494c5220 0x55f7494c5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:18.806 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.791+0000 7f14b4fa4700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f7494c5220 0x55f7494c5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44562/0 (socket says 172.21.15.67:44562) 2023-10-14T10:36:18.807 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.791+0000 7f14b4fa4700 1 -- 172.21.15.67:0/2915502409 learned_addr learned my addr 172.21.15.67:0/2915502409 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:18.807 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.791+0000 7f14b5fa6700 1 --2- 172.21.15.67:0/2915502409 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x55f7494ca630 0x55f7494c99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:18.807 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.791+0000 7f14b57a5700 1 --2- 172.21.15.67:0/2915502409 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f7494a0010 0x55f74954fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:18.807 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.791+0000 7f14b4fa4700 1 -- 172.21.15.67:0/2915502409 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x55f7494ca630 msgr2=0x55f7494c99a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:18.807 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.791+0000 7f14b4fa4700 1 --2- 172.21.15.67:0/2915502409 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x55f7494ca630 0x55f7494c99a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:18.808 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.791+0000 7f14b4fa4700 1 -- 172.21.15.67:0/2915502409 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f7494a0010 msgr2=0x55f74954fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:18.808 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.791+0000 7f14b4fa4700 1 --2- 172.21.15.67:0/2915502409 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f7494a0010 0x55f74954fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:18.808 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.791+0000 7f14b4fa4700 1 -- 172.21.15.67:0/2915502409 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f74921fd90 con 0x55f7494c5220 2023-10-14T10:36:18.808 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.791+0000 7f14b4fa4700 1 --2- 172.21.15.67:0/2915502409 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f7494c5220 0x55f7494c5a10 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f14ac0591d0 tx=0x7f14ac059350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a96609402e3f34b server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:18.808 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.791+0000 7f148ffff700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2023-10-14T10:36:18.809 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.792+0000 7f148ffff700 1 -- 172.21.15.67:0/2915502409 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14ac05d620 con 0x55f7494c5220 2023-10-14T10:36:18.809 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.792+0000 7f14c3bf3b80 1 -- 172.21.15.67:0/2915502409 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f14a8016650 con 0x55f7494c5220 2023-10-14T10:36:18.809 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.792+0000 7f148ffff700 1 -- 172.21.15.67:0/2915502409 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f14ac0617d0 con 0x55f7494c5220 2023-10-14T10:36:18.809 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.792+0000 7f148ffff700 1 -- 172.21.15.67:0/2915502409 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f14ac079070 con 0x55f7494c5220 2023-10-14T10:36:18.809 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.792+0000 7f14c3bf3b80 10 client.4662 fetch_fsmap learned FSMap version 8 2023-10-14T10:36:18.809 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.792+0000 7f14c3bf3b80 1 -- 172.21.15.67:0/2915502409 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x55f749208c10 con 0x55f7494c5220 2023-10-14T10:36:18.810 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.792+0000 7f148ffff700 1 -- 172.21.15.67:0/2915502409 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14a8016650 con 0x55f7494c5220 2023-10-14T10:36:18.810 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.792+0000 7f148ffff700 1 -- 172.21.15.67:0/2915502409 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 8) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f14ac00b3e0 con 0x55f7494c5220 2023-10-14T10:36:18.810 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.792+0000 7f14c3bf3b80 10 client.4662 fetch_fsmap finished waiting for FSMap version 8 2023-10-14T10:36:18.810 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.792+0000 7f14c3bf3b80 10 client.4662 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-14T10:36:18.810 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.792+0000 7f14c3bf3b80 20 client.4662 populate_metadata read hostname 'smithi067' 2023-10-14T10:36:18.811 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.792+0000 7f14c3bf3b80 1 --2- 172.21.15.67:0/2915502409 >> [v2:172.21.15.72:6835/757500960,v1:172.21.15.72:6837/757500960] conn(0x55f7494d05e0 0x55f7494c9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:18.811 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.792+0000 7f14c3bf3b80 4 client.4662 mds_command: new command op to 4267 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-14T10:36:18.811 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.792+0000 7f14c3bf3b80 1 -- 172.21.15.67:0/2915502409 --> [v2:172.21.15.72:6835/757500960,v1:172.21.15.72:6837/757500960] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55f7493edb20 con 0x55f7494d05e0 2023-10-14T10:36:18.811 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.792+0000 7f14b57a5700 1 --2- 172.21.15.67:0/2915502409 >> [v2:172.21.15.72:6835/757500960,v1:172.21.15.72:6837/757500960] conn(0x55f7494d05e0 0x55f7494c9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:18.811 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.793+0000 7f148ffff700 10 client.4662 ms_handle_connect on v2:172.21.15.72:6835/757500960 2023-10-14T10:36:18.812 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.793+0000 7f14b57a5700 1 --2- 172.21.15.67:0/2915502409 >> [v2:172.21.15.72:6835/757500960,v1:172.21.15.72:6837/757500960] conn(0x55f7494d05e0 0x55f7494c9ee0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a9e24859e677db1 server_cookie=3d8005d249fd28bb in_seq=0 out_seq=0 2023-10-14T10:36:18.812 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.793+0000 7f148ffff700 1 -- 172.21.15.67:0/2915502409 <== mds.0 v2:172.21.15.72:6835/757500960 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x55f7493edb20 con 0x55f7494d05e0 2023-10-14T10:36:18.812 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.793+0000 7f148ffff700 10 client.4662 handle_command_reply: tid=0 2023-10-14T10:36:18.812 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.793+0000 7f14c3bf3b80 1 client.4662 shutdown 2023-10-14T10:36:18.812 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.793+0000 7f14c3bf3b80 1 -- 172.21.15.67:0/2915502409 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f7494c5220 msgr2=0x55f7494c5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:18.812 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.793+0000 7f14c3bf3b80 1 --2- 172.21.15.67:0/2915502409 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f7494c5220 0x55f7494c5a10 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f14ac0591d0 tx=0x7f14ac059350 comp rx=0 tx=0).stop 2023-10-14T10:36:18.813 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.793+0000 7f14c3bf3b80 1 -- 172.21.15.67:0/2915502409 shutdown_connections 2023-10-14T10:36:18.813 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.793+0000 7f14c3bf3b80 1 --2- 172.21.15.67:0/2915502409 >> [v2:172.21.15.72:6835/757500960,v1:172.21.15.72:6837/757500960] conn(0x55f7494d05e0 0x55f7494c9ee0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:18.813 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.793+0000 7f14c3bf3b80 1 --2- 172.21.15.67:0/2915502409 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x55f7494ca630 0x55f7494c99a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:18.813 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.793+0000 7f14c3bf3b80 1 --2- 172.21.15.67:0/2915502409 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f7494a0010 0x55f74954fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:18.813 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.793+0000 7f148ffff700 0 client.4662 ms_handle_reset on v2:172.21.15.72:6835/757500960 2023-10-14T10:36:18.814 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.793+0000 7f14c3bf3b80 1 --2- 172.21.15.67:0/2915502409 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f7494c5220 0x55f7494c5a10 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:18.814 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.793+0000 7f14c3bf3b80 1 -- 172.21.15.67:0/2915502409 >> 172.21.15.67:0/2915502409 conn(0x55f7493cfd80 msgr2=0x55f749500d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:18.814 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.793+0000 7f14c3bf3b80 1 -- 172.21.15.67:0/2915502409 shutdown_connections 2023-10-14T10:36:18.814 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.793+0000 7f14c3bf3b80 1 -- 172.21.15.67:0/2915502409 wait complete. 2023-10-14T10:36:18.814 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.793+0000 7f14c3bf3b80 20 client.4662 trim_cache size 0 max 16384 2023-10-14T10:36:18.815 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2023-10-14T10:36:18.815 INFO:teuthology.orchestra.run.smithi067.stdout: "cluster_fsid": "37f9ab46-3082-4b71-8b24-08d39897b008", 2023-10-14T10:36:18.815 INFO:teuthology.orchestra.run.smithi067.stdout: "whoami": 0, 2023-10-14T10:36:18.816 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 4267, 2023-10-14T10:36:18.816 INFO:teuthology.orchestra.run.smithi067.stdout: "want_state": "up:active", 2023-10-14T10:36:18.816 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "up:active", 2023-10-14T10:36:18.816 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs", 2023-10-14T10:36:18.816 INFO:teuthology.orchestra.run.smithi067.stdout: "rank_uptime": 14.955707844000001, 2023-10-14T10:36:18.816 INFO:teuthology.orchestra.run.smithi067.stdout: "mdsmap_epoch": 8, 2023-10-14T10:36:18.817 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch": 22, 2023-10-14T10:36:18.817 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch_barrier": 22, 2023-10-14T10:36:18.817 INFO:teuthology.orchestra.run.smithi067.stdout: "uptime": 21.674762182999999 2023-10-14T10:36:18.817 INFO:teuthology.orchestra.run.smithi067.stdout:} 2023-10-14T10:36:18.819 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.794+0000 7f14b7a09700 1 -- 172.21.15.67:0/3518902999 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f149406af00 msgr2=0x7f149406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:18.819 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.794+0000 7f14b7a09700 1 --2- 172.21.15.67:0/3518902999 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f149406af00 0x7f149406d3c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f14a0009be0 tx=0x7f14a0009210 comp rx=0 tx=0).stop 2023-10-14T10:36:18.819 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.794+0000 7f14b7a09700 1 -- 172.21.15.67:0/3518902999 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14b015dce0 msgr2=0x7f14b00cb2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:18.819 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.794+0000 7f14b7a09700 1 --2- 172.21.15.67:0/3518902999 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14b015dce0 0x7f14b00cb2f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f14ac014040 tx=0x7f14ac004750 comp rx=0 tx=0).stop 2023-10-14T10:36:18.819 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.794+0000 7f14b7a09700 1 -- 172.21.15.67:0/3518902999 shutdown_connections 2023-10-14T10:36:18.820 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.794+0000 7f14b7a09700 1 --2- 172.21.15.67:0/3518902999 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f149406af00 0x7f149406d3c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:18.820 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.794+0000 7f14b7a09700 1 --2- 172.21.15.67:0/3518902999 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f14b015e0c0 0x7f14b01f6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:18.820 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.794+0000 7f14b7a09700 1 --2- 172.21.15.67:0/3518902999 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f14b00cfa30 0x7f14b00c7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:18.820 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.794+0000 7f14b7a09700 1 --2- 172.21.15.67:0/3518902999 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14b015dce0 0x7f14b00cb2f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:18.820 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.794+0000 7f14b7a09700 1 -- 172.21.15.67:0/3518902999 >> 172.21.15.67:0/3518902999 conn(0x7f14b00d15f0 msgr2=0x7f14b015ccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:18.821 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.794+0000 7f14b7a09700 1 -- 172.21.15.67:0/3518902999 shutdown_connections 2023-10-14T10:36:18.821 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:18.794+0000 7f14b7a09700 1 -- 172.21.15.67:0/3518902999 wait complete. 2023-10-14T10:36:18.821 INFO:teuthology.run_tasks:Running task kclient... 2023-10-14T10:36:18.843 INFO:tasks.kclient:Mounting kernel clients... 2023-10-14T10:36:18.844 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-10-14T10:36:18.844 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:36:18.845 DEBUG:teuthology.orchestra.run.smithi067:> ip netns list 2023-10-14T10:36:18.862 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:36:18.863 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link delete ceph-brx 2023-10-14T10:36:18.932 INFO:teuthology.orchestra.run.smithi067.stderr:Cannot find device "ceph-brx" 2023-10-14T10:36:18.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-14T10:36:18.934 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:36:18.934 DEBUG:teuthology.orchestra.run.smithi067:> ip netns list 2023-10-14T10:36:18.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:36:18.991 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link delete ceph-brx 2023-10-14T10:36:19.059 INFO:teuthology.orchestra.run.smithi067.stderr:Cannot find device "ceph-brx" 2023-10-14T10:36:19.060 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-14T10:36:19.061 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:36:19.061 DEBUG:teuthology.orchestra.run.smithi072:> ip netns list 2023-10-14T10:36:19.082 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:36:19.083 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip link delete ceph-brx 2023-10-14T10:36:19.157 INFO:teuthology.orchestra.run.smithi072.stderr:Cannot find device "ceph-brx" 2023-10-14T10:36:19.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-14T10:36:19.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:36:19.159 DEBUG:teuthology.orchestra.run.smithi072:> ip netns list 2023-10-14T10:36:19.218 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:36:19.218 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip link delete ceph-brx 2023-10-14T10:36:19.286 INFO:teuthology.orchestra.run.smithi072.stderr:Cannot find device "ceph-brx" 2023-10-14T10:36:19.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-14T10:36:19.288 INFO:tasks.kclient:client.0 config is {'syntax': 'v2'} 2023-10-14T10:36:19.289 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-10-14T10:36:19.289 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-14T10:36:19.290 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-14T10:36:19.417 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:19.415+0000 7f2599713700 1 Processor -- start 2023-10-14T10:36:19.417 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:19.415+0000 7f2599713700 1 -- start start 2023-10-14T10:36:19.423 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:19.416+0000 7f2599713700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f25940d3fc0 0x7f259415dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:19.423 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:19.416+0000 7f2599713700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f25940b7050 0x7f25940d21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:19.424 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:19.416+0000 7f2599713700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f259415e4e0 con 0x7f259415dbc0 2023-10-14T10:36:19.424 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:19.416+0000 7f2599713700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25940b7560 con 0x7f25940b7050 2023-10-14T10:36:19.424 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:19.416+0000 7f2599713700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25940b7840 con 0x7f25940d3fc0 2023-10-14T10:36:19.424 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:19.416+0000 7f2592ffd700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f259415dbc0 0x7f25940d1ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:52334/0 (socket says 172.21.15.67:52334) 2023-10-14T10:36:19.424 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:19.416+0000 7f2592ffd700 1 -- 172.21.15.67:0/3354906560 learned_addr learned my addr 172.21.15.67:0/3354906560 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:36:19.425 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:19.416+0000 7f25937fe700 1 --2- 172.21.15.67:0/3354906560 >> v2:172.21.15.72:3300/0 conn(0x7f25940b7050 0x7f25940d21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:19.425 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:19.416+0000 7f2591ffb700 1 -- 172.21.15.67:0/3354906560 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3507908548 0 0) 0x7f259415e4e0 con 0x7f259415dbc0 2023-10-14T10:36:19.425 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:19.416+0000 7f25927fc700 1 --2- 172.21.15.67:0/3354906560 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f25940d3fc0 0x7f259415dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:19.425 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:19.416+0000 7f2591ffb700 1 -- 172.21.15.67:0/3354906560 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f257c0035b0 con 0x7f259415dbc0 2023-10-14T10:36:19.426 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:19.416+0000 7f2591ffb700 1 -- 172.21.15.67:0/3354906560 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3632220858 0 0) 0x7f257c0035b0 con 0x7f259415dbc0 2023-10-14T10:36:19.426 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:19.416+0000 7f2591ffb700 1 -- 172.21.15.67:0/3354906560 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f25940d3fc0 msgr2=0x7f259415dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:19.426 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:19.416+0000 7f2591ffb700 1 --2- 172.21.15.67:0/3354906560 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f25940d3fc0 0x7f259415dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:19.426 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:19.416+0000 7f2591ffb700 1 -- 172.21.15.67:0/3354906560 >> v2:172.21.15.72:3300/0 conn(0x7f25940b7050 msgr2=0x7f25940d21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:19.426 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:19.416+0000 7f2591ffb700 1 --2- 172.21.15.67:0/3354906560 >> v2:172.21.15.72:3300/0 conn(0x7f25940b7050 0x7f25940d21d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:19.427 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:19.416+0000 7f2591ffb700 1 -- 172.21.15.67:0/3354906560 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25940b7b20 con 0x7f259415dbc0 2023-10-14T10:36:19.427 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:19.416+0000 7f2591ffb700 1 -- 172.21.15.67:0/3354906560 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3180043755 0 0) 0x7f25880017f0 con 0x7f259415dbc0 2023-10-14T10:36:19.427 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:19.416+0000 7f2591ffb700 1 -- 172.21.15.67:0/3354906560 >> v1:172.21.15.67:6789/0 conn(0x7f259415dbc0 legacy=0x7f25940d1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:19.427 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:19.416+0000 7f2591ffb700 1 --2- 172.21.15.67:0/3354906560 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f257c003a00 0x7f257c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:19.427 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:19.416+0000 7f2591ffb700 1 --2- 172.21.15.67:0/3354906560 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f25940d3fc0 0x7f257c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:19.428 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:19.416+0000 7f2591ffb700 1 --2- 172.21.15.67:0/3354906560 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f257c004d90 0x7f257c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:19.428 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:19.417+0000 7f2591ffb700 1 -- 172.21.15.67:0/3354906560 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25940b7560 con 0x7f257c004d90 2023-10-14T10:36:19.428 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:19.417+0000 7f2591ffb700 1 -- 172.21.15.67:0/3354906560 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25940b7840 con 0x7f257c003a00 2023-10-14T10:36:19.428 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:19.417+0000 7f2591ffb700 1 -- 172.21.15.67:0/3354906560 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f257c00ba10 con 0x7f25940d3fc0 2023-10-14T10:36:19.428 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:19.417+0000 7f25937fe700 1 --2- 172.21.15.67:0/3354906560 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f257c004d90 0x7f257c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:19.428 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:19.417+0000 7f25927fc700 1 --2- 172.21.15.67:0/3354906560 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f25940d3fc0 0x7f257c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:19.429 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:19.417+0000 7f2592ffd700 1 --2- 172.21.15.67:0/3354906560 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f257c003a00 0x7f257c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:19.429 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:19.417+0000 7f25937fe700 1 -- 172.21.15.67:0/3354906560 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f25940d3fc0 msgr2=0x7f257c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:19.429 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:19.417+0000 7f25937fe700 1 --2- 172.21.15.67:0/3354906560 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f25940d3fc0 0x7f257c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:19.429 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:19.417+0000 7f25937fe700 1 -- 172.21.15.67:0/3354906560 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f257c003a00 msgr2=0x7f257c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:19.430 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:19.417+0000 7f25937fe700 1 --2- 172.21.15.67:0/3354906560 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f257c003a00 0x7f257c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:19.430 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:19.417+0000 7f25937fe700 1 -- 172.21.15.67:0/3354906560 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f257c0035b0 con 0x7f257c004d90 2023-10-14T10:36:19.430 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:19.417+0000 7f25927fc700 1 --2- 172.21.15.67:0/3354906560 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f25940d3fc0 0x7f257c004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-14T10:36:19.430 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:19.417+0000 7f25937fe700 1 --2- 172.21.15.67:0/3354906560 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f257c004d90 0x7f257c005210 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f258c009f90 tx=0x7f257c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdbbbd05ea028c23 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:19.430 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:19.417+0000 7f2591ffb700 1 -- 172.21.15.67:0/3354906560 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258c00ec30 con 0x7f257c004d90 2023-10-14T10:36:19.431 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:19.417+0000 7f2591ffb700 1 -- 172.21.15.67:0/3354906560 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f258c00f040 con 0x7f257c004d90 2023-10-14T10:36:19.431 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:19.417+0000 7f2591ffb700 1 -- 172.21.15.67:0/3354906560 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258c014480 con 0x7f257c004d90 2023-10-14T10:36:19.431 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:19.418+0000 7f2599713700 1 -- 172.21.15.67:0/3354906560 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f257c004d90 msgr2=0x7f257c005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:19.431 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:19.418+0000 7f2599713700 1 --2- 172.21.15.67:0/3354906560 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f257c004d90 0x7f257c005210 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f258c009f90 tx=0x7f257c005880 comp rx=0 tx=0).stop 2023-10-14T10:36:19.431 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:19.418+0000 7f25937fe700 1 -- 172.21.15.67:0/3354906560 reap_dead start 2023-10-14T10:36:19.432 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:19.418+0000 7f2599713700 1 -- 172.21.15.67:0/3354906560 shutdown_connections 2023-10-14T10:36:19.432 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:19.418+0000 7f2599713700 1 -- 172.21.15.67:0/3354906560 >> 172.21.15.67:0/3354906560 conn(0x7f259409cb40 msgr2=0x7f25940d3820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:19.432 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:19.418+0000 7f2599713700 1 -- 172.21.15.67:0/3354906560 shutdown_connections 2023-10-14T10:36:19.432 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:19.418+0000 7f2599713700 1 -- 172.21.15.67:0/3354906560 wait complete. 2023-10-14T10:36:19.432 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:19.419+0000 7f2599713700 1 Processor -- start 2023-10-14T10:36:19.433 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:19.419+0000 7f2599713700 1 -- start start 2023-10-14T10:36:19.433 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:19.420+0000 7f2599713700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f257c004d90 0x7f2594152db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:19.433 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:19.420+0000 7f2599713700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2594158780 0x7f2594158c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:19.433 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:19.420+0000 7f2599713700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25940ccb30 0x7f25940ccfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:19.433 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:19.420+0000 7f2599713700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25940cd4f0 con 0x7f25940ccb30 2023-10-14T10:36:19.433 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:19.420+0000 7f2599713700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25941f76b0 con 0x7f2594158780 2023-10-14T10:36:19.434 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:19.420+0000 7f2599713700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25941f6aa0 con 0x7f257c004d90 2023-10-14T10:36:19.434 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:19.420+0000 7f25937fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25940ccb30 0x7f25940ccfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:19.434 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:19.420+0000 7f25937fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25940ccb30 0x7f25940ccfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44582/0 (socket says 172.21.15.67:44582) 2023-10-14T10:36:19.434 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:19.420+0000 7f25937fe700 1 -- 172.21.15.67:0/1437522643 learned_addr learned my addr 172.21.15.67:0/1437522643 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:19.435 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:19.420+0000 7f2592ffd700 1 --2- 172.21.15.67:0/1437522643 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f257c004d90 0x7f2594152db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:19.435 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:19.420+0000 7f25927fc700 1 --2- 172.21.15.67:0/1437522643 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2594158780 0x7f2594158c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:19.435 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:19.420+0000 7f25937fe700 1 -- 172.21.15.67:0/1437522643 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f257c004d90 msgr2=0x7f2594152db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:19.435 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:19.420+0000 7f25937fe700 1 --2- 172.21.15.67:0/1437522643 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f257c004d90 0x7f2594152db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:19.435 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:19.420+0000 7f25937fe700 1 -- 172.21.15.67:0/1437522643 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2594158780 msgr2=0x7f2594158c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:19.436 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:19.420+0000 7f25937fe700 1 --2- 172.21.15.67:0/1437522643 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2594158780 0x7f2594158c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:19.436 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:19.420+0000 7f25937fe700 1 -- 172.21.15.67:0/1437522643 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25941f6d80 con 0x7f25940ccb30 2023-10-14T10:36:19.436 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:19.420+0000 7f25927fc700 1 --2- 172.21.15.67:0/1437522643 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2594158780 0x7f2594158c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-14T10:36:19.436 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:19.420+0000 7f25937fe700 1 --2- 172.21.15.67:0/1437522643 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25940ccb30 0x7f25940ccfb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f257c0046e0 tx=0x7f258c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:19.436 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:19.421+0000 7f257b7fe700 1 -- 172.21.15.67:0/1437522643 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258c007000 con 0x7f25940ccb30 2023-10-14T10:36:19.437 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:19.421+0000 7f257b7fe700 1 -- 172.21.15.67:0/1437522643 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f258c003e50 con 0x7f25940ccb30 2023-10-14T10:36:19.437 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:19.421+0000 7f2599713700 1 -- 172.21.15.67:0/1437522643 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25941f7060 con 0x7f25940ccb30 2023-10-14T10:36:19.437 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:19.421+0000 7f257b7fe700 1 -- 172.21.15.67:0/1437522643 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258c014a20 con 0x7f25940ccb30 2023-10-14T10:36:19.437 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:19.421+0000 7f2599713700 1 -- 172.21.15.67:0/1437522643 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f259409d4b0 con 0x7f25940ccb30 2023-10-14T10:36:19.438 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:19.422+0000 7f2599713700 1 -- 172.21.15.67:0/1437522643 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f259404ebb0 con 0x7f25940ccb30 2023-10-14T10:36:19.439 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:19.422+0000 7f257b7fe700 1 -- 172.21.15.67:0/1437522643 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f258c024020 con 0x7f25940ccb30 2023-10-14T10:36:19.439 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:19.422+0000 7f257b7fe700 1 --2- 172.21.15.67:0/1437522643 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f258006af00 0x7f258006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:19.439 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:19.423+0000 7f257b7fe700 1 -- 172.21.15.67:0/1437522643 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f258c08aa10 con 0x7f25940ccb30 2023-10-14T10:36:19.439 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:19.423+0000 7f2592ffd700 1 --2- 172.21.15.67:0/1437522643 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f258006af00 0x7f258006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:19.439 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:19.423+0000 7f2592ffd700 1 --2- 172.21.15.67:0/1437522643 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f258006af00 0x7f258006d3c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f25940cda20 tx=0x7f2588009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:19.440 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:19.427+0000 7f257b7fe700 1 -- 172.21.15.67:0/1437522643 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f258c055b50 con 0x7f25940ccb30 2023-10-14T10:36:19.674 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:19.672+0000 7f2599713700 1 -- 172.21.15.67:0/1437522643 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f2594073be0 con 0x7f25940ccb30 2023-10-14T10:36:19.674 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:19.672+0000 7f257b7fe700 1 -- 172.21.15.67:0/1437522643 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f258c02c070 con 0x7f25940ccb30 2023-10-14T10:36:19.675 INFO:teuthology.orchestra.run.smithi067.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-14T10:36:19.676 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:19.674+0000 7f2599713700 1 -- 172.21.15.67:0/1437522643 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f258006af00 msgr2=0x7f258006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:19.676 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:19.674+0000 7f2599713700 1 --2- 172.21.15.67:0/1437522643 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f258006af00 0x7f258006d3c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f25940cda20 tx=0x7f2588009040 comp rx=0 tx=0).stop 2023-10-14T10:36:19.676 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:19.674+0000 7f2599713700 1 -- 172.21.15.67:0/1437522643 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25940ccb30 msgr2=0x7f25940ccfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:19.677 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:19.674+0000 7f2599713700 1 --2- 172.21.15.67:0/1437522643 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25940ccb30 0x7f25940ccfb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f257c0046e0 tx=0x7f258c009750 comp rx=0 tx=0).stop 2023-10-14T10:36:19.677 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:19.674+0000 7f2599713700 1 -- 172.21.15.67:0/1437522643 shutdown_connections 2023-10-14T10:36:19.677 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:19.674+0000 7f2599713700 1 --2- 172.21.15.67:0/1437522643 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f258006af00 0x7f258006d3c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:19.677 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:19.674+0000 7f2599713700 1 --2- 172.21.15.67:0/1437522643 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25940ccb30 0x7f25940ccfb0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:19.677 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:19.674+0000 7f2599713700 1 --2- 172.21.15.67:0/1437522643 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f257c004d90 0x7f2594152db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:19.678 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:19.674+0000 7f2599713700 1 --2- 172.21.15.67:0/1437522643 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2594158780 0x7f2594158c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:19.678 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:19.674+0000 7f2599713700 1 -- 172.21.15.67:0/1437522643 >> 172.21.15.67:0/1437522643 conn(0x7f259409cb40 msgr2=0x7f25940cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:19.678 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:19.674+0000 7f2599713700 1 -- 172.21.15.67:0/1437522643 shutdown_connections 2023-10-14T10:36:19.678 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:19.674+0000 7f2599713700 1 -- 172.21.15.67:0/1437522643 wait complete. 2023-10-14T10:36:19.688 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-14T10:36:19.689 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-14T10:36:19.689 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi067.front.sepia.ceph.com 2023-10-14T10:36:19.689 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-14T10:36:19.689 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-14T10:36:19.690 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-14T10:36:19.690 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-14T10:36:19.690 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-14T10:36:19.690 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-14T10:36:19.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:36:19.691 DEBUG:teuthology.orchestra.run.smithi067:> ip addr 2023-10-14T10:36:19.708 INFO:teuthology.orchestra.run.smithi067.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-14T10:36:19.708 INFO:teuthology.orchestra.run.smithi067.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-14T10:36:19.708 INFO:teuthology.orchestra.run.smithi067.stdout: inet 127.0.0.1/8 scope host lo 2023-10-14T10:36:19.708 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2023-10-14T10:36:19.709 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 ::1/128 scope host 2023-10-14T10:36:19.709 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2023-10-14T10:36:19.709 INFO:teuthology.orchestra.run.smithi067.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-14T10:36:19.709 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:a8:b4:7e brd ff:ff:ff:ff:ff:ff 2023-10-14T10:36:19.709 INFO:teuthology.orchestra.run.smithi067.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-14T10:36:19.709 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:a8:b4:7f brd ff:ff:ff:ff:ff:ff 2023-10-14T10:36:19.710 INFO:teuthology.orchestra.run.smithi067.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-14T10:36:19.710 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:b6:fc:0a brd ff:ff:ff:ff:ff:ff 2023-10-14T10:36:19.710 INFO:teuthology.orchestra.run.smithi067.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-14T10:36:19.710 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:b6:fc:0b brd ff:ff:ff:ff:ff:ff 2023-10-14T10:36:19.710 INFO:teuthology.orchestra.run.smithi067.stdout: inet 172.21.15.67/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-14T10:36:19.711 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft 42359sec preferred_lft 42359sec 2023-10-14T10:36:19.711 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 fe80::5cc4:6a4b:a14e:a6d7/64 scope link noprefixroute 2023-10-14T10:36:19.711 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2023-10-14T10:36:19.711 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-14T10:36:19.712 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:36:19.712 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec bash -c ' 2023-10-14T10:36:19.712 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2023-10-14T10:36:19.713 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link add name ceph-brx type bridge 2023-10-14T10:36:19.713 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip addr flush dev ceph-brx 2023-10-14T10:36:19.713 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link set ceph-brx up 2023-10-14T10:36:19.713 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-14T10:36:19.714 DEBUG:teuthology.orchestra.run.smithi067:> ') 2023-10-14T10:36:19.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:36:19.882 DEBUG:teuthology.orchestra.run.smithi067:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-14T10:36:19.910 INFO:teuthology.orchestra.run.smithi067.stdout:1 2023-10-14T10:36:19.912 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:36:19.912 DEBUG:teuthology.orchestra.run.smithi067:> route 2023-10-14T10:36:19.984 INFO:teuthology.orchestra.run.smithi067.stdout:Kernel IP routing table 2023-10-14T10:36:19.984 INFO:teuthology.orchestra.run.smithi067.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-14T10:36:19.984 INFO:teuthology.orchestra.run.smithi067.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-14T10:36:19.984 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-14T10:36:19.985 INFO:teuthology.orchestra.run.smithi067.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-14T10:36:19.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:36:19.986 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec bash -c ' 2023-10-14T10:36:19.986 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2023-10-14T10:36:19.986 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-14T10:36:19.987 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-14T10:36:19.987 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-14T10:36:19.987 DEBUG:teuthology.orchestra.run.smithi067:> ') 2023-10-14T10:36:20.244 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:36:20.245 DEBUG:teuthology.orchestra.run.smithi067:> ip netns list 2023-10-14T10:36:20.264 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:36:20.264 DEBUG:teuthology.orchestra.run.smithi067:> ip netns list-id 2023-10-14T10:36:20.320 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:36:20.321 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec bash -c ' 2023-10-14T10:36:20.321 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2023-10-14T10:36:20.322 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-14T10:36:20.322 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-10-14T10:36:20.322 DEBUG:teuthology.orchestra.run.smithi067:> ') 2023-10-14T10:36:20.415 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-10-14T10:36:20.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:36:20.416 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec bash -c ' 2023-10-14T10:36:20.416 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2023-10-14T10:36:20.416 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-10-14T10:36:20.417 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-14T10:36:20.417 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-10-14T10:36:20.417 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-10-14T10:36:20.418 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-10-14T10:36:20.418 DEBUG:teuthology.orchestra.run.smithi067:> ') 2023-10-14T10:36:20.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:36:20.626 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec bash -c ' 2023-10-14T10:36:20.626 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2023-10-14T10:36:20.626 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link set brx.0 up 2023-10-14T10:36:20.627 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link set dev brx.0 master ceph-brx 2023-10-14T10:36:20.627 DEBUG:teuthology.orchestra.run.smithi067:> ') 2023-10-14T10:36:20.729 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-14T10:36:20.730 DEBUG:teuthology.orchestra.run.smithi067:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-14T10:36:20.750 INFO:teuthology.orchestra.run.smithi067.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-14T10:36:20.750 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-14T10:36:20.751 DEBUG:teuthology.orchestra.run.smithi067:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-14T10:36:20.807 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-14T10:36:20.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:36:20.808 DEBUG:teuthology.orchestra.run.smithi067:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-14T10:36:20.902 INFO:teuthology.orchestra.run.smithi067.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-14T10:36:20.902 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-14T10:36:21.007 INFO:teuthology.orchestra.run.smithi067.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-14T10:36:21.007 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-14T10:36:21.008 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: resolved to: "172.21.15.67:3300,172.21.15.72:3300,172.21.15.72:3301" 2023-10-14T10:36:21.008 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: trying mount with new device syntax: 0@37f9ab46-3082-4b71-8b24-08d39897b008.cephfs=/ 2023-10-14T10:36:21.008 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.67:3300/172.21.15.72:3300/172.21.15.72:3301" will pass to kernel 2023-10-14T10:36:21.009 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: v2 mount support check returned 2 2023-10-14T10:36:21.009 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-14T10:36:21.009 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-14T10:36:21.036 INFO:teuthology.orchestra.run.smithi067.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-14T10:36:21.036 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-14T10:36:21.036 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: resolved to: "172.21.15.67:3300,172.21.15.72:3300,172.21.15.72:3301" 2023-10-14T10:36:21.036 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: trying mount with new device syntax: 0@37f9ab46-3082-4b71-8b24-08d39897b008.cephfs=/ 2023-10-14T10:36:21.037 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.67:3300/172.21.15.72:3300/172.21.15.72:3301" will pass to kernel 2023-10-14T10:36:21.037 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: v2 mount support check returned 2 2023-10-14T10:36:21.037 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-14T10:36:21.037 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-14T10:36:21.037 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: resolved to: "172.21.15.67:6789,172.21.15.72:6789,172.21.15.72:6790" 2023-10-14T10:36:21.037 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.67:6789,172.21.15.72:6789,172.21.15.72:6790:/ 2023-10-14T10:36:21.038 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=37f9ab46-3082-4b71-8b24-08d39897b008" will pass to kernel 2023-10-14T10:36:21.053 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-14T10:36:21.054 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:36:21.054 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-14T10:36:21.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:36:21.083 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-14T10:36:21.172 INFO:teuthology.orchestra.run.smithi067.stdout:client4585 2023-10-14T10:36:21.173 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2023-10-14T10:36:21.173 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-14T10:36:21.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:36:21.191 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-14T10:36:21.278 INFO:teuthology.orchestra.run.smithi067.stdout:client4585 2023-10-14T10:36:21.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:36:21.279 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-14T10:36:21.327 INFO:teuthology.orchestra.run.smithi067.stdout:client4585 2023-10-14T10:36:21.328 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:36:21.328 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/37f9ab46-3082-4b71-8b24-08d39897b008.client4585/status') 2023-10-14T10:36:21.383 INFO:teuthology.orchestra.run.smithi067.stdout:instance: client.4585 (3)172.21.15.67:0/3036898314 2023-10-14T10:36:21.383 INFO:teuthology.orchestra.run.smithi067.stdout:blocklisted: false 2023-10-14T10:36:21.384 INFO:teuthology.orchestra.run.smithi067.stderr:0+1 records in 2023-10-14T10:36:21.384 INFO:teuthology.orchestra.run.smithi067.stderr:0+1 records out 2023-10-14T10:36:21.384 INFO:teuthology.orchestra.run.smithi067.stderr:70 bytes copied, 2.5462e-05 s, 2.7 MB/s 2023-10-14T10:36:21.385 INFO:tasks.kclient:client.2 config is {'syntax': 'v2'} 2023-10-14T10:36:21.385 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-10-14T10:36:21.386 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-10-14T10:36:21.386 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-14T10:36:21.511 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:21.509+0000 7f76b6f0c700 1 Processor -- start 2023-10-14T10:36:21.512 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:21.509+0000 7f76b6f0c700 1 -- start start 2023-10-14T10:36:21.515 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:21.510+0000 7f76b6f0c700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f76b0153ff0 0x7f76b01543d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:21.515 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:21.510+0000 7f76b6f0c700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f76b015dac0 0x7f76b00ce770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:21.516 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:21.510+0000 7f76b6f0c700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f76b00cecb0 0x7f76b00b0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:21.516 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:21.510+0000 7f76b6f0c700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76b015e9d0 con 0x7f76b00cecb0 2023-10-14T10:36:21.516 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:21.510+0000 7f76b6f0c700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76b00ce2e0 con 0x7f76b015dac0 2023-10-14T10:36:21.517 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:21.510+0000 7f76b6f0c700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76b0154910 con 0x7f76b0153ff0 2023-10-14T10:36:21.517 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:21.510+0000 7f76b54a9700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f76b00cecb0 0x7f76b00b0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:21.517 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:21.510+0000 7f76b54a9700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f76b00cecb0 0x7f76b00b0e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44590/0 (socket says 172.21.15.67:44590) 2023-10-14T10:36:21.518 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:21.510+0000 7f76b54a9700 1 -- 172.21.15.67:0/66801877 learned_addr learned my addr 172.21.15.67:0/66801877 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:21.518 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:21.510+0000 7f76affff700 1 --2- 172.21.15.67:0/66801877 >> v2:172.21.15.72:3300/0 conn(0x7f76b015dac0 0x7f76b00ce770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:21.518 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:21.510+0000 7f76b4ca8700 1 --2- 172.21.15.67:0/66801877 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f76b0153ff0 0x7f76b01543d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:21.518 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:21.510+0000 7f76b54a9700 1 -- 172.21.15.67:0/66801877 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f76b0153ff0 msgr2=0x7f76b01543d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:21.519 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:21.510+0000 7f76b54a9700 1 --2- 172.21.15.67:0/66801877 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f76b0153ff0 0x7f76b01543d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:21.519 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:21.510+0000 7f76b54a9700 1 -- 172.21.15.67:0/66801877 >> v2:172.21.15.72:3300/0 conn(0x7f76b015dac0 msgr2=0x7f76b00ce770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:21.519 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:21.510+0000 7f76b54a9700 1 --2- 172.21.15.67:0/66801877 >> v2:172.21.15.72:3300/0 conn(0x7f76b015dac0 0x7f76b00ce770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:21.519 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:21.510+0000 7f76b54a9700 1 -- 172.21.15.67:0/66801877 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76b015e410 con 0x7f76b00cecb0 2023-10-14T10:36:21.519 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:21.511+0000 7f76affff700 1 --2- 172.21.15.67:0/66801877 >> v2:172.21.15.72:3300/0 conn(0x7f76b015dac0 0x7f76b00ce770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-14T10:36:21.520 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:21.511+0000 7f76b54a9700 1 --2- 172.21.15.67:0/66801877 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76b00cecb0 0x7f76b00b0e70 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f76b006da80 tx=0x7f76a800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8f8ee5b19f0e65a server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:21.520 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:21.511+0000 7f76af7fe700 1 -- 172.21.15.67:0/66801877 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a800f070 con 0x7f76b00cecb0 2023-10-14T10:36:21.520 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:21.511+0000 7f76af7fe700 1 -- 172.21.15.67:0/66801877 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76a800a2e0 con 0x7f76b00cecb0 2023-10-14T10:36:21.520 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:21.511+0000 7f76af7fe700 1 -- 172.21.15.67:0/66801877 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a800ccc0 con 0x7f76b00cecb0 2023-10-14T10:36:21.520 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:21.511+0000 7f76b6f0c700 1 -- 172.21.15.67:0/66801877 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76b00cecb0 msgr2=0x7f76b00b0e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:21.521 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:21.511+0000 7f76b6f0c700 1 --2- 172.21.15.67:0/66801877 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76b00cecb0 0x7f76b00b0e70 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f76b006da80 tx=0x7f76a800c330 comp rx=0 tx=0).stop 2023-10-14T10:36:21.521 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:21.511+0000 7f76b6f0c700 1 -- 172.21.15.67:0/66801877 shutdown_connections 2023-10-14T10:36:21.521 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:21.511+0000 7f76b6f0c700 1 --2- 172.21.15.67:0/66801877 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76b00cecb0 0x7f76b00b0e70 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:21.521 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:21.511+0000 7f76b6f0c700 1 --2- 172.21.15.67:0/66801877 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f76b0153ff0 0x7f76b01543d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:21.522 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:21.511+0000 7f76b6f0c700 1 --2- 172.21.15.67:0/66801877 >> v2:172.21.15.72:3300/0 conn(0x7f76b015dac0 0x7f76b00ce770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:21.522 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:21.511+0000 7f76b6f0c700 1 -- 172.21.15.67:0/66801877 >> 172.21.15.67:0/66801877 conn(0x7f76b006afb0 msgr2=0x7f76b00cbe70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:21.522 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:21.511+0000 7f76b6f0c700 1 -- 172.21.15.67:0/66801877 shutdown_connections 2023-10-14T10:36:21.522 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:21.511+0000 7f76b6f0c700 1 -- 172.21.15.67:0/66801877 wait complete. 2023-10-14T10:36:21.522 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:21.513+0000 7f76b6f0c700 1 Processor -- start 2023-10-14T10:36:21.523 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:21.513+0000 7f76b6f0c700 1 -- start start 2023-10-14T10:36:21.526 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:21.514+0000 7f76b6f0c700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f76b00cecb0 0x7f76b00b3db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:21.527 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:21.514+0000 7f76b6f0c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76b0153ff0 0x7f76b0154a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:21.527 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:21.514+0000 7f76b6f0c700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f76b015dac0 0x7f76b0154f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:21.527 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:21.514+0000 7f76b6f0c700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76b00ae260 con 0x7f76b0153ff0 2023-10-14T10:36:21.527 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:21.514+0000 7f76b6f0c700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76b00c3680 con 0x7f76b015dac0 2023-10-14T10:36:21.527 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:21.514+0000 7f76b6f0c700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76b00c3960 con 0x7f76b00cecb0 2023-10-14T10:36:21.528 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:21.514+0000 7f76affff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76b0153ff0 0x7f76b0154a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:21.528 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:21.514+0000 7f76affff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76b0153ff0 0x7f76b0154a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44598/0 (socket says 172.21.15.67:44598) 2023-10-14T10:36:21.528 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:21.514+0000 7f76affff700 1 -- 172.21.15.67:0/606894640 learned_addr learned my addr 172.21.15.67:0/606894640 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:21.529 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:21.514+0000 7f76b4ca8700 1 --2- 172.21.15.67:0/606894640 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f76b00cecb0 0x7f76b00b3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:21.529 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:21.514+0000 7f76b54a9700 1 --2- 172.21.15.67:0/606894640 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f76b015dac0 0x7f76b0154f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:21.529 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:21.514+0000 7f76affff700 1 -- 172.21.15.67:0/606894640 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f76b00cecb0 msgr2=0x7f76b00b3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:21.529 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:21.514+0000 7f76affff700 1 --2- 172.21.15.67:0/606894640 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f76b00cecb0 0x7f76b00b3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:21.530 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:21.514+0000 7f76affff700 1 -- 172.21.15.67:0/606894640 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f76b015dac0 msgr2=0x7f76b0154f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:21.530 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:21.514+0000 7f76affff700 1 --2- 172.21.15.67:0/606894640 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f76b015dac0 0x7f76b0154f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:21.530 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:21.514+0000 7f76affff700 1 -- 172.21.15.67:0/606894640 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76a800ac40 con 0x7f76b0153ff0 2023-10-14T10:36:21.530 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:21.514+0000 7f76affff700 1 --2- 172.21.15.67:0/606894640 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76b0153ff0 0x7f76b0154a50 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f769c00c7f0 tx=0x7f769c00cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:21.531 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:21.514+0000 7f76ad7fa700 1 -- 172.21.15.67:0/606894640 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f769c011800 con 0x7f76b0153ff0 2023-10-14T10:36:21.531 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:21.514+0000 7f76ad7fa700 1 -- 172.21.15.67:0/606894640 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f769c011960 con 0x7f76b0153ff0 2023-10-14T10:36:21.531 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:21.514+0000 7f76b6f0c700 1 -- 172.21.15.67:0/606894640 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76b00c3c40 con 0x7f76b0153ff0 2023-10-14T10:36:21.531 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:21.514+0000 7f76ad7fa700 1 -- 172.21.15.67:0/606894640 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f769c011cf0 con 0x7f76b0153ff0 2023-10-14T10:36:21.532 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:21.515+0000 7f76b6f0c700 1 -- 172.21.15.67:0/606894640 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76b00c4200 con 0x7f76b0153ff0 2023-10-14T10:36:21.532 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:21.516+0000 7f76b6f0c700 1 -- 172.21.15.67:0/606894640 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76b00c4420 con 0x7f76b0153ff0 2023-10-14T10:36:21.532 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:21.516+0000 7f76ad7fa700 1 -- 172.21.15.67:0/606894640 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f769c018360 con 0x7f76b0153ff0 2023-10-14T10:36:21.532 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:21.516+0000 7f76ad7fa700 1 --2- 172.21.15.67:0/606894640 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f769406af00 0x7f769406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:21.532 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:21.517+0000 7f76ad7fa700 1 -- 172.21.15.67:0/606894640 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f769c089430 con 0x7f76b0153ff0 2023-10-14T10:36:21.533 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:21.517+0000 7f76b4ca8700 1 --2- 172.21.15.67:0/606894640 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f769406af00 0x7f769406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:21.533 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:21.517+0000 7f76b4ca8700 1 --2- 172.21.15.67:0/606894640 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f769406af00 0x7f769406d3c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f76a400ab70 tx=0x7f76a4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:21.533 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:21.521+0000 7f76ad7fa700 1 -- 172.21.15.67:0/606894640 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f769c0544e0 con 0x7f76b0153ff0 2023-10-14T10:36:21.773 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:21.771+0000 7f76b6f0c700 1 -- 172.21.15.67:0/606894640 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f76b004ebb0 con 0x7f76b0153ff0 2023-10-14T10:36:21.773 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:21.771+0000 7f76ad7fa700 1 -- 172.21.15.67:0/606894640 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f769c057b00 con 0x7f76b0153ff0 2023-10-14T10:36:21.774 INFO:teuthology.orchestra.run.smithi067.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-14T10:36:21.777 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:21.773+0000 7f76b6f0c700 1 -- 172.21.15.67:0/606894640 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f769406af00 msgr2=0x7f769406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:21.777 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:21.773+0000 7f76b6f0c700 1 --2- 172.21.15.67:0/606894640 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f769406af00 0x7f769406d3c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f76a400ab70 tx=0x7f76a4008040 comp rx=0 tx=0).stop 2023-10-14T10:36:21.777 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:21.773+0000 7f76b6f0c700 1 -- 172.21.15.67:0/606894640 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76b0153ff0 msgr2=0x7f76b0154a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:21.777 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:21.773+0000 7f76b6f0c700 1 --2- 172.21.15.67:0/606894640 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76b0153ff0 0x7f76b0154a50 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f769c00c7f0 tx=0x7f769c00cb00 comp rx=0 tx=0).stop 2023-10-14T10:36:21.777 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:21.773+0000 7f76b6f0c700 1 -- 172.21.15.67:0/606894640 shutdown_connections 2023-10-14T10:36:21.778 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:21.773+0000 7f76b6f0c700 1 --2- 172.21.15.67:0/606894640 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f769406af00 0x7f769406d3c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:21.778 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:21.773+0000 7f76b6f0c700 1 --2- 172.21.15.67:0/606894640 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f76b015dac0 0x7f76b0154f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:21.778 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:21.773+0000 7f76b6f0c700 1 --2- 172.21.15.67:0/606894640 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f76b00cecb0 0x7f76b00b3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:21.778 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:21.773+0000 7f76b6f0c700 1 --2- 172.21.15.67:0/606894640 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76b0153ff0 0x7f76b0154a50 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:21.778 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:21.773+0000 7f76b6f0c700 1 -- 172.21.15.67:0/606894640 >> 172.21.15.67:0/606894640 conn(0x7f76b006afb0 msgr2=0x7f76b00ca480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:21.779 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:21.773+0000 7f76b6f0c700 1 -- 172.21.15.67:0/606894640 shutdown_connections 2023-10-14T10:36:21.779 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:21.773+0000 7f76b6f0c700 1 -- 172.21.15.67:0/606894640 wait complete. 2023-10-14T10:36:21.787 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-14T10:36:21.788 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-14T10:36:21.788 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi067.front.sepia.ceph.com 2023-10-14T10:36:21.788 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-10-14T10:36:21.788 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-10-14T10:36:21.789 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-14T10:36:21.789 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-14T10:36:21.789 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-14T10:36:21.789 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-10-14T10:36:21.790 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:36:21.790 DEBUG:teuthology.orchestra.run.smithi067:> ip addr 2023-10-14T10:36:21.807 INFO:teuthology.orchestra.run.smithi067.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-14T10:36:21.808 INFO:teuthology.orchestra.run.smithi067.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-14T10:36:21.808 INFO:teuthology.orchestra.run.smithi067.stdout: inet 127.0.0.1/8 scope host lo 2023-10-14T10:36:21.808 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2023-10-14T10:36:21.808 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 ::1/128 scope host 2023-10-14T10:36:21.809 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2023-10-14T10:36:21.809 INFO:teuthology.orchestra.run.smithi067.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-14T10:36:21.809 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:a8:b4:7e brd ff:ff:ff:ff:ff:ff 2023-10-14T10:36:21.809 INFO:teuthology.orchestra.run.smithi067.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-14T10:36:21.809 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:a8:b4:7f brd ff:ff:ff:ff:ff:ff 2023-10-14T10:36:21.810 INFO:teuthology.orchestra.run.smithi067.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-14T10:36:21.810 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:b6:fc:0a brd ff:ff:ff:ff:ff:ff 2023-10-14T10:36:21.810 INFO:teuthology.orchestra.run.smithi067.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-14T10:36:21.810 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:b6:fc:0b brd ff:ff:ff:ff:ff:ff 2023-10-14T10:36:21.810 INFO:teuthology.orchestra.run.smithi067.stdout: inet 172.21.15.67/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-14T10:36:21.811 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft 42357sec preferred_lft 42357sec 2023-10-14T10:36:21.811 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 fe80::5cc4:6a4b:a14e:a6d7/64 scope link noprefixroute 2023-10-14T10:36:21.811 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2023-10-14T10:36:21.811 INFO:teuthology.orchestra.run.smithi067.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-14T10:36:21.811 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether d6:9f:d7:a3:00:d7 brd ff:ff:ff:ff:ff:ff 2023-10-14T10:36:21.811 INFO:teuthology.orchestra.run.smithi067.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-14T10:36:21.812 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2023-10-14T10:36:21.812 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 fe80::c86:b0ff:fe46:c30c/64 scope link 2023-10-14T10:36:21.812 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2023-10-14T10:36:21.812 INFO:teuthology.orchestra.run.smithi067.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-14T10:36:21.812 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether d6:9f:d7:a3:00:d7 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-14T10:36:21.813 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 fe80::d49f:d7ff:fea3:d7/64 scope link tentative 2023-10-14T10:36:21.813 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2023-10-14T10:36:21.813 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:36:21.813 DEBUG:teuthology.orchestra.run.smithi067:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-14T10:36:21.873 INFO:teuthology.orchestra.run.smithi067.stdout:1 2023-10-14T10:36:21.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:36:21.875 DEBUG:teuthology.orchestra.run.smithi067:> route 2023-10-14T10:36:21.933 INFO:teuthology.orchestra.run.smithi067.stdout:Kernel IP routing table 2023-10-14T10:36:21.934 INFO:teuthology.orchestra.run.smithi067.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-14T10:36:21.934 INFO:teuthology.orchestra.run.smithi067.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-14T10:36:21.934 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-14T10:36:21.935 INFO:teuthology.orchestra.run.smithi067.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-14T10:36:21.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:36:21.935 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec bash -c ' 2023-10-14T10:36:21.936 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2023-10-14T10:36:21.936 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-14T10:36:21.936 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-14T10:36:21.937 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-14T10:36:21.937 DEBUG:teuthology.orchestra.run.smithi067:> ') 2023-10-14T10:36:22.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:36:22.041 DEBUG:teuthology.orchestra.run.smithi067:> ip netns list 2023-10-14T10:36:22.058 INFO:teuthology.orchestra.run.smithi067.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-14T10:36:22.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:36:22.059 DEBUG:teuthology.orchestra.run.smithi067:> ip netns list-id 2023-10-14T10:36:22.116 INFO:teuthology.orchestra.run.smithi067.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-10-14T10:36:22.116 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:36:22.117 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec bash -c ' 2023-10-14T10:36:22.117 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2023-10-14T10:36:22.117 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-14T10:36:22.118 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-10-14T10:36:22.118 DEBUG:teuthology.orchestra.run.smithi067:> ') 2023-10-14T10:36:22.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:36:22.211 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-10-14T10:36:22.253 INFO:teuthology.orchestra.run.smithi067.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-14T10:36:22.253 INFO:teuthology.orchestra.run.smithi067.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-14T10:36:22.253 INFO:teuthology.orchestra.run.smithi067.stdout: inet 127.0.0.1/8 scope host lo 2023-10-14T10:36:22.254 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2023-10-14T10:36:22.254 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 ::1/128 scope host 2023-10-14T10:36:22.254 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2023-10-14T10:36:22.254 INFO:teuthology.orchestra.run.smithi067.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-14T10:36:22.254 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 5e:2d:99:10:30:15 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-14T10:36:22.255 INFO:teuthology.orchestra.run.smithi067.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-14T10:36:22.255 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2023-10-14T10:36:22.255 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 fe80::5c2d:99ff:fe10:3015/64 scope link tentative 2023-10-14T10:36:22.255 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2023-10-14T10:36:22.264 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:36:22.264 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-10-14T10:36:22.345 INFO:teuthology.orchestra.run.smithi067.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-14T10:36:22.345 INFO:teuthology.orchestra.run.smithi067.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-14T10:36:22.345 INFO:teuthology.orchestra.run.smithi067.stdout: inet 127.0.0.1/8 scope host lo 2023-10-14T10:36:22.345 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2023-10-14T10:36:22.346 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 ::1/128 scope host 2023-10-14T10:36:22.346 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2023-10-14T10:36:22.346 INFO:teuthology.orchestra.run.smithi067.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-14T10:36:22.346 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 5e:2d:99:10:30:15 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-14T10:36:22.346 INFO:teuthology.orchestra.run.smithi067.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-14T10:36:22.347 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2023-10-14T10:36:22.347 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 fe80::5c2d:99ff:fe10:3015/64 scope link tentative 2023-10-14T10:36:22.347 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2023-10-14T10:36:22.357 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-10-14T10:36:22.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:36:22.358 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec bash -c ' 2023-10-14T10:36:22.358 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2023-10-14T10:36:22.358 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-10-14T10:36:22.359 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-10-14T10:36:22.359 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-10-14T10:36:22.359 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-10-14T10:36:22.360 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-10-14T10:36:22.360 DEBUG:teuthology.orchestra.run.smithi067:> ') 2023-10-14T10:36:22.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:36:22.596 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec bash -c ' 2023-10-14T10:36:22.596 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2023-10-14T10:36:22.596 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link set brx.1 up 2023-10-14T10:36:22.597 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link set dev brx.1 master ceph-brx 2023-10-14T10:36:22.597 DEBUG:teuthology.orchestra.run.smithi067:> ') 2023-10-14T10:36:22.730 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-14T10:36:22.731 DEBUG:teuthology.orchestra.run.smithi067:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-10-14T10:36:22.748 INFO:teuthology.orchestra.run.smithi067.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-10-14T10:36:22.748 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-14T10:36:22.748 DEBUG:teuthology.orchestra.run.smithi067:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-10-14T10:36:22.804 INFO:tasks.cephfs.kernel_mount:mounting using device: 2@.cephfs=/ 2023-10-14T10:36:22.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:36:22.805 DEBUG:teuthology.orchestra.run.smithi067:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 2@.cephfs=/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-14T10:36:22.892 INFO:teuthology.orchestra.run.smithi067.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-14T10:36:22.892 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-14T10:36:22.919 INFO:teuthology.orchestra.run.smithi067.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-14T10:36:22.919 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-14T10:36:22.919 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: resolved to: "172.21.15.67:3300,172.21.15.72:3300,172.21.15.72:3301" 2023-10-14T10:36:22.920 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: trying mount with new device syntax: 2@37f9ab46-3082-4b71-8b24-08d39897b008.cephfs=/ 2023-10-14T10:36:22.920 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.67:3300/172.21.15.72:3300/172.21.15.72:3301" will pass to kernel 2023-10-14T10:36:22.920 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: v2 mount support check returned 2 2023-10-14T10:36:22.920 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-14T10:36:22.920 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-14T10:36:22.952 INFO:teuthology.orchestra.run.smithi067.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-14T10:36:22.952 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-14T10:36:22.952 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: resolved to: "172.21.15.67:3300,172.21.15.72:3300,172.21.15.72:3301" 2023-10-14T10:36:22.953 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: trying mount with new device syntax: 2@37f9ab46-3082-4b71-8b24-08d39897b008.cephfs=/ 2023-10-14T10:36:22.953 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.67:3300/172.21.15.72:3300/172.21.15.72:3301" will pass to kernel 2023-10-14T10:36:22.953 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: v2 mount support check returned 2 2023-10-14T10:36:22.953 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-14T10:36:22.953 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-14T10:36:22.954 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: resolved to: "172.21.15.67:6789,172.21.15.72:6789,172.21.15.72:6790" 2023-10-14T10:36:22.954 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.67:6789,172.21.15.72:6789,172.21.15.72:6790:/ 2023-10-14T10:36:22.954 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,key=2,mds_namespace=cephfs,fsid=37f9ab46-3082-4b71-8b24-08d39897b008" will pass to kernel 2023-10-14T10:36:22.973 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-14T10:36:22.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:36:22.974 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-10-14T10:36:23.007 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:36:23.007 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-14T10:36:23.080 INFO:teuthology.orchestra.run.smithi067.stdout:client4677 2023-10-14T10:36:23.080 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2023-10-14T10:36:23.080 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-14T10:36:23.137 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:36:23.138 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-14T10:36:23.211 INFO:teuthology.orchestra.run.smithi067.stdout:client4677 2023-10-14T10:36:23.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:36:23.212 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-14T10:36:23.284 INFO:teuthology.orchestra.run.smithi067.stdout:client4677 2023-10-14T10:36:23.284 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:36:23.284 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/37f9ab46-3082-4b71-8b24-08d39897b008.client4677/status') 2023-10-14T10:36:23.365 INFO:teuthology.orchestra.run.smithi067.stdout:instance: client.4677 (3)192.168.0.2:0/2449858215 2023-10-14T10:36:23.365 INFO:teuthology.orchestra.run.smithi067.stdout:blocklisted: false 2023-10-14T10:36:23.366 INFO:teuthology.orchestra.run.smithi067.stderr:0+1 records in 2023-10-14T10:36:23.366 INFO:teuthology.orchestra.run.smithi067.stderr:0+1 records out 2023-10-14T10:36:23.366 INFO:teuthology.orchestra.run.smithi067.stderr:69 bytes copied, 3.5772e-05 s, 1.9 MB/s 2023-10-14T10:36:23.367 INFO:tasks.kclient:client.1 config is {'syntax': 'v2'} 2023-10-14T10:36:23.367 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-10-14T10:36:23.367 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-14T10:36:23.367 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-14T10:36:23.531 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:23.529+0000 7fc145e04700 1 Processor -- start 2023-10-14T10:36:23.531 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:23.529+0000 7fc145e04700 1 -- start start 2023-10-14T10:36:23.534 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:23.530+0000 7fc145e04700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fc14015e2f0 0x7fc1400cede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:23.534 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:23.530+0000 7fc145e04700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1400b2d60 con 0x7fc1400cd950 2023-10-14T10:36:23.535 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:23.530+0000 7fc145e04700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1400cdd30 con 0x7fc14006a840 2023-10-14T10:36:23.535 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:23.530+0000 7fc145e04700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1401619a0 con 0x7fc14015e2f0 2023-10-14T10:36:23.535 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:23.530+0000 7fc13f7fe700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fc1400cd950 0x7fc14015f570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:52378/0 (socket says 172.21.15.67:52378) 2023-10-14T10:36:23.535 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:23.530+0000 7fc13f7fe700 1 -- 172.21.15.67:0/4168902754 learned_addr learned my addr 172.21.15.67:0/4168902754 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:36:23.536 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:23.530+0000 7fc13effd700 1 --2- 172.21.15.67:0/4168902754 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fc14015e2f0 0x7fc1400cede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:23.536 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:23.530+0000 7fc13e7fc700 1 -- 172.21.15.67:0/4168902754 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 174165193 0 0) 0x7fc1400b2d60 con 0x7fc1400cd950 2023-10-14T10:36:23.536 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:23.530+0000 7fc13e7fc700 1 -- 172.21.15.67:0/4168902754 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc12c0035b0 con 0x7fc1400cd950 2023-10-14T10:36:23.536 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:23.530+0000 7fc13e7fc700 1 -- 172.21.15.67:0/4168902754 <== mon.1 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2437984543 0 0) 0x7fc1400cdd30 con 0x7fc14006a840 2023-10-14T10:36:23.537 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:23.530+0000 7fc13e7fc700 1 -- 172.21.15.67:0/4168902754 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1400b2d60 con 0x7fc14006a840 2023-10-14T10:36:23.537 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:23.531+0000 7fc13effd700 1 -- 172.21.15.67:0/4168902754 >> v1:172.21.15.72:6789/0 conn(0x7fc14006a840 legacy=0x7fc140161df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:23.537 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:23.531+0000 7fc13effd700 1 -- 172.21.15.67:0/4168902754 >> v1:172.21.15.67:6789/0 conn(0x7fc1400cd950 legacy=0x7fc14015f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:23.537 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:23.531+0000 7fc13e7fc700 1 -- 172.21.15.67:0/4168902754 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3101573107 0 0) 0x7fc12c0035b0 con 0x7fc1400cd950 2023-10-14T10:36:23.538 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:23.531+0000 7fc13effd700 1 -- 172.21.15.67:0/4168902754 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc14015e920 con 0x7fc14015e2f0 2023-10-14T10:36:23.538 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:23.531+0000 7fc13effd700 1 --2- 172.21.15.67:0/4168902754 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fc14015e2f0 0x7fc1400cede0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc130009850 tx=0x7fc130009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b97a3950a8abcfdc server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:23.538 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:23.531+0000 7fc13e7fc700 1 -- 172.21.15.67:0/4168902754 <== mon.2 v2:172.21.15.72:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc128001670 con 0x7fc14015e2f0 2023-10-14T10:36:23.538 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:23.531+0000 7fc13e7fc700 1 -- 172.21.15.67:0/4168902754 <== mon.2 v2:172.21.15.72:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc130005c30 con 0x7fc14015e2f0 2023-10-14T10:36:23.539 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:23.531+0000 7fc13e7fc700 1 -- 172.21.15.67:0/4168902754 <== mon.2 v2:172.21.15.72:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc130004030 con 0x7fc14015e2f0 2023-10-14T10:36:23.539 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:23.531+0000 7fc145e04700 1 -- 172.21.15.67:0/4168902754 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fc14015e2f0 msgr2=0x7fc1400cede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:23.539 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:23.531+0000 7fc145e04700 1 --2- 172.21.15.67:0/4168902754 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fc14015e2f0 0x7fc1400cede0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc130009850 tx=0x7fc130009b60 comp rx=0 tx=0).stop 2023-10-14T10:36:23.539 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:23.532+0000 7fc145e04700 1 -- 172.21.15.67:0/4168902754 shutdown_connections 2023-10-14T10:36:23.540 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:23.532+0000 7fc145e04700 1 --2- 172.21.15.67:0/4168902754 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fc14015e2f0 0x7fc1400cede0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:23.540 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:23.532+0000 7fc145e04700 1 -- 172.21.15.67:0/4168902754 >> 172.21.15.67:0/4168902754 conn(0x7fc14006dae0 msgr2=0x7fc14006def0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:23.540 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:23.532+0000 7fc145e04700 1 -- 172.21.15.67:0/4168902754 shutdown_connections 2023-10-14T10:36:23.540 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:23.532+0000 7fc145e04700 1 -- 172.21.15.67:0/4168902754 wait complete. 2023-10-14T10:36:23.543 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:23.533+0000 7fc145e04700 1 Processor -- start 2023-10-14T10:36:23.544 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:23.533+0000 7fc145e04700 1 -- start start 2023-10-14T10:36:23.544 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:23.534+0000 7fc145e04700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc14006a840 0x7fc14014b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:23.544 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:23.534+0000 7fc145e04700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc1400cd950 0x7fc1400ace20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:23.544 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:23.534+0000 7fc145e04700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fc14015e2f0 0x7fc1401f6e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:23.544 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:23.534+0000 7fc145e04700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1400aeb50 con 0x7fc14006a840 2023-10-14T10:36:23.545 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:23.534+0000 7fc145e04700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1401f76b0 con 0x7fc1400cd950 2023-10-14T10:36:23.545 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:23.534+0000 7fc145e04700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1401f7990 con 0x7fc14015e2f0 2023-10-14T10:36:23.545 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:23.534+0000 7fc13f7fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc14006a840 0x7fc14014b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:23.545 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:23.534+0000 7fc13f7fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc14006a840 0x7fc14014b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44614/0 (socket says 172.21.15.67:44614) 2023-10-14T10:36:23.545 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:23.534+0000 7fc13f7fe700 1 -- 172.21.15.67:0/2843854200 learned_addr learned my addr 172.21.15.67:0/2843854200 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:23.546 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:23.534+0000 7fc13effd700 1 --2- 172.21.15.67:0/2843854200 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc1400cd950 0x7fc1400ace20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:23.546 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:23.534+0000 7fc13ffff700 1 --2- 172.21.15.67:0/2843854200 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fc14015e2f0 0x7fc1401f6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:23.546 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:23.534+0000 7fc13f7fe700 1 -- 172.21.15.67:0/2843854200 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fc14015e2f0 msgr2=0x7fc1401f6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:23.546 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:23.534+0000 7fc13f7fe700 1 --2- 172.21.15.67:0/2843854200 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fc14015e2f0 0x7fc1401f6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:23.547 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:23.534+0000 7fc13f7fe700 1 -- 172.21.15.67:0/2843854200 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc1400cd950 msgr2=0x7fc1400ace20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:23.547 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:23.534+0000 7fc13f7fe700 1 --2- 172.21.15.67:0/2843854200 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc1400cd950 0x7fc1400ace20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:23.547 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:23.534+0000 7fc13f7fe700 1 -- 172.21.15.67:0/2843854200 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc130009430 con 0x7fc14006a840 2023-10-14T10:36:23.547 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:23.534+0000 7fc13f7fe700 1 --2- 172.21.15.67:0/2843854200 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc14006a840 0x7fc14014b3f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc12800c750 tx=0x7fc12800ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:23.547 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:23.534+0000 7fc137fff700 1 -- 172.21.15.67:0/2843854200 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc128004d40 con 0x7fc14006a840 2023-10-14T10:36:23.548 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:23.534+0000 7fc137fff700 1 -- 172.21.15.67:0/2843854200 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1280070d0 con 0x7fc14006a840 2023-10-14T10:36:23.548 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:23.534+0000 7fc145e04700 1 -- 172.21.15.67:0/2843854200 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1400ae7e0 con 0x7fc14006a840 2023-10-14T10:36:23.548 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:23.534+0000 7fc137fff700 1 -- 172.21.15.67:0/2843854200 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc128004ea0 con 0x7fc14006a840 2023-10-14T10:36:23.548 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:23.535+0000 7fc145e04700 1 -- 172.21.15.67:0/2843854200 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1401f7c70 con 0x7fc14006a840 2023-10-14T10:36:23.548 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:23.535+0000 7fc145e04700 1 -- 172.21.15.67:0/2843854200 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1400ae640 con 0x7fc14006a840 2023-10-14T10:36:23.549 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:23.536+0000 7fc137fff700 1 -- 172.21.15.67:0/2843854200 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fc128007460 con 0x7fc14006a840 2023-10-14T10:36:23.549 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:23.536+0000 7fc137fff700 1 --2- 172.21.15.67:0/2843854200 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fc12006af00 0x7fc12006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:23.550 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:23.536+0000 7fc137fff700 1 -- 172.21.15.67:0/2843854200 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc1280893a0 con 0x7fc14006a840 2023-10-14T10:36:23.550 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:23.537+0000 7fc13effd700 1 --2- 172.21.15.67:0/2843854200 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fc12006af00 0x7fc12006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:23.550 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:23.537+0000 7fc13effd700 1 --2- 172.21.15.67:0/2843854200 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fc12006af00 0x7fc12006d3c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc1300097a0 tx=0x7fc130008070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:23.550 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:23.541+0000 7fc137fff700 1 -- 172.21.15.67:0/2843854200 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc1400ae640 con 0x7fc14006a840 2023-10-14T10:36:23.790 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:23.788+0000 7fc145e04700 1 -- 172.21.15.67:0/2843854200 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fc1400ae640 con 0x7fc14006a840 2023-10-14T10:36:23.791 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:23.788+0000 7fc137fff700 1 -- 172.21.15.67:0/2843854200 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7fc1400ae640 con 0x7fc14006a840 2023-10-14T10:36:23.791 INFO:teuthology.orchestra.run.smithi067.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-14T10:36:23.793 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:23.791+0000 7fc145e04700 1 -- 172.21.15.67:0/2843854200 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fc12006af00 msgr2=0x7fc12006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:23.793 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:23.791+0000 7fc145e04700 1 --2- 172.21.15.67:0/2843854200 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fc12006af00 0x7fc12006d3c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc1300097a0 tx=0x7fc130008070 comp rx=0 tx=0).stop 2023-10-14T10:36:23.793 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:23.791+0000 7fc145e04700 1 -- 172.21.15.67:0/2843854200 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc14006a840 msgr2=0x7fc14014b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:23.794 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:23.791+0000 7fc145e04700 1 --2- 172.21.15.67:0/2843854200 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc14006a840 0x7fc14014b3f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc12800c750 tx=0x7fc12800ca60 comp rx=0 tx=0).stop 2023-10-14T10:36:23.794 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:23.791+0000 7fc145e04700 1 -- 172.21.15.67:0/2843854200 shutdown_connections 2023-10-14T10:36:23.794 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:23.791+0000 7fc145e04700 1 --2- 172.21.15.67:0/2843854200 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fc12006af00 0x7fc12006d3c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:23.794 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:23.791+0000 7fc145e04700 1 --2- 172.21.15.67:0/2843854200 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fc14015e2f0 0x7fc1401f6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:23.794 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:23.791+0000 7fc145e04700 1 --2- 172.21.15.67:0/2843854200 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc14006a840 0x7fc14014b3f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:23.795 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:23.791+0000 7fc145e04700 1 --2- 172.21.15.67:0/2843854200 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc1400cd950 0x7fc1400ace20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:23.795 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:23.791+0000 7fc145e04700 1 -- 172.21.15.67:0/2843854200 >> 172.21.15.67:0/2843854200 conn(0x7fc14006dae0 msgr2=0x7fc14006dfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:23.795 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:23.791+0000 7fc145e04700 1 -- 172.21.15.67:0/2843854200 shutdown_connections 2023-10-14T10:36:23.795 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:23.791+0000 7fc145e04700 1 -- 172.21.15.67:0/2843854200 wait complete. 2023-10-14T10:36:23.805 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-14T10:36:23.805 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-14T10:36:23.806 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi072.front.sepia.ceph.com 2023-10-14T10:36:23.806 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-14T10:36:23.806 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-14T10:36:23.807 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-14T10:36:23.807 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-14T10:36:23.807 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-14T10:36:23.807 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-14T10:36:23.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:36:23.808 DEBUG:teuthology.orchestra.run.smithi072:> ip addr 2023-10-14T10:36:23.829 INFO:teuthology.orchestra.run.smithi072.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-14T10:36:23.829 INFO:teuthology.orchestra.run.smithi072.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-14T10:36:23.830 INFO:teuthology.orchestra.run.smithi072.stdout: inet 127.0.0.1/8 scope host lo 2023-10-14T10:36:23.830 INFO:teuthology.orchestra.run.smithi072.stdout: valid_lft forever preferred_lft forever 2023-10-14T10:36:23.830 INFO:teuthology.orchestra.run.smithi072.stdout: inet6 ::1/128 scope host 2023-10-14T10:36:23.830 INFO:teuthology.orchestra.run.smithi072.stdout: valid_lft forever preferred_lft forever 2023-10-14T10:36:23.830 INFO:teuthology.orchestra.run.smithi072.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-14T10:36:23.831 INFO:teuthology.orchestra.run.smithi072.stdout: link/ether 0c:c4:7a:d8:da:f4 brd ff:ff:ff:ff:ff:ff 2023-10-14T10:36:23.831 INFO:teuthology.orchestra.run.smithi072.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-14T10:36:23.831 INFO:teuthology.orchestra.run.smithi072.stdout: link/ether 0c:c4:7a:d8:da:f5 brd ff:ff:ff:ff:ff:ff 2023-10-14T10:36:23.831 INFO:teuthology.orchestra.run.smithi072.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-14T10:36:23.832 INFO:teuthology.orchestra.run.smithi072.stdout: link/ether 0c:c4:7a:8f:b7:64 brd ff:ff:ff:ff:ff:ff 2023-10-14T10:36:23.832 INFO:teuthology.orchestra.run.smithi072.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-14T10:36:23.832 INFO:teuthology.orchestra.run.smithi072.stdout: link/ether 0c:c4:7a:8f:b7:65 brd ff:ff:ff:ff:ff:ff 2023-10-14T10:36:23.832 INFO:teuthology.orchestra.run.smithi072.stdout: inet 172.21.15.72/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-14T10:36:23.833 INFO:teuthology.orchestra.run.smithi072.stdout: valid_lft 42385sec preferred_lft 42385sec 2023-10-14T10:36:23.833 INFO:teuthology.orchestra.run.smithi072.stdout: inet6 fe80::b3b8:cd9b:8a75:edd6/64 scope link noprefixroute 2023-10-14T10:36:23.833 INFO:teuthology.orchestra.run.smithi072.stdout: valid_lft forever preferred_lft forever 2023-10-14T10:36:23.833 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-14T10:36:23.834 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:36:23.834 DEBUG:teuthology.orchestra.run.smithi072:> (cd / && exec bash -c ' 2023-10-14T10:36:23.834 DEBUG:teuthology.orchestra.run.smithi072:> set -e 2023-10-14T10:36:23.835 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip link add name ceph-brx type bridge 2023-10-14T10:36:23.835 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip addr flush dev ceph-brx 2023-10-14T10:36:23.835 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip link set ceph-brx up 2023-10-14T10:36:23.837 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-14T10:36:23.838 DEBUG:teuthology.orchestra.run.smithi072:> ') 2023-10-14T10:36:24.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:36:24.003 DEBUG:teuthology.orchestra.run.smithi072:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-14T10:36:24.031 INFO:teuthology.orchestra.run.smithi072.stdout:1 2023-10-14T10:36:24.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:36:24.033 DEBUG:teuthology.orchestra.run.smithi072:> route 2023-10-14T10:36:24.109 INFO:teuthology.orchestra.run.smithi072.stdout:Kernel IP routing table 2023-10-14T10:36:24.109 INFO:teuthology.orchestra.run.smithi072.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-14T10:36:24.109 INFO:teuthology.orchestra.run.smithi072.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-14T10:36:24.109 INFO:teuthology.orchestra.run.smithi072.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-14T10:36:24.110 INFO:teuthology.orchestra.run.smithi072.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-14T10:36:24.110 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:36:24.110 DEBUG:teuthology.orchestra.run.smithi072:> (cd / && exec bash -c ' 2023-10-14T10:36:24.111 DEBUG:teuthology.orchestra.run.smithi072:> set -e 2023-10-14T10:36:24.111 DEBUG:teuthology.orchestra.run.smithi072:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-14T10:36:24.111 DEBUG:teuthology.orchestra.run.smithi072:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-14T10:36:24.112 DEBUG:teuthology.orchestra.run.smithi072:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-14T10:36:24.112 DEBUG:teuthology.orchestra.run.smithi072:> ') 2023-10-14T10:36:24.394 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:36:24.395 DEBUG:teuthology.orchestra.run.smithi072:> ip netns list 2023-10-14T10:36:24.412 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:36:24.412 DEBUG:teuthology.orchestra.run.smithi072:> ip netns list-id 2023-10-14T10:36:24.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:36:24.469 DEBUG:teuthology.orchestra.run.smithi072:> (cd / && exec bash -c ' 2023-10-14T10:36:24.469 DEBUG:teuthology.orchestra.run.smithi072:> set -e 2023-10-14T10:36:24.469 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-14T10:36:24.470 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-10-14T10:36:24.470 DEBUG:teuthology.orchestra.run.smithi072:> ') 2023-10-14T10:36:24.563 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-10-14T10:36:24.564 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:36:24.564 DEBUG:teuthology.orchestra.run.smithi072:> (cd / && exec bash -c ' 2023-10-14T10:36:24.564 DEBUG:teuthology.orchestra.run.smithi072:> set -e 2023-10-14T10:36:24.565 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-10-14T10:36:24.565 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-14T10:36:24.565 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-10-14T10:36:24.565 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-10-14T10:36:24.566 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-10-14T10:36:24.566 DEBUG:teuthology.orchestra.run.smithi072:> ') 2023-10-14T10:36:24.787 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:36:24.788 DEBUG:teuthology.orchestra.run.smithi072:> (cd / && exec bash -c ' 2023-10-14T10:36:24.788 DEBUG:teuthology.orchestra.run.smithi072:> set -e 2023-10-14T10:36:24.789 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip link set brx.0 up 2023-10-14T10:36:24.789 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip link set dev brx.0 master ceph-brx 2023-10-14T10:36:24.789 DEBUG:teuthology.orchestra.run.smithi072:> ') 2023-10-14T10:36:24.891 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-14T10:36:24.891 DEBUG:teuthology.orchestra.run.smithi072:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-14T10:36:24.910 INFO:teuthology.orchestra.run.smithi072.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-14T10:36:24.911 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-14T10:36:24.911 DEBUG:teuthology.orchestra.run.smithi072:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-14T10:36:24.968 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-14T10:36:24.968 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:36:24.968 DEBUG:teuthology.orchestra.run.smithi072:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-14T10:36:25.062 INFO:teuthology.orchestra.run.smithi072.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-14T10:36:25.062 INFO:teuthology.orchestra.run.smithi072.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-14T10:36:25.181 INFO:teuthology.orchestra.run.smithi072.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-14T10:36:25.181 INFO:teuthology.orchestra.run.smithi072.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-14T10:36:25.182 INFO:teuthology.orchestra.run.smithi072.stdout:mount.ceph: resolved to: "172.21.15.67:3300,172.21.15.72:3300,172.21.15.72:3301" 2023-10-14T10:36:25.182 INFO:teuthology.orchestra.run.smithi072.stdout:mount.ceph: trying mount with new device syntax: 1@37f9ab46-3082-4b71-8b24-08d39897b008.cephfs=/ 2023-10-14T10:36:25.182 INFO:teuthology.orchestra.run.smithi072.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.67:3300/172.21.15.72:3300/172.21.15.72:3301" will pass to kernel 2023-10-14T10:36:25.182 INFO:teuthology.orchestra.run.smithi072.stdout:mount.ceph: v2 mount support check returned 2 2023-10-14T10:36:25.183 INFO:teuthology.orchestra.run.smithi072.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-14T10:36:25.183 INFO:teuthology.orchestra.run.smithi072.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-14T10:36:25.216 INFO:teuthology.orchestra.run.smithi072.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-14T10:36:25.216 INFO:teuthology.orchestra.run.smithi072.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-14T10:36:25.216 INFO:teuthology.orchestra.run.smithi072.stdout:mount.ceph: resolved to: "172.21.15.67:3300,172.21.15.72:3300,172.21.15.72:3301" 2023-10-14T10:36:25.216 INFO:teuthology.orchestra.run.smithi072.stdout:mount.ceph: trying mount with new device syntax: 1@37f9ab46-3082-4b71-8b24-08d39897b008.cephfs=/ 2023-10-14T10:36:25.217 INFO:teuthology.orchestra.run.smithi072.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.67:3300/172.21.15.72:3300/172.21.15.72:3301" will pass to kernel 2023-10-14T10:36:25.217 INFO:teuthology.orchestra.run.smithi072.stdout:mount.ceph: v2 mount support check returned 2 2023-10-14T10:36:25.217 INFO:teuthology.orchestra.run.smithi072.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-14T10:36:25.217 INFO:teuthology.orchestra.run.smithi072.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-14T10:36:25.217 INFO:teuthology.orchestra.run.smithi072.stdout:mount.ceph: resolved to: "172.21.15.67:6789,172.21.15.72:6789,172.21.15.72:6790" 2023-10-14T10:36:25.218 INFO:teuthology.orchestra.run.smithi072.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.67:6789,172.21.15.72:6789,172.21.15.72:6790:/ 2023-10-14T10:36:25.218 INFO:teuthology.orchestra.run.smithi072.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=37f9ab46-3082-4b71-8b24-08d39897b008" will pass to kernel 2023-10-14T10:36:25.227 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-14T10:36:25.227 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:36:25.228 DEBUG:teuthology.orchestra.run.smithi072:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-14T10:36:25.259 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:36:25.260 DEBUG:teuthology.orchestra.run.smithi072:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-14T10:36:25.332 INFO:teuthology.orchestra.run.smithi072.stdout:client4597 2023-10-14T10:36:25.332 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-10-14T10:36:25.333 DEBUG:teuthology.orchestra.run.smithi072:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-14T10:36:25.389 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:36:25.389 DEBUG:teuthology.orchestra.run.smithi072:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-14T10:36:25.461 INFO:teuthology.orchestra.run.smithi072.stdout:client4597 2023-10-14T10:36:25.462 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:36:25.462 DEBUG:teuthology.orchestra.run.smithi072:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-14T10:36:25.534 INFO:teuthology.orchestra.run.smithi072.stdout:client4597 2023-10-14T10:36:25.534 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:36:25.535 DEBUG:teuthology.orchestra.run.smithi072:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/37f9ab46-3082-4b71-8b24-08d39897b008.client4597/status') 2023-10-14T10:36:25.613 INFO:teuthology.orchestra.run.smithi072.stdout:instance: client.4597 (3)192.168.0.1:0/3326855812 2023-10-14T10:36:25.614 INFO:teuthology.orchestra.run.smithi072.stdout:blocklisted: false 2023-10-14T10:36:25.615 INFO:teuthology.orchestra.run.smithi072.stderr:0+1 records in 2023-10-14T10:36:25.615 INFO:teuthology.orchestra.run.smithi072.stderr:0+1 records out 2023-10-14T10:36:25.615 INFO:teuthology.orchestra.run.smithi072.stderr:69 bytes copied, 3.7631e-05 s, 1.8 MB/s 2023-10-14T10:36:25.616 INFO:tasks.kclient:client.3 config is {'syntax': 'v2'} 2023-10-14T10:36:25.616 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-10-14T10:36:25.616 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-10-14T10:36:25.617 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-14T10:36:25.755 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:25.753+0000 7f579dab6700 1 Processor -- start 2023-10-14T10:36:25.756 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:25.753+0000 7f579dab6700 1 -- start start 2023-10-14T10:36:25.758 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:25.754+0000 7f579dab6700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f579806af50 0x7f57980cede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:25.758 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:25.754+0000 7f579dab6700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57980b3530 con 0x7f57980b28b0 2023-10-14T10:36:25.758 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:25.754+0000 7f579dab6700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57980cd180 con 0x7f579815f130 2023-10-14T10:36:25.759 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:25.754+0000 7f579dab6700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57980cde00 con 0x7f579806af50 2023-10-14T10:36:25.759 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:25.754+0000 7f5796ffd700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f579806af50 0x7f57980cede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:25.759 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:25.754+0000 7f5797fff700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f57980b28b0 0x7f57980b3960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:52390/0 (socket says 172.21.15.67:52390) 2023-10-14T10:36:25.759 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:25.754+0000 7f5797fff700 1 -- 172.21.15.67:0/286954675 learned_addr learned my addr 172.21.15.67:0/286954675 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:36:25.759 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:25.755+0000 7f5796ffd700 1 -- 172.21.15.67:0/286954675 >> v1:172.21.15.72:6789/0 conn(0x7f579815f130 legacy=0x7f579806a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:25.760 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:25.755+0000 7f5796ffd700 1 -- 172.21.15.67:0/286954675 >> v1:172.21.15.67:6789/0 conn(0x7f57980b28b0 legacy=0x7f57980b3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:25.760 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:25.755+0000 7f57967fc700 1 -- 172.21.15.67:0/286954675 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3778129079 0 0) 0x7f57980b3530 con 0x7f57980b28b0 2023-10-14T10:36:25.760 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:25.755+0000 7f5796ffd700 1 -- 172.21.15.67:0/286954675 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57980cd460 con 0x7f579806af50 2023-10-14T10:36:25.760 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:25.755+0000 7f5796ffd700 1 --2- 172.21.15.67:0/286954675 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f579806af50 0x7f57980cede0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5780009820 tx=0x7f5780009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3bb57e6b22617dc server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:25.760 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:25.755+0000 7f57967fc700 1 -- 172.21.15.67:0/286954675 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5780005e30 con 0x7f579806af50 2023-10-14T10:36:25.761 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:25.755+0000 7f57967fc700 1 -- 172.21.15.67:0/286954675 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5780006030 con 0x7f579806af50 2023-10-14T10:36:25.761 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:25.755+0000 7f57967fc700 1 -- 172.21.15.67:0/286954675 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57800063e0 con 0x7f579806af50 2023-10-14T10:36:25.761 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:25.756+0000 7f579dab6700 1 -- 172.21.15.67:0/286954675 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f579806af50 msgr2=0x7f57980cede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:25.761 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:25.756+0000 7f579dab6700 1 --2- 172.21.15.67:0/286954675 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f579806af50 0x7f57980cede0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5780009820 tx=0x7f5780009b30 comp rx=0 tx=0).stop 2023-10-14T10:36:25.761 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:25.756+0000 7f579dab6700 1 -- 172.21.15.67:0/286954675 shutdown_connections 2023-10-14T10:36:25.762 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:25.756+0000 7f579dab6700 1 --2- 172.21.15.67:0/286954675 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f579806af50 0x7f57980cede0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:25.762 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:25.756+0000 7f579dab6700 1 -- 172.21.15.67:0/286954675 >> 172.21.15.67:0/286954675 conn(0x7f579806dae0 msgr2=0x7f579806def0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:25.762 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:25.756+0000 7f579dab6700 1 -- 172.21.15.67:0/286954675 shutdown_connections 2023-10-14T10:36:25.762 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:25.756+0000 7f579dab6700 1 -- 172.21.15.67:0/286954675 wait complete. 2023-10-14T10:36:25.765 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:25.757+0000 7f579dab6700 1 Processor -- start 2023-10-14T10:36:25.765 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:25.757+0000 7f579dab6700 1 -- start start 2023-10-14T10:36:25.765 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:25.758+0000 7f579dab6700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f579806af50 0x7f5798157020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:25.766 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:25.758+0000 7f579dab6700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f57980b28b0 0x7f57980c4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:25.766 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:25.758+0000 7f579dab6700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f579815f130 0x7f57980b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:25.766 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:25.758+0000 7f579dab6700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57980c4700 con 0x7f579815f130 2023-10-14T10:36:25.766 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:25.758+0000 7f579dab6700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57980c73d0 con 0x7f57980b28b0 2023-10-14T10:36:25.767 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:25.758+0000 7f579dab6700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57980b6020 con 0x7f579806af50 2023-10-14T10:36:25.767 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:25.758+0000 7f5797fff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f579815f130 0x7f57980b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:25.767 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:25.758+0000 7f5797fff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f579815f130 0x7f57980b5ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44630/0 (socket says 172.21.15.67:44630) 2023-10-14T10:36:25.767 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:25.758+0000 7f5797fff700 1 -- 172.21.15.67:0/1939479950 learned_addr learned my addr 172.21.15.67:0/1939479950 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:25.767 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:25.758+0000 7f5796ffd700 1 --2- 172.21.15.67:0/1939479950 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f57980b28b0 0x7f57980c4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:25.768 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:25.758+0000 7f57977fe700 1 --2- 172.21.15.67:0/1939479950 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f579806af50 0x7f5798157020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:25.768 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:25.758+0000 7f5797fff700 1 -- 172.21.15.67:0/1939479950 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f579806af50 msgr2=0x7f5798157020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:25.768 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:25.758+0000 7f5797fff700 1 --2- 172.21.15.67:0/1939479950 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f579806af50 0x7f5798157020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:25.768 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:25.758+0000 7f5797fff700 1 -- 172.21.15.67:0/1939479950 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f57980b28b0 msgr2=0x7f57980c4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:25.768 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:25.758+0000 7f5797fff700 1 --2- 172.21.15.67:0/1939479950 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f57980b28b0 0x7f57980c4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:25.769 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:25.758+0000 7f5797fff700 1 -- 172.21.15.67:0/1939479950 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5780009430 con 0x7f579815f130 2023-10-14T10:36:25.769 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:25.759+0000 7f5797fff700 1 --2- 172.21.15.67:0/1939479950 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f579815f130 0x7f57980b5ae0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5790002230 tx=0x7f579000cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:25.769 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:25.759+0000 7f578ffff700 1 -- 172.21.15.67:0/1939479950 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f579000aa60 con 0x7f579815f130 2023-10-14T10:36:25.769 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:25.759+0000 7f578ffff700 1 -- 172.21.15.67:0/1939479950 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f579000abe0 con 0x7f579815f130 2023-10-14T10:36:25.769 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:25.759+0000 7f579dab6700 1 -- 172.21.15.67:0/1939479950 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57980c81e0 con 0x7f579815f130 2023-10-14T10:36:25.770 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:25.759+0000 7f578ffff700 1 -- 172.21.15.67:0/1939479950 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57900070d0 con 0x7f579815f130 2023-10-14T10:36:25.770 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:25.759+0000 7f579dab6700 1 -- 172.21.15.67:0/1939479950 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57980c70f0 con 0x7f579815f130 2023-10-14T10:36:25.770 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:25.760+0000 7f579dab6700 1 -- 172.21.15.67:0/1939479950 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57980c8370 con 0x7f579815f130 2023-10-14T10:36:25.770 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:25.760+0000 7f578ffff700 1 -- 172.21.15.67:0/1939479950 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f5790007230 con 0x7f579815f130 2023-10-14T10:36:25.770 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:25.761+0000 7f578ffff700 1 --2- 172.21.15.67:0/1939479950 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f577806af00 0x7f577806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:25.771 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:25.761+0000 7f578ffff700 1 -- 172.21.15.67:0/1939479950 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5790089330 con 0x7f579815f130 2023-10-14T10:36:25.771 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:25.761+0000 7f57977fe700 1 --2- 172.21.15.67:0/1939479950 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f577806af00 0x7f577806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:25.771 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:25.761+0000 7f57977fe700 1 --2- 172.21.15.67:0/1939479950 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f577806af00 0x7f577806d3c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5788007fd0 tx=0x7f5788009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:25.771 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:25.765+0000 7f578ffff700 1 -- 172.21.15.67:0/1939479950 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f57900543e0 con 0x7f579815f130 2023-10-14T10:36:26.015 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:26.012+0000 7f579dab6700 1 -- 172.21.15.67:0/1939479950 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f57980c7fb0 con 0x7f579815f130 2023-10-14T10:36:26.015 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:26.013+0000 7f578ffff700 1 -- 172.21.15.67:0/1939479950 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f5790057a00 con 0x7f579815f130 2023-10-14T10:36:26.015 INFO:teuthology.orchestra.run.smithi067.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-14T10:36:26.017 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:26.015+0000 7f579dab6700 1 -- 172.21.15.67:0/1939479950 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f577806af00 msgr2=0x7f577806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:26.017 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:26.015+0000 7f579dab6700 1 --2- 172.21.15.67:0/1939479950 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f577806af00 0x7f577806d3c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5788007fd0 tx=0x7f5788009040 comp rx=0 tx=0).stop 2023-10-14T10:36:26.018 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:26.015+0000 7f579dab6700 1 -- 172.21.15.67:0/1939479950 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f579815f130 msgr2=0x7f57980b5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:26.018 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:26.015+0000 7f579dab6700 1 --2- 172.21.15.67:0/1939479950 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f579815f130 0x7f57980b5ae0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5790002230 tx=0x7f579000cf50 comp rx=0 tx=0).stop 2023-10-14T10:36:26.018 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:26.015+0000 7f579dab6700 1 -- 172.21.15.67:0/1939479950 shutdown_connections 2023-10-14T10:36:26.018 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:26.015+0000 7f579dab6700 1 --2- 172.21.15.67:0/1939479950 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f577806af00 0x7f577806d3c0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:26.018 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:26.015+0000 7f579dab6700 1 --2- 172.21.15.67:0/1939479950 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f579815f130 0x7f57980b5ae0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:26.019 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:26.015+0000 7f579dab6700 1 --2- 172.21.15.67:0/1939479950 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f579806af50 0x7f5798157020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:26.019 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:26.015+0000 7f579dab6700 1 --2- 172.21.15.67:0/1939479950 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f57980b28b0 0x7f57980c4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:26.019 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:26.015+0000 7f579dab6700 1 -- 172.21.15.67:0/1939479950 >> 172.21.15.67:0/1939479950 conn(0x7f579806dae0 msgr2=0x7f57980ce920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:26.019 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:26.015+0000 7f579dab6700 1 -- 172.21.15.67:0/1939479950 shutdown_connections 2023-10-14T10:36:26.019 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:26.015+0000 7f579dab6700 1 -- 172.21.15.67:0/1939479950 wait complete. 2023-10-14T10:36:26.029 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-14T10:36:26.029 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-14T10:36:26.030 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi072.front.sepia.ceph.com 2023-10-14T10:36:26.030 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-10-14T10:36:26.030 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-10-14T10:36:26.031 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-14T10:36:26.031 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-14T10:36:26.031 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-14T10:36:26.031 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-10-14T10:36:26.032 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:36:26.032 DEBUG:teuthology.orchestra.run.smithi072:> ip addr 2023-10-14T10:36:26.051 INFO:teuthology.orchestra.run.smithi072.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-14T10:36:26.051 INFO:teuthology.orchestra.run.smithi072.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-14T10:36:26.051 INFO:teuthology.orchestra.run.smithi072.stdout: inet 127.0.0.1/8 scope host lo 2023-10-14T10:36:26.052 INFO:teuthology.orchestra.run.smithi072.stdout: valid_lft forever preferred_lft forever 2023-10-14T10:36:26.052 INFO:teuthology.orchestra.run.smithi072.stdout: inet6 ::1/128 scope host 2023-10-14T10:36:26.052 INFO:teuthology.orchestra.run.smithi072.stdout: valid_lft forever preferred_lft forever 2023-10-14T10:36:26.052 INFO:teuthology.orchestra.run.smithi072.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-14T10:36:26.052 INFO:teuthology.orchestra.run.smithi072.stdout: link/ether 0c:c4:7a:d8:da:f4 brd ff:ff:ff:ff:ff:ff 2023-10-14T10:36:26.053 INFO:teuthology.orchestra.run.smithi072.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-14T10:36:26.053 INFO:teuthology.orchestra.run.smithi072.stdout: link/ether 0c:c4:7a:d8:da:f5 brd ff:ff:ff:ff:ff:ff 2023-10-14T10:36:26.053 INFO:teuthology.orchestra.run.smithi072.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-14T10:36:26.053 INFO:teuthology.orchestra.run.smithi072.stdout: link/ether 0c:c4:7a:8f:b7:64 brd ff:ff:ff:ff:ff:ff 2023-10-14T10:36:26.053 INFO:teuthology.orchestra.run.smithi072.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-14T10:36:26.054 INFO:teuthology.orchestra.run.smithi072.stdout: link/ether 0c:c4:7a:8f:b7:65 brd ff:ff:ff:ff:ff:ff 2023-10-14T10:36:26.054 INFO:teuthology.orchestra.run.smithi072.stdout: inet 172.21.15.72/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-14T10:36:26.054 INFO:teuthology.orchestra.run.smithi072.stdout: valid_lft 42383sec preferred_lft 42383sec 2023-10-14T10:36:26.054 INFO:teuthology.orchestra.run.smithi072.stdout: inet6 fe80::b3b8:cd9b:8a75:edd6/64 scope link noprefixroute 2023-10-14T10:36:26.054 INFO:teuthology.orchestra.run.smithi072.stdout: valid_lft forever preferred_lft forever 2023-10-14T10:36:26.055 INFO:teuthology.orchestra.run.smithi072.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-14T10:36:26.055 INFO:teuthology.orchestra.run.smithi072.stdout: link/ether 3a:1d:0f:d9:5d:f7 brd ff:ff:ff:ff:ff:ff 2023-10-14T10:36:26.055 INFO:teuthology.orchestra.run.smithi072.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-14T10:36:26.055 INFO:teuthology.orchestra.run.smithi072.stdout: valid_lft forever preferred_lft forever 2023-10-14T10:36:26.055 INFO:teuthology.orchestra.run.smithi072.stdout: inet6 fe80::c0a4:c5ff:fe12:e90/64 scope link 2023-10-14T10:36:26.056 INFO:teuthology.orchestra.run.smithi072.stdout: valid_lft forever preferred_lft forever 2023-10-14T10:36:26.056 INFO:teuthology.orchestra.run.smithi072.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-14T10:36:26.056 INFO:teuthology.orchestra.run.smithi072.stdout: link/ether 3a:1d:0f:d9:5d:f7 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-14T10:36:26.056 INFO:teuthology.orchestra.run.smithi072.stdout: inet6 fe80::381d:fff:fed9:5df7/64 scope link tentative 2023-10-14T10:36:26.056 INFO:teuthology.orchestra.run.smithi072.stdout: valid_lft forever preferred_lft forever 2023-10-14T10:36:26.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:36:26.057 DEBUG:teuthology.orchestra.run.smithi072:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-14T10:36:26.117 INFO:teuthology.orchestra.run.smithi072.stdout:1 2023-10-14T10:36:26.118 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:36:26.118 DEBUG:teuthology.orchestra.run.smithi072:> route 2023-10-14T10:36:26.178 INFO:teuthology.orchestra.run.smithi072.stdout:Kernel IP routing table 2023-10-14T10:36:26.179 INFO:teuthology.orchestra.run.smithi072.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-14T10:36:26.179 INFO:teuthology.orchestra.run.smithi072.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-14T10:36:26.179 INFO:teuthology.orchestra.run.smithi072.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-14T10:36:26.179 INFO:teuthology.orchestra.run.smithi072.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-14T10:36:26.180 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:36:26.180 DEBUG:teuthology.orchestra.run.smithi072:> (cd / && exec bash -c ' 2023-10-14T10:36:26.180 DEBUG:teuthology.orchestra.run.smithi072:> set -e 2023-10-14T10:36:26.181 DEBUG:teuthology.orchestra.run.smithi072:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-14T10:36:26.181 DEBUG:teuthology.orchestra.run.smithi072:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-14T10:36:26.181 DEBUG:teuthology.orchestra.run.smithi072:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-14T10:36:26.182 DEBUG:teuthology.orchestra.run.smithi072:> ') 2023-10-14T10:36:26.284 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:36:26.285 DEBUG:teuthology.orchestra.run.smithi072:> ip netns list 2023-10-14T10:36:26.301 INFO:teuthology.orchestra.run.smithi072.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-14T10:36:26.302 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:36:26.302 DEBUG:teuthology.orchestra.run.smithi072:> ip netns list-id 2023-10-14T10:36:26.357 INFO:teuthology.orchestra.run.smithi072.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-10-14T10:36:26.358 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:36:26.359 DEBUG:teuthology.orchestra.run.smithi072:> (cd / && exec bash -c ' 2023-10-14T10:36:26.359 DEBUG:teuthology.orchestra.run.smithi072:> set -e 2023-10-14T10:36:26.359 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-14T10:36:26.359 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-10-14T10:36:26.360 DEBUG:teuthology.orchestra.run.smithi072:> ') 2023-10-14T10:36:26.451 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:36:26.451 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-10-14T10:36:26.490 INFO:teuthology.orchestra.run.smithi072.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-14T10:36:26.490 INFO:teuthology.orchestra.run.smithi072.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-14T10:36:26.490 INFO:teuthology.orchestra.run.smithi072.stdout: inet 127.0.0.1/8 scope host lo 2023-10-14T10:36:26.491 INFO:teuthology.orchestra.run.smithi072.stdout: valid_lft forever preferred_lft forever 2023-10-14T10:36:26.491 INFO:teuthology.orchestra.run.smithi072.stdout: inet6 ::1/128 scope host 2023-10-14T10:36:26.491 INFO:teuthology.orchestra.run.smithi072.stdout: valid_lft forever preferred_lft forever 2023-10-14T10:36:26.491 INFO:teuthology.orchestra.run.smithi072.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-14T10:36:26.492 INFO:teuthology.orchestra.run.smithi072.stdout: link/ether 9e:62:c0:09:6f:69 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-14T10:36:26.492 INFO:teuthology.orchestra.run.smithi072.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-14T10:36:26.492 INFO:teuthology.orchestra.run.smithi072.stdout: valid_lft forever preferred_lft forever 2023-10-14T10:36:26.492 INFO:teuthology.orchestra.run.smithi072.stdout: inet6 fe80::9c62:c0ff:fe09:6f69/64 scope link 2023-10-14T10:36:26.492 INFO:teuthology.orchestra.run.smithi072.stdout: valid_lft forever preferred_lft forever 2023-10-14T10:36:26.502 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:36:26.502 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-10-14T10:36:26.576 INFO:teuthology.orchestra.run.smithi072.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-14T10:36:26.576 INFO:teuthology.orchestra.run.smithi072.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-14T10:36:26.576 INFO:teuthology.orchestra.run.smithi072.stdout: inet 127.0.0.1/8 scope host lo 2023-10-14T10:36:26.577 INFO:teuthology.orchestra.run.smithi072.stdout: valid_lft forever preferred_lft forever 2023-10-14T10:36:26.577 INFO:teuthology.orchestra.run.smithi072.stdout: inet6 ::1/128 scope host 2023-10-14T10:36:26.577 INFO:teuthology.orchestra.run.smithi072.stdout: valid_lft forever preferred_lft forever 2023-10-14T10:36:26.578 INFO:teuthology.orchestra.run.smithi072.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-14T10:36:26.578 INFO:teuthology.orchestra.run.smithi072.stdout: link/ether 9e:62:c0:09:6f:69 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-14T10:36:26.578 INFO:teuthology.orchestra.run.smithi072.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-14T10:36:26.578 INFO:teuthology.orchestra.run.smithi072.stdout: valid_lft forever preferred_lft forever 2023-10-14T10:36:26.579 INFO:teuthology.orchestra.run.smithi072.stdout: inet6 fe80::9c62:c0ff:fe09:6f69/64 scope link 2023-10-14T10:36:26.579 INFO:teuthology.orchestra.run.smithi072.stdout: valid_lft forever preferred_lft forever 2023-10-14T10:36:26.591 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-10-14T10:36:26.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:36:26.592 DEBUG:teuthology.orchestra.run.smithi072:> (cd / && exec bash -c ' 2023-10-14T10:36:26.592 DEBUG:teuthology.orchestra.run.smithi072:> set -e 2023-10-14T10:36:26.593 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-10-14T10:36:26.593 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-10-14T10:36:26.593 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-10-14T10:36:26.593 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-10-14T10:36:26.594 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-10-14T10:36:26.594 DEBUG:teuthology.orchestra.run.smithi072:> ') 2023-10-14T10:36:26.851 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:36:26.851 DEBUG:teuthology.orchestra.run.smithi072:> (cd / && exec bash -c ' 2023-10-14T10:36:26.851 DEBUG:teuthology.orchestra.run.smithi072:> set -e 2023-10-14T10:36:26.852 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip link set brx.1 up 2023-10-14T10:36:26.852 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip link set dev brx.1 master ceph-brx 2023-10-14T10:36:26.852 DEBUG:teuthology.orchestra.run.smithi072:> ') 2023-10-14T10:36:26.967 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-14T10:36:26.967 DEBUG:teuthology.orchestra.run.smithi072:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-10-14T10:36:26.988 INFO:teuthology.orchestra.run.smithi072.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-10-14T10:36:26.989 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-14T10:36:26.989 DEBUG:teuthology.orchestra.run.smithi072:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-10-14T10:36:27.045 INFO:tasks.cephfs.kernel_mount:mounting using device: 3@.cephfs=/ 2023-10-14T10:36:27.045 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:36:27.046 DEBUG:teuthology.orchestra.run.smithi072:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 3@.cephfs=/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-14T10:36:27.131 INFO:teuthology.orchestra.run.smithi072.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-14T10:36:27.131 INFO:teuthology.orchestra.run.smithi072.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-14T10:36:27.158 INFO:teuthology.orchestra.run.smithi072.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-14T10:36:27.159 INFO:teuthology.orchestra.run.smithi072.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-14T10:36:27.159 INFO:teuthology.orchestra.run.smithi072.stdout:mount.ceph: resolved to: "172.21.15.67:3300,172.21.15.72:3300,172.21.15.72:3301" 2023-10-14T10:36:27.159 INFO:teuthology.orchestra.run.smithi072.stdout:mount.ceph: trying mount with new device syntax: 3@37f9ab46-3082-4b71-8b24-08d39897b008.cephfs=/ 2023-10-14T10:36:27.159 INFO:teuthology.orchestra.run.smithi072.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.67:3300/172.21.15.72:3300/172.21.15.72:3301" will pass to kernel 2023-10-14T10:36:27.159 INFO:teuthology.orchestra.run.smithi072.stdout:mount.ceph: v2 mount support check returned 2 2023-10-14T10:36:27.160 INFO:teuthology.orchestra.run.smithi072.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-14T10:36:27.160 INFO:teuthology.orchestra.run.smithi072.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-14T10:36:27.234 INFO:teuthology.orchestra.run.smithi072.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-14T10:36:27.234 INFO:teuthology.orchestra.run.smithi072.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-14T10:36:27.235 INFO:teuthology.orchestra.run.smithi072.stdout:mount.ceph: resolved to: "172.21.15.67:3300,172.21.15.72:3300,172.21.15.72:3301" 2023-10-14T10:36:27.235 INFO:teuthology.orchestra.run.smithi072.stdout:mount.ceph: trying mount with new device syntax: 3@37f9ab46-3082-4b71-8b24-08d39897b008.cephfs=/ 2023-10-14T10:36:27.235 INFO:teuthology.orchestra.run.smithi072.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.67:3300/172.21.15.72:3300/172.21.15.72:3301" will pass to kernel 2023-10-14T10:36:27.235 INFO:teuthology.orchestra.run.smithi072.stdout:mount.ceph: v2 mount support check returned 2 2023-10-14T10:36:27.236 INFO:teuthology.orchestra.run.smithi072.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-14T10:36:27.236 INFO:teuthology.orchestra.run.smithi072.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-14T10:36:27.236 INFO:teuthology.orchestra.run.smithi072.stdout:mount.ceph: resolved to: "172.21.15.67:6789,172.21.15.72:6789,172.21.15.72:6790" 2023-10-14T10:36:27.236 INFO:teuthology.orchestra.run.smithi072.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.67:6789,172.21.15.72:6789,172.21.15.72:6790:/ 2023-10-14T10:36:27.236 INFO:teuthology.orchestra.run.smithi072.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,key=3,mds_namespace=cephfs,fsid=37f9ab46-3082-4b71-8b24-08d39897b008" will pass to kernel 2023-10-14T10:36:27.237 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-14T10:36:27.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:36:27.238 DEBUG:teuthology.orchestra.run.smithi072:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-10-14T10:36:27.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:36:27.265 DEBUG:teuthology.orchestra.run.smithi072:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-14T10:36:27.337 INFO:teuthology.orchestra.run.smithi072.stdout:client4695 2023-10-14T10:36:27.337 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-10-14T10:36:27.338 DEBUG:teuthology.orchestra.run.smithi072:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-14T10:36:27.394 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:36:27.395 DEBUG:teuthology.orchestra.run.smithi072:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-14T10:36:27.466 INFO:teuthology.orchestra.run.smithi072.stdout:client4695 2023-10-14T10:36:27.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:36:27.467 DEBUG:teuthology.orchestra.run.smithi072:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-14T10:36:27.539 INFO:teuthology.orchestra.run.smithi072.stdout:client4695 2023-10-14T10:36:27.539 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:36:27.540 DEBUG:teuthology.orchestra.run.smithi072:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/37f9ab46-3082-4b71-8b24-08d39897b008.client4695/status') 2023-10-14T10:36:27.619 INFO:teuthology.orchestra.run.smithi072.stdout:instance: client.4695 (3)172.21.15.72:0/1775880675 2023-10-14T10:36:27.619 INFO:teuthology.orchestra.run.smithi072.stdout:blocklisted: false 2023-10-14T10:36:27.621 INFO:teuthology.orchestra.run.smithi072.stderr:0+1 records in 2023-10-14T10:36:27.621 INFO:teuthology.orchestra.run.smithi072.stderr:0+1 records out 2023-10-14T10:36:27.621 INFO:teuthology.orchestra.run.smithi072.stderr:70 bytes copied, 2.2716e-05 s, 3.1 MB/s 2023-10-14T10:36:27.622 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-10-14T10:36:27.646 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-14T10:36:27.786 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:27.783+0000 7febb30d9700 1 Processor -- start 2023-10-14T10:36:27.786 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:27.783+0000 7febb30d9700 1 -- start start 2023-10-14T10:36:27.788 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:27.784+0000 7febb30d9700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7febac15bb50 0x7febac06e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:27.789 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:27.784+0000 7febb30d9700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febac15bf30 con 0x7febac0ce280 2023-10-14T10:36:27.789 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:27.784+0000 7febb30d9700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febac06d500 con 0x7febac06ed50 2023-10-14T10:36:27.789 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:27.784+0000 7febb30d9700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febac06f1d0 con 0x7febac15bb50 2023-10-14T10:36:27.789 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:27.784+0000 7febb0e75700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7febac15bb50 0x7febac06e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:27.790 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:27.784+0000 7febb1676700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7febac0ce280 0x7febac06a840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:52406/0 (socket says 172.21.15.67:52406) 2023-10-14T10:36:27.790 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:27.784+0000 7febb1676700 1 -- 172.21.15.67:0/363572088 learned_addr learned my addr 172.21.15.67:0/363572088 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:36:27.790 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:27.785+0000 7febb0e75700 1 -- 172.21.15.67:0/363572088 >> v1:172.21.15.72:6789/0 conn(0x7febac06ed50 legacy=0x7febac0cdb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:27.791 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:27.785+0000 7febb0e75700 1 -- 172.21.15.67:0/363572088 >> v1:172.21.15.67:6789/0 conn(0x7febac0ce280 legacy=0x7febac06a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:27.791 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:27.785+0000 7febb0e75700 1 -- 172.21.15.67:0/363572088 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febac06d7e0 con 0x7febac15bb50 2023-10-14T10:36:27.791 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:27.785+0000 7febb0e75700 1 --2- 172.21.15.67:0/363572088 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7febac15bb50 0x7febac06e810 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7feba0009840 tx=0x7feba0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdca3b1df8b8d440 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:27.791 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:27.785+0000 7febab7fe700 1 -- 172.21.15.67:0/363572088 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba0011830 con 0x7febac15bb50 2023-10-14T10:36:27.792 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:27.785+0000 7febab7fe700 1 -- 172.21.15.67:0/363572088 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feba0011990 con 0x7febac15bb50 2023-10-14T10:36:27.792 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:27.785+0000 7febab7fe700 1 -- 172.21.15.67:0/363572088 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba0011d20 con 0x7febac15bb50 2023-10-14T10:36:27.792 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:27.785+0000 7febb30d9700 1 -- 172.21.15.67:0/363572088 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7febac15bb50 msgr2=0x7febac06e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:27.792 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:27.785+0000 7febb30d9700 1 --2- 172.21.15.67:0/363572088 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7febac15bb50 0x7febac06e810 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7feba0009840 tx=0x7feba0009b50 comp rx=0 tx=0).stop 2023-10-14T10:36:27.792 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:27.786+0000 7febb30d9700 1 -- 172.21.15.67:0/363572088 shutdown_connections 2023-10-14T10:36:27.793 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:27.786+0000 7febb30d9700 1 --2- 172.21.15.67:0/363572088 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7febac15bb50 0x7febac06e810 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:27.793 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:27.786+0000 7febb30d9700 1 -- 172.21.15.67:0/363572088 >> 172.21.15.67:0/363572088 conn(0x7febac0cc6f0 msgr2=0x7febac0ccb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:27.793 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:27.786+0000 7febb30d9700 1 -- 172.21.15.67:0/363572088 shutdown_connections 2023-10-14T10:36:27.793 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:27.786+0000 7febb30d9700 1 -- 172.21.15.67:0/363572088 wait complete. 2023-10-14T10:36:27.796 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:27.787+0000 7febb30d9700 1 Processor -- start 2023-10-14T10:36:27.796 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:27.787+0000 7febb30d9700 1 -- start start 2023-10-14T10:36:27.796 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:27.788+0000 7febb30d9700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7febac06ed50 0x7febac155480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:27.796 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:27.788+0000 7febb30d9700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7febac0ce280 0x7febac0a90c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:27.797 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:27.788+0000 7febb30d9700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7febac15bb50 0x7febac1596d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:27.797 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:27.788+0000 7febb30d9700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febac1f1de0 con 0x7febac15bb50 2023-10-14T10:36:27.797 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:27.788+0000 7febb30d9700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febac0a9b10 con 0x7febac06ed50 2023-10-14T10:36:27.797 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:27.788+0000 7febb30d9700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febac1f1790 con 0x7febac0ce280 2023-10-14T10:36:27.798 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:27.788+0000 7febb1676700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7febac15bb50 0x7febac1596d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:27.798 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:27.788+0000 7febb1676700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7febac15bb50 0x7febac1596d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44648/0 (socket says 172.21.15.67:44648) 2023-10-14T10:36:27.798 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:27.788+0000 7febb1676700 1 -- 172.21.15.67:0/2444497296 learned_addr learned my addr 172.21.15.67:0/2444497296 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:27.798 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:27.788+0000 7febb0e75700 1 --2- 172.21.15.67:0/2444497296 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7febac06ed50 0x7febac155480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:27.798 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:27.788+0000 7febabfff700 1 --2- 172.21.15.67:0/2444497296 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7febac0ce280 0x7febac0a90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:27.799 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:27.788+0000 7febb1676700 1 -- 172.21.15.67:0/2444497296 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7febac0ce280 msgr2=0x7febac0a90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:27.799 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:27.788+0000 7febb1676700 1 --2- 172.21.15.67:0/2444497296 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7febac0ce280 0x7febac0a90c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:27.799 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:27.788+0000 7febb1676700 1 -- 172.21.15.67:0/2444497296 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7febac06ed50 msgr2=0x7febac155480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:27.799 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:27.788+0000 7febb1676700 1 --2- 172.21.15.67:0/2444497296 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7febac06ed50 0x7febac155480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:27.799 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:27.788+0000 7febb1676700 1 -- 172.21.15.67:0/2444497296 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feba00094a0 con 0x7febac15bb50 2023-10-14T10:36:27.799 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:27.788+0000 7febabfff700 1 --2- 172.21.15.67:0/2444497296 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7febac0ce280 0x7febac0a90c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-14T10:36:27.800 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:27.788+0000 7febb1676700 1 --2- 172.21.15.67:0/2444497296 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7febac15bb50 0x7febac1596d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7feba400cbc0 tx=0x7feba400ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:27.800 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:27.789+0000 7feba97fa700 1 -- 172.21.15.67:0/2444497296 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba400aaa0 con 0x7febac15bb50 2023-10-14T10:36:27.800 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:27.789+0000 7feba97fa700 1 -- 172.21.15.67:0/2444497296 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feba400ac00 con 0x7febac15bb50 2023-10-14T10:36:27.800 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:27.789+0000 7febb30d9700 1 -- 172.21.15.67:0/2444497296 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febac1f1a70 con 0x7febac15bb50 2023-10-14T10:36:27.800 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:27.789+0000 7feba97fa700 1 -- 172.21.15.67:0/2444497296 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba4012400 con 0x7febac15bb50 2023-10-14T10:36:27.801 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:27.789+0000 7febb30d9700 1 -- 172.21.15.67:0/2444497296 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febac1f2270 con 0x7febac15bb50 2023-10-14T10:36:27.801 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:27.790+0000 7febb30d9700 1 -- 172.21.15.67:0/2444497296 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febac1f1f50 con 0x7febac15bb50 2023-10-14T10:36:27.801 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:27.790+0000 7feba97fa700 1 -- 172.21.15.67:0/2444497296 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7feba401b3b0 con 0x7febac15bb50 2023-10-14T10:36:27.801 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:27.791+0000 7feba97fa700 1 --2- 172.21.15.67:0/2444497296 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7feb9006af00 0x7feb9006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:27.801 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:27.791+0000 7feba97fa700 1 -- 172.21.15.67:0/2444497296 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7feba4089140 con 0x7febac15bb50 2023-10-14T10:36:27.802 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:27.791+0000 7febb0e75700 1 --2- 172.21.15.67:0/2444497296 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7feb9006af00 0x7feb9006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:27.802 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:27.791+0000 7febb0e75700 1 --2- 172.21.15.67:0/2444497296 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7feb9006af00 0x7feb9006d3c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7feba00090b0 tx=0x7feba001c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:27.802 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:27.795+0000 7feba97fa700 1 -- 172.21.15.67:0/2444497296 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7feba40542a0 con 0x7febac15bb50 2023-10-14T10:36:28.005 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.003+0000 7febb30d9700 1 -- 172.21.15.67:0/2444497296 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7febac0ca0a0 con 0x7febac15bb50 2023-10-14T10:36:28.006 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.004+0000 7feba97fa700 1 -- 172.21.15.67:0/2444497296 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16457 (secure 0 0 0) 0x7feba40578c0 con 0x7febac15bb50 2023-10-14T10:36:28.006 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-14T10:36:28.007 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":22,"fsid":"37f9ab46-3082-4b71-8b24-08d39897b008","created":"2023-10-14T10:35:46.067554+0000","modified":"2023-10-14T10:36:04.837296+0000","last_up_change":"2023-10-14T10:35:56.668142+0000","last_in_change":"2023-10-14T10:35:48.551490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-14T10:35:57.809566+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-14T10:36:01.667146+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-14T10:36:02.219700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2b59a07e-11ef-440c-b766-9f64ac885838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6801","nonce":1287255499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6803","nonce":1287255499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6807","nonce":1287255499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6805","nonce":1287255499}]},"public_addr":"172.21.15.67:6801/1287255499","cluster_addr":"172.21.15.67:6803/1287255499","heartbeat_back_addr":"172.21.15.67:6807/1287255499","heartbeat_front_addr":"172.21.15.67:6805/1287255499","state":["exists","up"]},{"osd":1,"uuid":"6f813bd6-fa0b-487d-a204-431eb1276a60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6818","nonce":2435538787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6822","nonce":2435538787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6830","nonce":2435538787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6826","nonce":2435538787}]},"public_addr":"172.21.15.67:6818/2435538787","cluster_addr":"172.21.15.67:6822/2435538787","heartbeat_back_addr":"172.21.15.67:6830/2435538787","heartbeat_front_addr":"172.21.15.67:6826/2435538787","state":["exists","up"]},{"osd":2,"uuid":"6af42c6a-1b59-4669-9c04-b6876788446d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6809","nonce":2827491791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6811","nonce":2827491791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6815","nonce":2827491791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6813","nonce":2827491791}]},"public_addr":"172.21.15.67:6809/2827491791","cluster_addr":"172.21.15.67:6811/2827491791","heartbeat_back_addr":"172.21.15.67:6815/2827491791","heartbeat_front_addr":"172.21.15.67:6813/2827491791","state":["exists","up"]},{"osd":3,"uuid":"1d889d76-a646-499d-bfff-ed5bae5434ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6819","nonce":1848001176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6821","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6823","nonce":1848001176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6831","nonce":1848001176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6825","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6827","nonce":1848001176}]},"public_addr":"172.21.15.67:6819/1848001176","cluster_addr":"172.21.15.67:6823/1848001176","heartbeat_back_addr":"172.21.15.67:6831/1848001176","heartbeat_front_addr":"172.21.15.67:6827/1848001176","state":["exists","up"]},{"osd":4,"uuid":"c3271dff-b437-4fdd-b13b-6e49a35473af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6804","nonce":1891230785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6808","nonce":1891230785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6815","nonce":1891230785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6811","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6813","nonce":1891230785}]},"public_addr":"172.21.15.72:6804/1891230785","cluster_addr":"172.21.15.72:6808/1891230785","heartbeat_back_addr":"172.21.15.72:6815/1891230785","heartbeat_front_addr":"172.21.15.72:6813/1891230785","state":["exists","up"]},{"osd":5,"uuid":"0337bb18-4b68-4eff-b528-a667275e6926","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6801","nonce":569686991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6803","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6805","nonce":569686991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6812","nonce":569686991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6807","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6809","nonce":569686991}]},"public_addr":"172.21.15.72:6801/569686991","cluster_addr":"172.21.15.72:6805/569686991","heartbeat_back_addr":"172.21.15.72:6812/569686991","heartbeat_front_addr":"172.21.15.72:6809/569686991","state":["exists","up"]},{"osd":6,"uuid":"8e92d9cf-0c8c-46eb-89a7-db2023ef1509","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6817","nonce":3767685934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6819","nonce":3767685934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6823","nonce":3767685934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6821","nonce":3767685934}]},"public_addr":"172.21.15.72:6817/3767685934","cluster_addr":"172.21.15.72:6819/3767685934","heartbeat_back_addr":"172.21.15.72:6823/3767685934","heartbeat_front_addr":"172.21.15.72:6821/3767685934","state":["exists","up"]},{"osd":7,"uuid":"3fd6aa7d-50a9-45c9-9561-5902a044a119","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6825","nonce":3567465926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6827","nonce":3567465926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6831","nonce":3567465926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6829","nonce":3567465926}]},"public_addr":"172.21.15.72:6825/3567465926","cluster_addr":"172.21.15.72:6827/3567465926","heartbeat_back_addr":"172.21.15.72:6831/3567465926","heartbeat_front_addr":"172.21.15.72:6829/3567465926","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.511466+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.795107+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.858674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.728895+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.794842+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.750064+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.732122+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.765185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-14T10:36:28.008 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.006+0000 7febb30d9700 1 -- 172.21.15.67:0/2444497296 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7feb9006af00 msgr2=0x7feb9006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:28.008 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.006+0000 7febb30d9700 1 --2- 172.21.15.67:0/2444497296 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7feb9006af00 0x7feb9006d3c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7feba00090b0 tx=0x7feba001c040 comp rx=0 tx=0).stop 2023-10-14T10:36:28.009 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.006+0000 7febb30d9700 1 -- 172.21.15.67:0/2444497296 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7febac15bb50 msgr2=0x7febac1596d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:28.009 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.006+0000 7febb30d9700 1 --2- 172.21.15.67:0/2444497296 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7febac15bb50 0x7febac1596d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7feba400cbc0 tx=0x7feba400ced0 comp rx=0 tx=0).stop 2023-10-14T10:36:28.009 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.006+0000 7febb30d9700 1 -- 172.21.15.67:0/2444497296 shutdown_connections 2023-10-14T10:36:28.009 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.006+0000 7febb30d9700 1 --2- 172.21.15.67:0/2444497296 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7feb9006af00 0x7feb9006d3c0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:28.010 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.006+0000 7febb30d9700 1 --2- 172.21.15.67:0/2444497296 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7febac15bb50 0x7febac1596d0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:28.010 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.006+0000 7febb30d9700 1 --2- 172.21.15.67:0/2444497296 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7febac06ed50 0x7febac155480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:28.010 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.006+0000 7febb30d9700 1 --2- 172.21.15.67:0/2444497296 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7febac0ce280 0x7febac0a90c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:28.011 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.006+0000 7febb30d9700 1 -- 172.21.15.67:0/2444497296 >> 172.21.15.67:0/2444497296 conn(0x7febac0cc6f0 msgr2=0x7febac0b2db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:28.011 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.006+0000 7febb30d9700 1 -- 172.21.15.67:0/2444497296 shutdown_connections 2023-10-14T10:36:28.011 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.006+0000 7febb30d9700 1 -- 172.21.15.67:0/2444497296 wait complete. 2023-10-14T10:36:28.020 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-14T10:35:57.809566+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-14T10:36:01.667146+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-14T10:36:02.219700+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-14T10:36:28.021 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-14T10:36:28.149 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.147+0000 7fa91d2f3700 1 Processor -- start 2023-10-14T10:36:28.149 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.147+0000 7fa91d2f3700 1 -- start start 2023-10-14T10:36:28.152 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.147+0000 7fa91d2f3700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa9180c67d0 0x7fa9180cec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:28.152 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.147+0000 7fa91d2f3700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fa91806bf60 0x7fa91815cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:28.152 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.147+0000 7fa91d2f3700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9180cbe80 con 0x7fa9180cab20 2023-10-14T10:36:28.152 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.147+0000 7fa91d2f3700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9180c5ae0 con 0x7fa9180c67d0 2023-10-14T10:36:28.152 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.147+0000 7fa91d2f3700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9180cafb0 con 0x7fa91806bf60 2023-10-14T10:36:28.153 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.148+0000 7fa916ffd700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa9180c67d0 0x7fa9180cec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:28.153 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.148+0000 7fa916ffd700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa9180c67d0 0x7fa9180cec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44654/0 (socket says 172.21.15.67:44654) 2023-10-14T10:36:28.153 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.148+0000 7fa916ffd700 1 -- 172.21.15.67:0/259588327 learned_addr learned my addr 172.21.15.67:0/259588327 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:28.153 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.148+0000 7fa9177fe700 1 --2- 172.21.15.67:0/259588327 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fa91806bf60 0x7fa91815cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:28.154 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.148+0000 7fa916ffd700 1 -- 172.21.15.67:0/259588327 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fa91806bf60 msgr2=0x7fa91815cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:28.154 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.148+0000 7fa916ffd700 1 --2- 172.21.15.67:0/259588327 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fa91806bf60 0x7fa91815cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:28.154 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.148+0000 7fa916ffd700 1 -- 172.21.15.67:0/259588327 >> v1:172.21.15.72:6789/0 conn(0x7fa9180cab20 legacy=0x7fa91806b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:28.154 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.148+0000 7fa916ffd700 1 -- 172.21.15.67:0/259588327 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9180cdfd0 con 0x7fa9180c67d0 2023-10-14T10:36:28.154 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.148+0000 7fa916ffd700 1 --2- 172.21.15.67:0/259588327 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9180c67d0 0x7fa9180cec20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa9080098f0 tx=0x7fa908009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=804f730c3ed8b4a0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:28.155 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.148+0000 7fa915ffb700 1 -- 172.21.15.67:0/259588327 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa908011830 con 0x7fa9180c67d0 2023-10-14T10:36:28.155 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.148+0000 7fa915ffb700 1 -- 172.21.15.67:0/259588327 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa908011990 con 0x7fa9180c67d0 2023-10-14T10:36:28.155 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.148+0000 7fa915ffb700 1 -- 172.21.15.67:0/259588327 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa908011d20 con 0x7fa9180c67d0 2023-10-14T10:36:28.155 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.149+0000 7fa91d2f3700 1 -- 172.21.15.67:0/259588327 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9180c67d0 msgr2=0x7fa9180cec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:28.155 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.149+0000 7fa91d2f3700 1 --2- 172.21.15.67:0/259588327 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9180c67d0 0x7fa9180cec20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa9080098f0 tx=0x7fa908009c00 comp rx=0 tx=0).stop 2023-10-14T10:36:28.156 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.149+0000 7fa91d2f3700 1 -- 172.21.15.67:0/259588327 shutdown_connections 2023-10-14T10:36:28.156 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.149+0000 7fa91d2f3700 1 --2- 172.21.15.67:0/259588327 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fa91806bf60 0x7fa91815cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:28.156 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.149+0000 7fa91d2f3700 1 --2- 172.21.15.67:0/259588327 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9180c67d0 0x7fa9180cec20 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:28.156 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.149+0000 7fa91d2f3700 1 -- 172.21.15.67:0/259588327 >> 172.21.15.67:0/259588327 conn(0x7fa918071310 msgr2=0x7fa918071720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:28.156 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.149+0000 7fa91d2f3700 1 -- 172.21.15.67:0/259588327 shutdown_connections 2023-10-14T10:36:28.157 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.149+0000 7fa91d2f3700 1 -- 172.21.15.67:0/259588327 wait complete. 2023-10-14T10:36:28.162 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.151+0000 7fa91d2f3700 1 Processor -- start 2023-10-14T10:36:28.163 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.151+0000 7fa91d2f3700 1 -- start start 2023-10-14T10:36:28.163 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.151+0000 7fa91d2f3700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa91806bf60 0x7fa918158160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:28.163 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.152+0000 7fa91d2f3700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fa9180c67d0 0x7fa9181f64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:28.163 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.152+0000 7fa91d2f3700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9180cab20 0x7fa9181f69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:28.163 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.152+0000 7fa91d2f3700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9180c8d90 con 0x7fa9180cab20 2023-10-14T10:36:28.164 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.152+0000 7fa91d2f3700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa91815b020 con 0x7fa91806bf60 2023-10-14T10:36:28.164 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.152+0000 7fa91d2f3700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9180b2930 con 0x7fa9180c67d0 2023-10-14T10:36:28.164 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.152+0000 7fa916ffd700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa91806bf60 0x7fa918158160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:28.164 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.152+0000 7fa9167fc700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fa9180c67d0 0x7fa9181f64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:28.164 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.152+0000 7fa916ffd700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa91806bf60 0x7fa918158160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.67:39450/0 (socket says 172.21.15.67:39450) 2023-10-14T10:36:28.165 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.152+0000 7fa916ffd700 1 -- 172.21.15.67:0/3662102416 learned_addr learned my addr 172.21.15.67:0/3662102416 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:28.165 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.152+0000 7fa9177fe700 1 --2- 172.21.15.67:0/3662102416 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9180cab20 0x7fa9181f69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:28.165 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.152+0000 7fa916ffd700 1 -- 172.21.15.67:0/3662102416 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fa9180c67d0 msgr2=0x7fa9181f64a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:28.165 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.152+0000 7fa916ffd700 1 --2- 172.21.15.67:0/3662102416 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fa9180c67d0 0x7fa9181f64a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:28.165 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.152+0000 7fa916ffd700 1 -- 172.21.15.67:0/3662102416 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9180cab20 msgr2=0x7fa9181f69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:28.166 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.152+0000 7fa916ffd700 1 --2- 172.21.15.67:0/3662102416 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9180cab20 0x7fa9181f69e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:28.166 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.152+0000 7fa916ffd700 1 -- 172.21.15.67:0/3662102416 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa908009550 con 0x7fa91806bf60 2023-10-14T10:36:28.166 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.152+0000 7fa9177fe700 1 --2- 172.21.15.67:0/3662102416 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9180cab20 0x7fa9181f69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-14T10:36:28.166 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.152+0000 7fa916ffd700 1 --2- 172.21.15.67:0/3662102416 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa91806bf60 0x7fa918158160 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa908000c00 tx=0x7fa908004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:28.166 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.153+0000 7fa90f7fe700 1 -- 172.21.15.67:0/3662102416 <== mon.1 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa908004480 con 0x7fa91806bf60 2023-10-14T10:36:28.167 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.153+0000 7fa90f7fe700 1 -- 172.21.15.67:0/3662102416 <== mon.1 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa90801f070 con 0x7fa91806bf60 2023-10-14T10:36:28.167 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.153+0000 7fa91d2f3700 1 -- 172.21.15.67:0/3662102416 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9180c84f0 con 0x7fa91806bf60 2023-10-14T10:36:28.167 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.153+0000 7fa90f7fe700 1 -- 172.21.15.67:0/3662102416 <== mon.1 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9080038a0 con 0x7fa91806bf60 2023-10-14T10:36:28.167 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.153+0000 7fa91d2f3700 1 -- 172.21.15.67:0/3662102416 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9180c8ab0 con 0x7fa91806bf60 2023-10-14T10:36:28.167 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.154+0000 7fa91d2f3700 1 -- 172.21.15.67:0/3662102416 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa91804ebb0 con 0x7fa91806bf60 2023-10-14T10:36:28.168 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.154+0000 7fa90f7fe700 1 -- 172.21.15.67:0/3662102416 <== mon.1 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fa908021030 con 0x7fa91806bf60 2023-10-14T10:36:28.168 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.155+0000 7fa90f7fe700 1 --2- 172.21.15.67:0/3662102416 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fa8f806af00 0x7fa8f806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:28.168 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.155+0000 7fa90f7fe700 1 -- 172.21.15.67:0/3662102416 <== mon.1 v2:172.21.15.72:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa90808aa60 con 0x7fa91806bf60 2023-10-14T10:36:28.168 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.155+0000 7fa9167fc700 1 --2- 172.21.15.67:0/3662102416 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fa8f806af00 0x7fa8f806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:28.169 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.155+0000 7fa9167fc700 1 --2- 172.21.15.67:0/3662102416 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fa8f806af00 0x7fa8f806d3c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa9180b4c10 tx=0x7fa900009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:28.169 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.160+0000 7fa90f7fe700 1 -- 172.21.15.67:0/3662102416 <== mon.1 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa908055bd0 con 0x7fa91806bf60 2023-10-14T10:36:28.360 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.358+0000 7fa91d2f3700 1 -- 172.21.15.67:0/3662102416 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa91806b5c0 con 0x7fa91806bf60 2023-10-14T10:36:28.361 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.359+0000 7fa90f7fe700 1 -- 172.21.15.67:0/3662102416 <== mon.1 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa9080591f0 con 0x7fa91806bf60 2023-10-14T10:36:28.361 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2023-10-14T10:36:28.366 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.361+0000 7fa91d2f3700 1 -- 172.21.15.67:0/3662102416 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fa8f806af00 msgr2=0x7fa8f806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:28.366 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.361+0000 7fa91d2f3700 1 --2- 172.21.15.67:0/3662102416 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fa8f806af00 0x7fa8f806d3c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa9180b4c10 tx=0x7fa900009040 comp rx=0 tx=0).stop 2023-10-14T10:36:28.366 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.361+0000 7fa91d2f3700 1 -- 172.21.15.67:0/3662102416 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa91806bf60 msgr2=0x7fa918158160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:28.366 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.361+0000 7fa91d2f3700 1 --2- 172.21.15.67:0/3662102416 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa91806bf60 0x7fa918158160 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa908000c00 tx=0x7fa908004750 comp rx=0 tx=0).stop 2023-10-14T10:36:28.367 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.361+0000 7fa91d2f3700 1 -- 172.21.15.67:0/3662102416 shutdown_connections 2023-10-14T10:36:28.367 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.361+0000 7fa91d2f3700 1 --2- 172.21.15.67:0/3662102416 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fa8f806af00 0x7fa8f806d3c0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:28.367 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.361+0000 7fa91d2f3700 1 --2- 172.21.15.67:0/3662102416 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9180cab20 0x7fa9181f69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:28.367 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.361+0000 7fa91d2f3700 1 --2- 172.21.15.67:0/3662102416 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa91806bf60 0x7fa918158160 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:28.367 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.361+0000 7fa91d2f3700 1 --2- 172.21.15.67:0/3662102416 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fa9180c67d0 0x7fa9181f64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:28.368 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.361+0000 7fa91d2f3700 1 -- 172.21.15.67:0/3662102416 >> 172.21.15.67:0/3662102416 conn(0x7fa918071310 msgr2=0x7fa91806f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:28.368 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.361+0000 7fa91d2f3700 1 -- 172.21.15.67:0/3662102416 shutdown_connections 2023-10-14T10:36:28.368 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.361+0000 7fa91d2f3700 1 -- 172.21.15.67:0/3662102416 wait complete. 2023-10-14T10:36:28.376 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-14T10:36:28.504 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.502+0000 7f5a7258f700 1 Processor -- start 2023-10-14T10:36:28.505 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.502+0000 7f5a7258f700 1 -- start start 2023-10-14T10:36:28.507 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.503+0000 7f5a7258f700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f5a6c15e2f0 0x7f5a6c0cede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:28.507 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.503+0000 7f5a7258f700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f5a6c06a840 0x7f5a6c06acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:28.507 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.503+0000 7f5a7258f700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a6c0b2d60 con 0x7f5a6c0cd950 2023-10-14T10:36:28.508 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.503+0000 7f5a7258f700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a6c1619a0 con 0x7f5a6c06a840 2023-10-14T10:36:28.508 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.503+0000 7f5a7258f700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a6c06b1f0 con 0x7f5a6c15e2f0 2023-10-14T10:36:28.508 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.503+0000 7f5a70b2c700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f5a6c06a840 0x7f5a6c06acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:28.508 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.503+0000 7f5a70b2c700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f5a6c06a840 0x7f5a6c06acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44672/0 (socket says 172.21.15.67:44672) 2023-10-14T10:36:28.508 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.503+0000 7f5a70b2c700 1 -- 172.21.15.67:0/677855582 learned_addr learned my addr 172.21.15.67:0/677855582 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:28.509 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.503+0000 7f5a6bfff700 1 --1- 172.21.15.67:0/677855582 >> v1:172.21.15.67:6789/0 conn(0x7f5a6c0cd950 0x7f5a6c15f570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:52436/0 (socket says 172.21.15.67:52436) 2023-10-14T10:36:28.509 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.503+0000 7f5a6b7fe700 1 --2- 172.21.15.67:0/677855582 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f5a6c15e2f0 0x7f5a6c0cede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:28.509 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.503+0000 7f5a70b2c700 1 -- 172.21.15.67:0/677855582 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f5a6c15e2f0 msgr2=0x7f5a6c0cede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:28.509 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.503+0000 7f5a70b2c700 1 --2- 172.21.15.67:0/677855582 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f5a6c15e2f0 0x7f5a6c0cede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:28.509 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.503+0000 7f5a70b2c700 1 -- 172.21.15.67:0/677855582 >> v1:172.21.15.67:6789/0 conn(0x7f5a6c0cd950 legacy=0x7f5a6c15f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:28.509 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.503+0000 7f5a6affd700 1 -- 172.21.15.67:0/677855582 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2739995142 0 0) 0x7f5a6c0b2d60 con 0x7f5a6c0cd950 2023-10-14T10:36:28.510 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.503+0000 7f5a70b2c700 1 -- 172.21.15.67:0/677855582 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a6c0cdd30 con 0x7f5a6c06a840 2023-10-14T10:36:28.510 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.503+0000 7f5a6b7fe700 1 --2- 172.21.15.67:0/677855582 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f5a6c15e2f0 0x7f5a6c0cede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-14T10:36:28.510 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.504+0000 7f5a70b2c700 1 --2- 172.21.15.67:0/677855582 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5a6c06a840 0x7f5a6c06acb0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5a64009e50 tx=0x7f5a6400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e51c67c9c37c513 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:28.510 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.504+0000 7f5a6affd700 1 -- 172.21.15.67:0/677855582 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6400e070 con 0x7f5a6c06a840 2023-10-14T10:36:28.510 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.504+0000 7f5a6affd700 1 -- 172.21.15.67:0/677855582 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a6400b9c0 con 0x7f5a6c06a840 2023-10-14T10:36:28.511 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.504+0000 7f5a6affd700 1 -- 172.21.15.67:0/677855582 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6400bd70 con 0x7f5a6c06a840 2023-10-14T10:36:28.511 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.504+0000 7f5a7258f700 1 -- 172.21.15.67:0/677855582 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5a6c06a840 msgr2=0x7f5a6c06acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:28.511 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.504+0000 7f5a7258f700 1 --2- 172.21.15.67:0/677855582 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5a6c06a840 0x7f5a6c06acb0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5a64009e50 tx=0x7f5a6400b330 comp rx=0 tx=0).stop 2023-10-14T10:36:28.511 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.504+0000 7f5a7258f700 1 -- 172.21.15.67:0/677855582 shutdown_connections 2023-10-14T10:36:28.511 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.504+0000 7f5a7258f700 1 --2- 172.21.15.67:0/677855582 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5a6c06a840 0x7f5a6c06acb0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:28.512 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.504+0000 7f5a7258f700 1 --2- 172.21.15.67:0/677855582 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f5a6c15e2f0 0x7f5a6c0cede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:28.512 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.504+0000 7f5a7258f700 1 -- 172.21.15.67:0/677855582 >> 172.21.15.67:0/677855582 conn(0x7f5a6c06dae0 msgr2=0x7f5a6c06def0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:28.512 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.504+0000 7f5a7258f700 1 -- 172.21.15.67:0/677855582 shutdown_connections 2023-10-14T10:36:28.512 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.504+0000 7f5a7258f700 1 -- 172.21.15.67:0/677855582 wait complete. 2023-10-14T10:36:28.515 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.506+0000 7f5a7258f700 1 Processor -- start 2023-10-14T10:36:28.515 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.506+0000 7f5a7258f700 1 -- start start 2023-10-14T10:36:28.515 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.506+0000 7f5a7258f700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5a6c06a840 0x7f5a6c1540c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:28.516 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.506+0000 7f5a7258f700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f5a6c0cd950 0x7f5a6c0b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:28.516 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.507+0000 7f5a7258f700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5a6c15e2f0 0x7f5a6c0aff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:28.516 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.507+0000 7f5a7258f700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a6c0ad880 con 0x7f5a6c15e2f0 2023-10-14T10:36:28.516 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.507+0000 7f5a7258f700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a6c0acc70 con 0x7f5a6c06a840 2023-10-14T10:36:28.516 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.507+0000 7f5a7258f700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a6c0acf50 con 0x7f5a6c0cd950 2023-10-14T10:36:28.517 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.507+0000 7f5a70b2c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5a6c15e2f0 0x7f5a6c0aff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:28.517 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.507+0000 7f5a70b2c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5a6c15e2f0 0x7f5a6c0aff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44678/0 (socket says 172.21.15.67:44678) 2023-10-14T10:36:28.517 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.507+0000 7f5a70b2c700 1 -- 172.21.15.67:0/1419025316 learned_addr learned my addr 172.21.15.67:0/1419025316 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:28.517 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.507+0000 7f5a6b7fe700 1 --2- 172.21.15.67:0/1419025316 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f5a6c0cd950 0x7f5a6c0b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:28.517 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.507+0000 7f5a6bfff700 1 --2- 172.21.15.67:0/1419025316 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5a6c06a840 0x7f5a6c1540c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:28.518 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.507+0000 7f5a70b2c700 1 -- 172.21.15.67:0/1419025316 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f5a6c0cd950 msgr2=0x7f5a6c0b4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:28.518 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.507+0000 7f5a70b2c700 1 --2- 172.21.15.67:0/1419025316 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f5a6c0cd950 0x7f5a6c0b4d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:28.518 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.507+0000 7f5a70b2c700 1 -- 172.21.15.67:0/1419025316 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5a6c06a840 msgr2=0x7f5a6c1540c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:28.518 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.507+0000 7f5a70b2c700 1 --2- 172.21.15.67:0/1419025316 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5a6c06a840 0x7f5a6c1540c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:28.518 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.507+0000 7f5a70b2c700 1 -- 172.21.15.67:0/1419025316 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a64009b30 con 0x7f5a6c15e2f0 2023-10-14T10:36:28.519 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.507+0000 7f5a6b7fe700 1 --2- 172.21.15.67:0/1419025316 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f5a6c0cd950 0x7f5a6c0b4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-14T10:36:28.519 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.507+0000 7f5a6bfff700 1 --2- 172.21.15.67:0/1419025316 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5a6c06a840 0x7f5a6c1540c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-14T10:36:28.519 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.507+0000 7f5a70b2c700 1 --2- 172.21.15.67:0/1419025316 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5a6c15e2f0 0x7f5a6c0aff30 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5a6400b730 tx=0x7f5a64004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:28.519 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.507+0000 7f5a68ff9700 1 -- 172.21.15.67:0/1419025316 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a640092e0 con 0x7f5a6c15e2f0 2023-10-14T10:36:28.519 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.507+0000 7f5a68ff9700 1 -- 172.21.15.67:0/1419025316 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a6401c440 con 0x7f5a6c15e2f0 2023-10-14T10:36:28.520 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.507+0000 7f5a68ff9700 1 -- 172.21.15.67:0/1419025316 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6401c810 con 0x7f5a6c15e2f0 2023-10-14T10:36:28.520 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.507+0000 7f5a7258f700 1 -- 172.21.15.67:0/1419025316 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a6c0ad230 con 0x7f5a6c15e2f0 2023-10-14T10:36:28.520 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.507+0000 7f5a7258f700 1 -- 172.21.15.67:0/1419025316 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a6c154600 con 0x7f5a6c15e2f0 2023-10-14T10:36:28.520 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.508+0000 7f5a7258f700 1 -- 172.21.15.67:0/1419025316 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a6c0ada30 con 0x7f5a6c15e2f0 2023-10-14T10:36:28.520 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.509+0000 7f5a68ff9700 1 -- 172.21.15.67:0/1419025316 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f5a6401c970 con 0x7f5a6c15e2f0 2023-10-14T10:36:28.521 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.509+0000 7f5a68ff9700 1 --2- 172.21.15.67:0/1419025316 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f5a4c06af00 0x7f5a4c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:28.521 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.509+0000 7f5a68ff9700 1 -- 172.21.15.67:0/1419025316 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5a6408af50 con 0x7f5a6c15e2f0 2023-10-14T10:36:28.521 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.510+0000 7f5a6bfff700 1 --2- 172.21.15.67:0/1419025316 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f5a4c06af00 0x7f5a4c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:28.521 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.510+0000 7f5a6bfff700 1 --2- 172.21.15.67:0/1419025316 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f5a4c06af00 0x7f5a4c06d3c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5a54000f60 tx=0x7f5a54007ca0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:28.522 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.514+0000 7f5a68ff9700 1 -- 172.21.15.67:0/1419025316 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5a6c0ada30 con 0x7f5a6c15e2f0 2023-10-14T10:36:28.717 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.715+0000 7f5a7258f700 1 -- 172.21.15.67:0/1419025316 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5a6c067d20 con 0x7f5a6c15e2f0 2023-10-14T10:36:28.718 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.716+0000 7f5a68ff9700 1 -- 172.21.15.67:0/1419025316 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5a64056090 con 0x7f5a6c15e2f0 2023-10-14T10:36:28.718 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2023-10-14T10:36:28.720 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.717+0000 7f5a7258f700 1 -- 172.21.15.67:0/1419025316 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f5a4c06af00 msgr2=0x7f5a4c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:28.720 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.717+0000 7f5a7258f700 1 --2- 172.21.15.67:0/1419025316 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f5a4c06af00 0x7f5a4c06d3c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5a54000f60 tx=0x7f5a54007ca0 comp rx=0 tx=0).stop 2023-10-14T10:36:28.721 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.717+0000 7f5a7258f700 1 -- 172.21.15.67:0/1419025316 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5a6c15e2f0 msgr2=0x7f5a6c0aff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:28.721 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.717+0000 7f5a7258f700 1 --2- 172.21.15.67:0/1419025316 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5a6c15e2f0 0x7f5a6c0aff30 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5a6400b730 tx=0x7f5a64004ad0 comp rx=0 tx=0).stop 2023-10-14T10:36:28.721 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.717+0000 7f5a7258f700 1 -- 172.21.15.67:0/1419025316 shutdown_connections 2023-10-14T10:36:28.721 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.717+0000 7f5a7258f700 1 --2- 172.21.15.67:0/1419025316 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f5a4c06af00 0x7f5a4c06d3c0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:28.721 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.717+0000 7f5a7258f700 1 --2- 172.21.15.67:0/1419025316 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5a6c15e2f0 0x7f5a6c0aff30 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:28.722 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.718+0000 7f5a7258f700 1 --2- 172.21.15.67:0/1419025316 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5a6c06a840 0x7f5a6c1540c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:28.722 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.718+0000 7f5a7258f700 1 --2- 172.21.15.67:0/1419025316 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f5a6c0cd950 0x7f5a6c0b4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:28.722 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.718+0000 7f5a7258f700 1 -- 172.21.15.67:0/1419025316 >> 172.21.15.67:0/1419025316 conn(0x7f5a6c06dae0 msgr2=0x7f5a6c0b0b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:28.722 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.718+0000 7f5a7258f700 1 -- 172.21.15.67:0/1419025316 shutdown_connections 2023-10-14T10:36:28.723 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.718+0000 7f5a7258f700 1 -- 172.21.15.67:0/1419025316 wait complete. 2023-10-14T10:36:28.732 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-14T10:36:28.859 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.857+0000 7f3243fdd700 1 Processor -- start 2023-10-14T10:36:28.860 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.857+0000 7f3243fdd700 1 -- start start 2023-10-14T10:36:28.862 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.858+0000 7f3243fdd700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f323c15b740 0x7f323c0d1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:28.862 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.858+0000 7f3243fdd700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f323c0ce6e0 0x7f323c0cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:28.862 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.858+0000 7f3243fdd700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f323c0cf190 con 0x7f323c0d0c70 2023-10-14T10:36:28.863 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.858+0000 7f3243fdd700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f323c15bb20 con 0x7f323c0ce6e0 2023-10-14T10:36:28.863 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.858+0000 7f3243fdd700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f323c0cbd40 con 0x7f323c15b740 2023-10-14T10:36:28.863 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.858+0000 7f324257a700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f323c0ce6e0 0x7f323c0cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:28.863 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.858+0000 7f324257a700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f323c0ce6e0 0x7f323c0cf690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44684/0 (socket says 172.21.15.67:44684) 2023-10-14T10:36:28.863 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.858+0000 7f324257a700 1 -- 172.21.15.67:0/2874834448 learned_addr learned my addr 172.21.15.67:0/2874834448 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:28.864 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.858+0000 7f3241d79700 1 --2- 172.21.15.67:0/2874834448 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f323c15b740 0x7f323c0d1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:28.864 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.859+0000 7f324257a700 1 -- 172.21.15.67:0/2874834448 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f323c15b740 msgr2=0x7f323c0d1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:28.864 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.859+0000 7f324257a700 1 --2- 172.21.15.67:0/2874834448 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f323c15b740 0x7f323c0d1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:28.864 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.859+0000 7f324257a700 1 -- 172.21.15.67:0/2874834448 >> v1:172.21.15.72:6789/0 conn(0x7f323c0d0c70 legacy=0x7f323c0cdfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:28.864 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.859+0000 7f324257a700 1 -- 172.21.15.67:0/2874834448 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f323c0cc020 con 0x7f323c0ce6e0 2023-10-14T10:36:28.865 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.859+0000 7f324257a700 1 --2- 172.21.15.67:0/2874834448 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f323c0ce6e0 0x7f323c0cf690 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3234009e50 tx=0x7f323400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f3e9d4ef29a3db8 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:28.865 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.859+0000 7f3240d77700 1 -- 172.21.15.67:0/2874834448 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f323400e070 con 0x7f323c0ce6e0 2023-10-14T10:36:28.865 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.859+0000 7f3240d77700 1 -- 172.21.15.67:0/2874834448 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f323400b9c0 con 0x7f323c0ce6e0 2023-10-14T10:36:28.865 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.859+0000 7f3240d77700 1 -- 172.21.15.67:0/2874834448 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f323400bd70 con 0x7f323c0ce6e0 2023-10-14T10:36:28.865 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.859+0000 7f3243fdd700 1 -- 172.21.15.67:0/2874834448 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f323c0ce6e0 msgr2=0x7f323c0cf690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:28.866 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.859+0000 7f3243fdd700 1 --2- 172.21.15.67:0/2874834448 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f323c0ce6e0 0x7f323c0cf690 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3234009e50 tx=0x7f323400b330 comp rx=0 tx=0).stop 2023-10-14T10:36:28.866 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.859+0000 7f3243fdd700 1 -- 172.21.15.67:0/2874834448 shutdown_connections 2023-10-14T10:36:28.866 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.859+0000 7f3243fdd700 1 --2- 172.21.15.67:0/2874834448 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f323c0ce6e0 0x7f323c0cf690 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:28.866 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.859+0000 7f3243fdd700 1 --2- 172.21.15.67:0/2874834448 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f323c15b740 0x7f323c0d1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:28.866 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.860+0000 7f3243fdd700 1 -- 172.21.15.67:0/2874834448 >> 172.21.15.67:0/2874834448 conn(0x7f323c0665a0 msgr2=0x7f323c066980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:28.867 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.860+0000 7f3243fdd700 1 -- 172.21.15.67:0/2874834448 shutdown_connections 2023-10-14T10:36:28.867 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.860+0000 7f3243fdd700 1 -- 172.21.15.67:0/2874834448 wait complete. 2023-10-14T10:36:28.869 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.861+0000 7f3243fdd700 1 Processor -- start 2023-10-14T10:36:28.870 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.861+0000 7f3243fdd700 1 -- start start 2023-10-14T10:36:28.870 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.861+0000 7f3243fdd700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f323c0ce6e0 0x7f323c0caf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:28.870 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.861+0000 7f3243fdd700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f323c0d0c70 0x7f323c0a84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:28.870 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.862+0000 7f3243fdd700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f323c15b740 0x7f323c0a89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:28.871 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.862+0000 7f3243fdd700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f323c15a6d0 con 0x7f323c0d0c70 2023-10-14T10:36:28.871 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.862+0000 7f3243fdd700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f323c0c8690 con 0x7f323c0ce6e0 2023-10-14T10:36:28.871 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.862+0000 7f3243fdd700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f323c0ac310 con 0x7f323c15b740 2023-10-14T10:36:28.871 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.862+0000 7f3241578700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f323c0d0c70 0x7f323c0a84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:28.871 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.862+0000 7f3241578700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f323c0d0c70 0x7f323c0a84a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44688/0 (socket says 172.21.15.67:44688) 2023-10-14T10:36:28.872 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.862+0000 7f3241578700 1 -- 172.21.15.67:0/1680120934 learned_addr learned my addr 172.21.15.67:0/1680120934 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:28.872 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.862+0000 7f3241d79700 1 --2- 172.21.15.67:0/1680120934 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f323c0ce6e0 0x7f323c0caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:28.872 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.862+0000 7f324257a700 1 --2- 172.21.15.67:0/1680120934 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f323c15b740 0x7f323c0a89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:28.872 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.862+0000 7f3241578700 1 -- 172.21.15.67:0/1680120934 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f323c15b740 msgr2=0x7f323c0a89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:28.872 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.862+0000 7f3241578700 1 --2- 172.21.15.67:0/1680120934 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f323c15b740 0x7f323c0a89e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:28.873 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.862+0000 7f3241578700 1 -- 172.21.15.67:0/1680120934 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f323c0ce6e0 msgr2=0x7f323c0caf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:28.873 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.862+0000 7f3241578700 1 --2- 172.21.15.67:0/1680120934 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f323c0ce6e0 0x7f323c0caf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:28.873 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.862+0000 7f3241578700 1 -- 172.21.15.67:0/1680120934 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3234009b30 con 0x7f323c0d0c70 2023-10-14T10:36:28.873 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.862+0000 7f3241578700 1 --2- 172.21.15.67:0/1680120934 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f323c0d0c70 0x7f323c0a84a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f323800c490 tx=0x7f323800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:28.873 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.862+0000 7f32327fc700 1 -- 172.21.15.67:0/1680120934 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32380052d0 con 0x7f323c0d0c70 2023-10-14T10:36:28.873 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.862+0000 7f32327fc700 1 -- 172.21.15.67:0/1680120934 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3238005450 con 0x7f323c0d0c70 2023-10-14T10:36:28.874 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.862+0000 7f32327fc700 1 -- 172.21.15.67:0/1680120934 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3238006aa0 con 0x7f323c0d0c70 2023-10-14T10:36:28.874 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.862+0000 7f3243fdd700 1 -- 172.21.15.67:0/1680120934 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f323c0a90b0 con 0x7f323c0d0c70 2023-10-14T10:36:28.874 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.862+0000 7f3243fdd700 1 -- 172.21.15.67:0/1680120934 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f323c0c88e0 con 0x7f323c0d0c70 2023-10-14T10:36:28.874 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.863+0000 7f3243fdd700 1 -- 172.21.15.67:0/1680120934 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f323c067d20 con 0x7f323c0d0c70 2023-10-14T10:36:28.874 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.864+0000 7f32327fc700 1 -- 172.21.15.67:0/1680120934 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f3238003680 con 0x7f323c0d0c70 2023-10-14T10:36:28.875 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.864+0000 7f32327fc700 1 --2- 172.21.15.67:0/1680120934 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f322006af00 0x7f322006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:28.875 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.864+0000 7f32327fc700 1 -- 172.21.15.67:0/1680120934 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3238088800 con 0x7f323c0d0c70 2023-10-14T10:36:28.875 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.865+0000 7f3241d79700 1 --2- 172.21.15.67:0/1680120934 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f322006af00 0x7f322006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:28.875 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.865+0000 7f3241d79700 1 --2- 172.21.15.67:0/1680120934 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f322006af00 0x7f322006d3c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f323c0cac40 tx=0x7f322c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:28.876 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:28.869+0000 7f32327fc700 1 -- 172.21.15.67:0/1680120934 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3238053940 con 0x7f323c0d0c70 2023-10-14T10:36:29.071 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.069+0000 7f3243fdd700 1 -- 172.21.15.67:0/1680120934 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f323c15cca0 con 0x7f323c0d0c70 2023-10-14T10:36:29.072 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.069+0000 7f32327fc700 1 -- 172.21.15.67:0/1680120934 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3238056f60 con 0x7f323c0d0c70 2023-10-14T10:36:29.072 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2023-10-14T10:36:29.074 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.072+0000 7f3243fdd700 1 -- 172.21.15.67:0/1680120934 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f322006af00 msgr2=0x7f322006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:29.074 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.072+0000 7f3243fdd700 1 --2- 172.21.15.67:0/1680120934 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f322006af00 0x7f322006d3c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f323c0cac40 tx=0x7f322c008040 comp rx=0 tx=0).stop 2023-10-14T10:36:29.075 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.072+0000 7f3243fdd700 1 -- 172.21.15.67:0/1680120934 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f323c0d0c70 msgr2=0x7f323c0a84a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:29.075 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.072+0000 7f3243fdd700 1 --2- 172.21.15.67:0/1680120934 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f323c0d0c70 0x7f323c0a84a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f323800c490 tx=0x7f323800c850 comp rx=0 tx=0).stop 2023-10-14T10:36:29.075 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.072+0000 7f3243fdd700 1 -- 172.21.15.67:0/1680120934 shutdown_connections 2023-10-14T10:36:29.075 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.072+0000 7f3243fdd700 1 --2- 172.21.15.67:0/1680120934 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f322006af00 0x7f322006d3c0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:29.075 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.072+0000 7f3243fdd700 1 --2- 172.21.15.67:0/1680120934 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f323c15b740 0x7f323c0a89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:29.075 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.072+0000 7f3243fdd700 1 --2- 172.21.15.67:0/1680120934 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f323c0ce6e0 0x7f323c0caf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:29.076 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.072+0000 7f3243fdd700 1 --2- 172.21.15.67:0/1680120934 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f323c0d0c70 0x7f323c0a84a0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:29.076 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.072+0000 7f3243fdd700 1 -- 172.21.15.67:0/1680120934 >> 172.21.15.67:0/1680120934 conn(0x7f323c0665a0 msgr2=0x7f323c0b5ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:29.076 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.072+0000 7f3243fdd700 1 -- 172.21.15.67:0/1680120934 shutdown_connections 2023-10-14T10:36:29.076 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.072+0000 7f3243fdd700 1 -- 172.21.15.67:0/1680120934 wait complete. 2023-10-14T10:36:29.086 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-14T10:36:29.210 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.208+0000 7f86bc7b2700 1 Processor -- start 2023-10-14T10:36:29.211 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.208+0000 7f86bc7b2700 1 -- start start 2023-10-14T10:36:29.214 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.209+0000 7f86bc7b2700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f86b40cd950 0x7f86b415e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:29.214 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.209+0000 7f86bc7b2700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86b40cf290 con 0x7f86b40cede0 2023-10-14T10:36:29.214 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.209+0000 7f86bc7b2700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86b40b2d60 con 0x7f86b406a840 2023-10-14T10:36:29.215 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.209+0000 7f86bc7b2700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86b41619a0 con 0x7f86b40cd950 2023-10-14T10:36:29.215 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.209+0000 7f86b9d4d700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f86b40cede0 0x7f86b415f570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:52462/0 (socket says 172.21.15.67:52462) 2023-10-14T10:36:29.215 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.209+0000 7f86b9d4d700 1 -- 172.21.15.67:0/3463250510 learned_addr learned my addr 172.21.15.67:0/3463250510 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:36:29.215 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.210+0000 7f86ba54e700 1 --2- 172.21.15.67:0/3463250510 >> v2:172.21.15.72:3300/0 conn(0x7f86b40cd950 0x7f86b415e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:29.215 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.210+0000 7f86b954c700 1 -- 172.21.15.67:0/3463250510 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1181765109 0 0) 0x7f86b40cf290 con 0x7f86b40cede0 2023-10-14T10:36:29.216 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.210+0000 7f86b954c700 1 -- 172.21.15.67:0/3463250510 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86a00035b0 con 0x7f86b40cede0 2023-10-14T10:36:29.216 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.210+0000 7f86b954c700 1 -- 172.21.15.67:0/3463250510 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3194626555 0 0) 0x7f86a00035b0 con 0x7f86b40cede0 2023-10-14T10:36:29.216 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.210+0000 7f86b954c700 1 -- 172.21.15.67:0/3463250510 >> v2:172.21.15.72:3300/0 conn(0x7f86b40cd950 msgr2=0x7f86b415e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:29.216 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.210+0000 7f86b954c700 1 --2- 172.21.15.67:0/3463250510 >> v2:172.21.15.72:3300/0 conn(0x7f86b40cd950 0x7f86b415e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:29.216 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.210+0000 7f86b954c700 1 -- 172.21.15.67:0/3463250510 >> v1:172.21.15.72:6789/0 conn(0x7f86b406a840 legacy=0x7f86b4161df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:29.217 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.210+0000 7f86b954c700 1 -- 172.21.15.67:0/3463250510 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86b40cdd30 con 0x7f86b40cede0 2023-10-14T10:36:29.217 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.210+0000 7f86b954c700 1 -- 172.21.15.67:0/3463250510 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3180043755 0 0) 0x7f86b00017f0 con 0x7f86b40cede0 2023-10-14T10:36:29.217 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.210+0000 7f86b954c700 1 -- 172.21.15.67:0/3463250510 >> v1:172.21.15.67:6789/0 conn(0x7f86b40cede0 legacy=0x7f86b415f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:29.217 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.210+0000 7f86b954c700 1 --2- 172.21.15.67:0/3463250510 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f86a0003a00 0x7f86a0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:29.217 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.210+0000 7f86b954c700 1 --2- 172.21.15.67:0/3463250510 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f86a0004850 0x7f86a0004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:29.218 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.210+0000 7f86b954c700 1 --2- 172.21.15.67:0/3463250510 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86a0005210 0x7f86a000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:29.218 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.210+0000 7f86b954c700 1 -- 172.21.15.67:0/3463250510 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86b41619a0 con 0x7f86a0005210 2023-10-14T10:36:29.218 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.210+0000 7f86b954c700 1 -- 172.21.15.67:0/3463250510 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86a000c0b0 con 0x7f86a0003a00 2023-10-14T10:36:29.218 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.210+0000 7f86b954c700 1 -- 172.21.15.67:0/3463250510 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86a000c270 con 0x7f86a0004850 2023-10-14T10:36:29.218 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.210+0000 7f86ba54e700 1 --2- 172.21.15.67:0/3463250510 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f86a0003a00 0x7f86a0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:29.219 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.210+0000 7f86bad4f700 1 --2- 172.21.15.67:0/3463250510 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86a0005210 0x7f86a000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:29.219 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.210+0000 7f86b9d4d700 1 --2- 172.21.15.67:0/3463250510 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f86a0004850 0x7f86a0004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:29.219 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.211+0000 7f86bad4f700 1 -- 172.21.15.67:0/3463250510 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f86a0004850 msgr2=0x7f86a0004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:29.219 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.211+0000 7f86bad4f700 1 --2- 172.21.15.67:0/3463250510 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f86a0004850 0x7f86a0004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:29.219 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.211+0000 7f86bad4f700 1 -- 172.21.15.67:0/3463250510 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f86a0003a00 msgr2=0x7f86a0003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:29.220 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.211+0000 7f86bad4f700 1 --2- 172.21.15.67:0/3463250510 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f86a0003a00 0x7f86a0003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:29.220 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.211+0000 7f86bad4f700 1 -- 172.21.15.67:0/3463250510 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86a00035b0 con 0x7f86a0005210 2023-10-14T10:36:29.220 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.211+0000 7f86ba54e700 1 --2- 172.21.15.67:0/3463250510 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f86a0003a00 0x7f86a0003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-14T10:36:29.220 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.211+0000 7f86bad4f700 1 -- 172.21.15.67:0/3463250510 reap_dead start 2023-10-14T10:36:29.220 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.211+0000 7f86bad4f700 1 --2- 172.21.15.67:0/3463250510 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86a0005210 0x7f86a000ba10 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f86ac00c190 tx=0x7f86a0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bce4c413aa7fe05 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:29.221 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.211+0000 7f86b954c700 1 -- 172.21.15.67:0/3463250510 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86ac001db0 con 0x7f86a0005210 2023-10-14T10:36:29.221 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.211+0000 7f86b954c700 1 -- 172.21.15.67:0/3463250510 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86ac00ccf0 con 0x7f86a0005210 2023-10-14T10:36:29.221 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.211+0000 7f86b954c700 1 -- 172.21.15.67:0/3463250510 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86ac003c40 con 0x7f86a0005210 2023-10-14T10:36:29.221 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.211+0000 7f86bc7b2700 1 -- 172.21.15.67:0/3463250510 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86a0005210 msgr2=0x7f86a000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:29.221 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.211+0000 7f86bc7b2700 1 --2- 172.21.15.67:0/3463250510 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86a0005210 0x7f86a000ba10 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f86ac00c190 tx=0x7f86a0005880 comp rx=0 tx=0).stop 2023-10-14T10:36:29.221 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.211+0000 7f86bc7b2700 1 -- 172.21.15.67:0/3463250510 shutdown_connections 2023-10-14T10:36:29.222 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.211+0000 7f86bc7b2700 1 --2- 172.21.15.67:0/3463250510 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86a0005210 0x7f86a000ba10 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:29.222 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.211+0000 7f86bc7b2700 1 -- 172.21.15.67:0/3463250510 >> 172.21.15.67:0/3463250510 conn(0x7f86b406dae0 msgr2=0x7f86b406def0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:29.222 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.211+0000 7f86bc7b2700 1 -- 172.21.15.67:0/3463250510 shutdown_connections 2023-10-14T10:36:29.222 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.211+0000 7f86bc7b2700 1 -- 172.21.15.67:0/3463250510 wait complete. 2023-10-14T10:36:29.225 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.213+0000 7f86bc7b2700 1 Processor -- start 2023-10-14T10:36:29.225 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.213+0000 7f86bc7b2700 1 -- start start 2023-10-14T10:36:29.225 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.213+0000 7f86bc7b2700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86a0005210 0x7f86b41f3420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:29.226 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.213+0000 7f86bc7b2700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f86b40aee70 0x7f86b41f7010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:29.226 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.213+0000 7f86bc7b2700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f86b41f7550 0x7f86b41f79d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:29.226 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.213+0000 7f86bc7b2700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86b40af6a0 con 0x7f86a0005210 2023-10-14T10:36:29.226 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.213+0000 7f86bc7b2700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86b40c9e80 con 0x7f86b40aee70 2023-10-14T10:36:29.226 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.213+0000 7f86bc7b2700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86b40ca440 con 0x7f86b41f7550 2023-10-14T10:36:29.227 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.213+0000 7f86ba54e700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86a0005210 0x7f86b41f3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:29.227 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.213+0000 7f86ba54e700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86a0005210 0x7f86b41f3420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44706/0 (socket says 172.21.15.67:44706) 2023-10-14T10:36:29.227 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.213+0000 7f86ba54e700 1 -- 172.21.15.67:0/3324126845 learned_addr learned my addr 172.21.15.67:0/3324126845 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:29.227 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.214+0000 7f86bad4f700 1 --2- 172.21.15.67:0/3324126845 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f86b41f7550 0x7f86b41f79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:29.227 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.214+0000 7f86b9d4d700 1 --2- 172.21.15.67:0/3324126845 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f86b40aee70 0x7f86b41f7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:29.227 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.214+0000 7f86ba54e700 1 -- 172.21.15.67:0/3324126845 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f86b41f7550 msgr2=0x7f86b41f79d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:29.228 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.214+0000 7f86ba54e700 1 --2- 172.21.15.67:0/3324126845 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f86b41f7550 0x7f86b41f79d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:29.228 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.214+0000 7f86ba54e700 1 -- 172.21.15.67:0/3324126845 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f86b40aee70 msgr2=0x7f86b41f7010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:29.228 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.214+0000 7f86ba54e700 1 --2- 172.21.15.67:0/3324126845 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f86b40aee70 0x7f86b41f7010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:29.228 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.214+0000 7f86ba54e700 1 -- 172.21.15.67:0/3324126845 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86b41f3960 con 0x7f86a0005210 2023-10-14T10:36:29.228 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.214+0000 7f86ba54e700 1 --2- 172.21.15.67:0/3324126845 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86a0005210 0x7f86b41f3420 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f86a4011cd0 tx=0x7f86a400f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:29.229 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.214+0000 7f86aaffd700 1 -- 172.21.15.67:0/3324126845 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86a400fb30 con 0x7f86a0005210 2023-10-14T10:36:29.229 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.214+0000 7f86aaffd700 1 -- 172.21.15.67:0/3324126845 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86a400fc90 con 0x7f86a0005210 2023-10-14T10:36:29.229 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.214+0000 7f86bc7b2700 1 -- 172.21.15.67:0/3324126845 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86b40b57b0 con 0x7f86a0005210 2023-10-14T10:36:29.229 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.214+0000 7f86aaffd700 1 -- 172.21.15.67:0/3324126845 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86a4004020 con 0x7f86a0005210 2023-10-14T10:36:29.229 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.214+0000 7f86bc7b2700 1 -- 172.21.15.67:0/3324126845 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86b40ca0d0 con 0x7f86a0005210 2023-10-14T10:36:29.230 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.215+0000 7f86bc7b2700 1 -- 172.21.15.67:0/3324126845 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86b40b2630 con 0x7f86a0005210 2023-10-14T10:36:29.230 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.216+0000 7f86aaffd700 1 -- 172.21.15.67:0/3324126845 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f86a4004180 con 0x7f86a0005210 2023-10-14T10:36:29.230 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.216+0000 7f86aaffd700 1 --2- 172.21.15.67:0/3324126845 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f869806af00 0x7f869806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:29.230 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.216+0000 7f86aaffd700 1 -- 172.21.15.67:0/3324126845 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f86a4089b50 con 0x7f86a0005210 2023-10-14T10:36:29.230 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.216+0000 7f86b9d4d700 1 --2- 172.21.15.67:0/3324126845 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f869806af00 0x7f869806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:29.231 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.217+0000 7f86b9d4d700 1 --2- 172.21.15.67:0/3324126845 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f869806af00 0x7f869806d3c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f86b41f6f70 tx=0x7f86b0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:29.231 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.220+0000 7f86aaffd700 1 -- 172.21.15.67:0/3324126845 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f86a4054c90 con 0x7f86a0005210 2023-10-14T10:36:29.421 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.419+0000 7f86bc7b2700 1 -- 172.21.15.67:0/3324126845 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f86b40b2100 con 0x7f86a0005210 2023-10-14T10:36:29.422 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.420+0000 7f86aaffd700 1 -- 172.21.15.67:0/3324126845 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16457 (secure 0 0 0) 0x7f86a4022070 con 0x7f86a0005210 2023-10-14T10:36:29.422 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-14T10:36:29.423 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":22,"fsid":"37f9ab46-3082-4b71-8b24-08d39897b008","created":"2023-10-14T10:35:46.067554+0000","modified":"2023-10-14T10:36:04.837296+0000","last_up_change":"2023-10-14T10:35:56.668142+0000","last_in_change":"2023-10-14T10:35:48.551490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-14T10:35:57.809566+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-14T10:36:01.667146+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-14T10:36:02.219700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2b59a07e-11ef-440c-b766-9f64ac885838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6801","nonce":1287255499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6803","nonce":1287255499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6807","nonce":1287255499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6805","nonce":1287255499}]},"public_addr":"172.21.15.67:6801/1287255499","cluster_addr":"172.21.15.67:6803/1287255499","heartbeat_back_addr":"172.21.15.67:6807/1287255499","heartbeat_front_addr":"172.21.15.67:6805/1287255499","state":["exists","up"]},{"osd":1,"uuid":"6f813bd6-fa0b-487d-a204-431eb1276a60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6818","nonce":2435538787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6822","nonce":2435538787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6830","nonce":2435538787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6826","nonce":2435538787}]},"public_addr":"172.21.15.67:6818/2435538787","cluster_addr":"172.21.15.67:6822/2435538787","heartbeat_back_addr":"172.21.15.67:6830/2435538787","heartbeat_front_addr":"172.21.15.67:6826/2435538787","state":["exists","up"]},{"osd":2,"uuid":"6af42c6a-1b59-4669-9c04-b6876788446d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6809","nonce":2827491791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6811","nonce":2827491791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6815","nonce":2827491791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6813","nonce":2827491791}]},"public_addr":"172.21.15.67:6809/2827491791","cluster_addr":"172.21.15.67:6811/2827491791","heartbeat_back_addr":"172.21.15.67:6815/2827491791","heartbeat_front_addr":"172.21.15.67:6813/2827491791","state":["exists","up"]},{"osd":3,"uuid":"1d889d76-a646-499d-bfff-ed5bae5434ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6819","nonce":1848001176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6821","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6823","nonce":1848001176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6831","nonce":1848001176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6825","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6827","nonce":1848001176}]},"public_addr":"172.21.15.67:6819/1848001176","cluster_addr":"172.21.15.67:6823/1848001176","heartbeat_back_addr":"172.21.15.67:6831/1848001176","heartbeat_front_addr":"172.21.15.67:6827/1848001176","state":["exists","up"]},{"osd":4,"uuid":"c3271dff-b437-4fdd-b13b-6e49a35473af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6804","nonce":1891230785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6808","nonce":1891230785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6815","nonce":1891230785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6811","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6813","nonce":1891230785}]},"public_addr":"172.21.15.72:6804/1891230785","cluster_addr":"172.21.15.72:6808/1891230785","heartbeat_back_addr":"172.21.15.72:6815/1891230785","heartbeat_front_addr":"172.21.15.72:6813/1891230785","state":["exists","up"]},{"osd":5,"uuid":"0337bb18-4b68-4eff-b528-a667275e6926","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6801","nonce":569686991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6803","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6805","nonce":569686991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6812","nonce":569686991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6807","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6809","nonce":569686991}]},"public_addr":"172.21.15.72:6801/569686991","cluster_addr":"172.21.15.72:6805/569686991","heartbeat_back_addr":"172.21.15.72:6812/569686991","heartbeat_front_addr":"172.21.15.72:6809/569686991","state":["exists","up"]},{"osd":6,"uuid":"8e92d9cf-0c8c-46eb-89a7-db2023ef1509","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6817","nonce":3767685934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6819","nonce":3767685934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6823","nonce":3767685934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6821","nonce":3767685934}]},"public_addr":"172.21.15.72:6817/3767685934","cluster_addr":"172.21.15.72:6819/3767685934","heartbeat_back_addr":"172.21.15.72:6823/3767685934","heartbeat_front_addr":"172.21.15.72:6821/3767685934","state":["exists","up"]},{"osd":7,"uuid":"3fd6aa7d-50a9-45c9-9561-5902a044a119","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6825","nonce":3567465926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6827","nonce":3567465926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6831","nonce":3567465926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6829","nonce":3567465926}]},"public_addr":"172.21.15.72:6825/3567465926","cluster_addr":"172.21.15.72:6827/3567465926","heartbeat_back_addr":"172.21.15.72:6831/3567465926","heartbeat_front_addr":"172.21.15.72:6829/3567465926","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.511466+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.795107+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.858674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.728895+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.794842+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.750064+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.732122+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.765185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-14T10:36:29.425 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.422+0000 7f86bc7b2700 1 -- 172.21.15.67:0/3324126845 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f869806af00 msgr2=0x7f869806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:29.425 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.422+0000 7f86bc7b2700 1 --2- 172.21.15.67:0/3324126845 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f869806af00 0x7f869806d3c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f86b41f6f70 tx=0x7f86b0009040 comp rx=0 tx=0).stop 2023-10-14T10:36:29.425 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.422+0000 7f86bc7b2700 1 -- 172.21.15.67:0/3324126845 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86a0005210 msgr2=0x7f86b41f3420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:29.425 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.422+0000 7f86bc7b2700 1 --2- 172.21.15.67:0/3324126845 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86a0005210 0x7f86b41f3420 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f86a4011cd0 tx=0x7f86a400f520 comp rx=0 tx=0).stop 2023-10-14T10:36:29.426 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.422+0000 7f86bc7b2700 1 -- 172.21.15.67:0/3324126845 shutdown_connections 2023-10-14T10:36:29.426 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.422+0000 7f86bc7b2700 1 --2- 172.21.15.67:0/3324126845 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f869806af00 0x7f869806d3c0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:29.426 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.422+0000 7f86bc7b2700 1 --2- 172.21.15.67:0/3324126845 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f86b41f7550 0x7f86b41f79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:29.426 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.422+0000 7f86bc7b2700 1 --2- 172.21.15.67:0/3324126845 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86a0005210 0x7f86b41f3420 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:29.426 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.422+0000 7f86bc7b2700 1 --2- 172.21.15.67:0/3324126845 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f86b40aee70 0x7f86b41f7010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:29.427 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.422+0000 7f86bc7b2700 1 -- 172.21.15.67:0/3324126845 >> 172.21.15.67:0/3324126845 conn(0x7f86b406dae0 msgr2=0x7f86b4160650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:29.427 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.422+0000 7f86bc7b2700 1 -- 172.21.15.67:0/3324126845 shutdown_connections 2023-10-14T10:36:29.427 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.422+0000 7f86bc7b2700 1 -- 172.21.15.67:0/3324126845 wait complete. 2023-10-14T10:36:29.437 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-14T10:35:57.809566+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-14T10:36:01.667146+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-14T10:36:02.219700+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-14T10:36:29.438 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-14T10:36:29.565 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.563+0000 7f32303de700 1 Processor -- start 2023-10-14T10:36:29.566 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.563+0000 7f32303de700 1 -- start start 2023-10-14T10:36:29.568 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.564+0000 7f32303de700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f322815bb50 0x7f322806e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:29.568 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.564+0000 7f32303de700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f32280ce280 0x7f322806a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:29.568 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.564+0000 7f32303de700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f322815bf30 con 0x7f322806ed50 2023-10-14T10:36:29.568 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.564+0000 7f32303de700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f322806f1d0 con 0x7f32280ce280 2023-10-14T10:36:29.569 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.564+0000 7f32303de700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32280b2d70 con 0x7f322815bb50 2023-10-14T10:36:29.569 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.564+0000 7f322e97b700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f32280ce280 0x7f322806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:29.569 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.564+0000 7f322e17a700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f322815bb50 0x7f322806e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:29.569 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.564+0000 7f322e97b700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f32280ce280 0x7f322806a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.67:39504/0 (socket says 172.21.15.67:39504) 2023-10-14T10:36:29.569 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.564+0000 7f322e97b700 1 -- 172.21.15.67:0/3673300186 learned_addr learned my addr 172.21.15.67:0/3673300186 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:29.570 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.564+0000 7f322d979700 1 --1- 172.21.15.67:0/3673300186 >> v1:172.21.15.72:6789/0 conn(0x7f322806ed50 0x7f32280cdb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.67:60722/0 (socket says 172.21.15.67:60722) 2023-10-14T10:36:29.570 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.565+0000 7f322e17a700 1 -- 172.21.15.67:0/3673300186 >> v2:172.21.15.72:3300/0 conn(0x7f32280ce280 msgr2=0x7f322806a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:29.570 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.565+0000 7f322e17a700 1 --2- 172.21.15.67:0/3673300186 >> v2:172.21.15.72:3300/0 conn(0x7f32280ce280 0x7f322806a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:29.570 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.565+0000 7f322e17a700 1 -- 172.21.15.67:0/3673300186 >> v1:172.21.15.72:6789/0 conn(0x7f322806ed50 legacy=0x7f32280cdb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:29.570 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.565+0000 7f322e17a700 1 -- 172.21.15.67:0/3673300186 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f322806d500 con 0x7f322815bb50 2023-10-14T10:36:29.571 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.565+0000 7f322e17a700 1 --2- 172.21.15.67:0/3673300186 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f322815bb50 0x7f322806e810 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3218009840 tx=0x7f3218009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b242d941bc82571f server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:29.571 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.565+0000 7f322d178700 1 -- 172.21.15.67:0/3673300186 <== mon.2 v2:172.21.15.72:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3218011830 con 0x7f322815bb50 2023-10-14T10:36:29.571 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.565+0000 7f322d178700 1 -- 172.21.15.67:0/3673300186 <== mon.2 v2:172.21.15.72:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3218011990 con 0x7f322815bb50 2023-10-14T10:36:29.571 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.565+0000 7f322d178700 1 -- 172.21.15.67:0/3673300186 <== mon.2 v2:172.21.15.72:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3218011d20 con 0x7f322815bb50 2023-10-14T10:36:29.571 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.566+0000 7f32303de700 1 -- 172.21.15.67:0/3673300186 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f322815bb50 msgr2=0x7f322806e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:29.571 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.566+0000 7f32303de700 1 --2- 172.21.15.67:0/3673300186 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f322815bb50 0x7f322806e810 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3218009840 tx=0x7f3218009b50 comp rx=0 tx=0).stop 2023-10-14T10:36:29.572 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.566+0000 7f32303de700 1 -- 172.21.15.67:0/3673300186 shutdown_connections 2023-10-14T10:36:29.572 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.566+0000 7f32303de700 1 --2- 172.21.15.67:0/3673300186 >> v2:172.21.15.72:3300/0 conn(0x7f32280ce280 0x7f322806a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:29.572 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.566+0000 7f32303de700 1 --2- 172.21.15.67:0/3673300186 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f322815bb50 0x7f322806e810 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:29.572 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.566+0000 7f32303de700 1 -- 172.21.15.67:0/3673300186 >> 172.21.15.67:0/3673300186 conn(0x7f32280cc6f0 msgr2=0x7f32280ccb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:29.572 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.566+0000 7f32303de700 1 -- 172.21.15.67:0/3673300186 shutdown_connections 2023-10-14T10:36:29.573 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.566+0000 7f32303de700 1 -- 172.21.15.67:0/3673300186 wait complete. 2023-10-14T10:36:29.575 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.567+0000 7f32303de700 1 Processor -- start 2023-10-14T10:36:29.575 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.567+0000 7f32303de700 1 -- start start 2023-10-14T10:36:29.576 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.568+0000 7f32303de700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f322806ed50 0x7f322815df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:29.576 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.568+0000 7f32303de700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f32280ce280 0x7f32281523e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:29.576 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.568+0000 7f32303de700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f322815bb50 0x7f32281f65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:29.576 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.568+0000 7f32303de700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3228158190 con 0x7f322806ed50 2023-10-14T10:36:29.577 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.568+0000 7f32303de700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322815cca0 con 0x7f32280ce280 2023-10-14T10:36:29.577 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.568+0000 7f32303de700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32281579f0 con 0x7f322815bb50 2023-10-14T10:36:29.577 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.568+0000 7f322e17a700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f322806ed50 0x7f322815df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:29.577 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.568+0000 7f322e17a700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f322806ed50 0x7f322815df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44716/0 (socket says 172.21.15.67:44716) 2023-10-14T10:36:29.577 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.568+0000 7f322e17a700 1 -- 172.21.15.67:0/2255705743 learned_addr learned my addr 172.21.15.67:0/2255705743 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:29.578 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.568+0000 7f322d979700 1 --2- 172.21.15.67:0/2255705743 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f32280ce280 0x7f32281523e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:29.578 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.568+0000 7f322e97b700 1 --2- 172.21.15.67:0/2255705743 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f322815bb50 0x7f32281f65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:29.578 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.568+0000 7f322e17a700 1 -- 172.21.15.67:0/2255705743 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f322815bb50 msgr2=0x7f32281f65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:29.578 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.568+0000 7f322e17a700 1 --2- 172.21.15.67:0/2255705743 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f322815bb50 0x7f32281f65e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:29.578 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.568+0000 7f322e17a700 1 -- 172.21.15.67:0/2255705743 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f32280ce280 msgr2=0x7f32281523e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:29.579 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.568+0000 7f322e17a700 1 --2- 172.21.15.67:0/2255705743 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f32280ce280 0x7f32281523e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:29.579 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.568+0000 7f322e17a700 1 -- 172.21.15.67:0/2255705743 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32180094a0 con 0x7f322806ed50 2023-10-14T10:36:29.579 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.568+0000 7f322e17a700 1 --2- 172.21.15.67:0/2255705743 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f322806ed50 0x7f322815df90 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f32180090b0 tx=0x7f3218004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:29.579 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.569+0000 7f321effd700 1 -- 172.21.15.67:0/2255705743 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32180038a0 con 0x7f322806ed50 2023-10-14T10:36:29.579 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.569+0000 7f321effd700 1 -- 172.21.15.67:0/2255705743 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f321801e070 con 0x7f322806ed50 2023-10-14T10:36:29.580 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.569+0000 7f32303de700 1 -- 172.21.15.67:0/2255705743 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f322815cef0 con 0x7f322806ed50 2023-10-14T10:36:29.580 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.569+0000 7f321effd700 1 -- 172.21.15.67:0/2255705743 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3218003c40 con 0x7f322806ed50 2023-10-14T10:36:29.580 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.569+0000 7f32303de700 1 -- 172.21.15.67:0/2255705743 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f322815d4b0 con 0x7f322806ed50 2023-10-14T10:36:29.580 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.570+0000 7f32303de700 1 -- 172.21.15.67:0/2255705743 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3228067d20 con 0x7f322806ed50 2023-10-14T10:36:29.580 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.570+0000 7f321effd700 1 -- 172.21.15.67:0/2255705743 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f321802d450 con 0x7f322806ed50 2023-10-14T10:36:29.581 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.571+0000 7f321effd700 1 --2- 172.21.15.67:0/2255705743 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f320c06af00 0x7f320c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:29.581 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.571+0000 7f321effd700 1 -- 172.21.15.67:0/2255705743 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f321808a560 con 0x7f322806ed50 2023-10-14T10:36:29.581 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.571+0000 7f322d979700 1 --2- 172.21.15.67:0/2255705743 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f320c06af00 0x7f320c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:29.581 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.571+0000 7f322d979700 1 --2- 172.21.15.67:0/2255705743 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f320c06af00 0x7f320c06d3c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3228072270 tx=0x7f3224007d10 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:29.582 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.575+0000 7f321effd700 1 -- 172.21.15.67:0/2255705743 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f32180556a0 con 0x7f322806ed50 2023-10-14T10:36:29.780 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.778+0000 7f32303de700 1 -- 172.21.15.67:0/2255705743 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f32280668b0 con 0x7f322806ed50 2023-10-14T10:36:29.780 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.778+0000 7f321effd700 1 -- 172.21.15.67:0/2255705743 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3218058cc0 con 0x7f322806ed50 2023-10-14T10:36:29.781 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2023-10-14T10:36:29.783 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.780+0000 7f32303de700 1 -- 172.21.15.67:0/2255705743 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f320c06af00 msgr2=0x7f320c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:29.783 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.780+0000 7f32303de700 1 --2- 172.21.15.67:0/2255705743 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f320c06af00 0x7f320c06d3c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3228072270 tx=0x7f3224007d10 comp rx=0 tx=0).stop 2023-10-14T10:36:29.783 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.780+0000 7f32303de700 1 -- 172.21.15.67:0/2255705743 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f322806ed50 msgr2=0x7f322815df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:29.783 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.780+0000 7f32303de700 1 --2- 172.21.15.67:0/2255705743 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f322806ed50 0x7f322815df90 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f32180090b0 tx=0x7f3218004750 comp rx=0 tx=0).stop 2023-10-14T10:36:29.784 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.780+0000 7f32303de700 1 -- 172.21.15.67:0/2255705743 shutdown_connections 2023-10-14T10:36:29.784 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.780+0000 7f32303de700 1 --2- 172.21.15.67:0/2255705743 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f320c06af00 0x7f320c06d3c0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:29.784 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.780+0000 7f32303de700 1 --2- 172.21.15.67:0/2255705743 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f322815bb50 0x7f32281f65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:29.784 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.780+0000 7f32303de700 1 --2- 172.21.15.67:0/2255705743 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f322806ed50 0x7f322815df90 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:29.784 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.780+0000 7f32303de700 1 --2- 172.21.15.67:0/2255705743 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f32280ce280 0x7f32281523e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:29.785 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.780+0000 7f32303de700 1 -- 172.21.15.67:0/2255705743 >> 172.21.15.67:0/2255705743 conn(0x7f32280cc6f0 msgr2=0x7f322815b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:29.785 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.780+0000 7f32303de700 1 -- 172.21.15.67:0/2255705743 shutdown_connections 2023-10-14T10:36:29.785 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.780+0000 7f32303de700 1 -- 172.21.15.67:0/2255705743 wait complete. 2023-10-14T10:36:29.795 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-14T10:36:29.923 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.921+0000 7f3c9bdb4700 1 Processor -- start 2023-10-14T10:36:29.923 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.921+0000 7f3c9bdb4700 1 -- start start 2023-10-14T10:36:29.926 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.922+0000 7f3c9bdb4700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f3c940cd950 0x7f3c9415e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:29.926 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.922+0000 7f3c9bdb4700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f3c940cede0 0x7f3c9415f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:29.926 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.922+0000 7f3c9bdb4700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c940cf260 con 0x7f3c9406a840 2023-10-14T10:36:29.926 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.922+0000 7f3c9bdb4700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c941619a0 con 0x7f3c940cd950 2023-10-14T10:36:29.926 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.922+0000 7f3c9bdb4700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c940b2d60 con 0x7f3c940cede0 2023-10-14T10:36:29.927 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.922+0000 7f3c9a351700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f3c9406a840 0x7f3c94161df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:52496/0 (socket says 172.21.15.67:52496) 2023-10-14T10:36:29.927 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.922+0000 7f3c9a351700 1 -- 172.21.15.67:0/3820077046 learned_addr learned my addr 172.21.15.67:0/3820077046 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:36:29.927 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.922+0000 7f3c99b50700 1 --2- 172.21.15.67:0/3820077046 >> v2:172.21.15.72:3300/0 conn(0x7f3c940cd950 0x7f3c9415e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:29.927 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.922+0000 7f3c9934f700 1 --2- 172.21.15.67:0/3820077046 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f3c940cede0 0x7f3c9415f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:29.927 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.922+0000 7f3c98b4e700 1 -- 172.21.15.67:0/3820077046 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2931111021 0 0) 0x7f3c940cf260 con 0x7f3c9406a840 2023-10-14T10:36:29.928 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.922+0000 7f3c98b4e700 1 -- 172.21.15.67:0/3820077046 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c800035b0 con 0x7f3c9406a840 2023-10-14T10:36:29.928 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.922+0000 7f3c99b50700 1 -- 172.21.15.67:0/3820077046 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f3c940cede0 msgr2=0x7f3c9415f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:29.928 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.922+0000 7f3c99b50700 1 --2- 172.21.15.67:0/3820077046 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f3c940cede0 0x7f3c9415f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:29.928 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.922+0000 7f3c99b50700 1 -- 172.21.15.67:0/3820077046 >> v1:172.21.15.67:6789/0 conn(0x7f3c9406a840 legacy=0x7f3c94161df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:29.928 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.922+0000 7f3c99b50700 1 -- 172.21.15.67:0/3820077046 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c940cdd30 con 0x7f3c940cd950 2023-10-14T10:36:29.929 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.923+0000 7f3c99b50700 1 --2- 172.21.15.67:0/3820077046 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3c940cd950 0x7f3c9415e2f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3c90009840 tx=0x7f3c90009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87fc52056ceadd1f server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:29.929 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.923+0000 7f3c98b4e700 1 -- 172.21.15.67:0/3820077046 <== mon.1 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c90011830 con 0x7f3c940cd950 2023-10-14T10:36:29.929 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.923+0000 7f3c98b4e700 1 -- 172.21.15.67:0/3820077046 <== mon.1 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c90011990 con 0x7f3c940cd950 2023-10-14T10:36:29.929 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.923+0000 7f3c98b4e700 1 -- 172.21.15.67:0/3820077046 <== mon.1 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c90011d20 con 0x7f3c940cd950 2023-10-14T10:36:29.929 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.923+0000 7f3c9bdb4700 1 -- 172.21.15.67:0/3820077046 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3c940cd950 msgr2=0x7f3c9415e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:29.930 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.923+0000 7f3c9bdb4700 1 --2- 172.21.15.67:0/3820077046 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3c940cd950 0x7f3c9415e2f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3c90009840 tx=0x7f3c90009b50 comp rx=0 tx=0).stop 2023-10-14T10:36:29.930 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.923+0000 7f3c9bdb4700 1 -- 172.21.15.67:0/3820077046 shutdown_connections 2023-10-14T10:36:29.930 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.923+0000 7f3c9bdb4700 1 --2- 172.21.15.67:0/3820077046 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3c940cd950 0x7f3c9415e2f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:29.930 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.923+0000 7f3c9bdb4700 1 --2- 172.21.15.67:0/3820077046 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f3c940cede0 0x7f3c9415f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:29.930 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.923+0000 7f3c9bdb4700 1 -- 172.21.15.67:0/3820077046 >> 172.21.15.67:0/3820077046 conn(0x7f3c9406dae0 msgr2=0x7f3c9406def0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:29.931 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.924+0000 7f3c9bdb4700 1 -- 172.21.15.67:0/3820077046 shutdown_connections 2023-10-14T10:36:29.931 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.924+0000 7f3c9bdb4700 1 -- 172.21.15.67:0/3820077046 wait complete. 2023-10-14T10:36:29.933 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.925+0000 7f3c9bdb4700 1 Processor -- start 2023-10-14T10:36:29.934 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.925+0000 7f3c9bdb4700 1 -- start start 2023-10-14T10:36:29.934 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.925+0000 7f3c9bdb4700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3c9406a840 0x7f3c940b40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:29.934 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.926+0000 7f3c9bdb4700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c940cd950 0x7f3c940af010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:29.934 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.926+0000 7f3c9bdb4700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f3c940cede0 0x7f3c940b5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:29.935 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.926+0000 7f3c9bdb4700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c940af550 con 0x7f3c940cd950 2023-10-14T10:36:29.935 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.926+0000 7f3c9bdb4700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c940b46c0 con 0x7f3c9406a840 2023-10-14T10:36:29.935 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.926+0000 7f3c9bdb4700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c940b49a0 con 0x7f3c940cede0 2023-10-14T10:36:29.935 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.926+0000 7f3c9934f700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c940cd950 0x7f3c940af010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:29.935 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.926+0000 7f3c9934f700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c940cd950 0x7f3c940af010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44732/0 (socket says 172.21.15.67:44732) 2023-10-14T10:36:29.936 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.926+0000 7f3c9934f700 1 -- 172.21.15.67:0/348925159 learned_addr learned my addr 172.21.15.67:0/348925159 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:29.936 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.926+0000 7f3c9a351700 1 --2- 172.21.15.67:0/348925159 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f3c940cede0 0x7f3c940b5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:29.936 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.926+0000 7f3c99b50700 1 --2- 172.21.15.67:0/348925159 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3c9406a840 0x7f3c940b40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:29.936 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.926+0000 7f3c9934f700 1 -- 172.21.15.67:0/348925159 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f3c940cede0 msgr2=0x7f3c940b5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:29.936 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.926+0000 7f3c9934f700 1 --2- 172.21.15.67:0/348925159 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f3c940cede0 0x7f3c940b5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:29.937 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.926+0000 7f3c9934f700 1 -- 172.21.15.67:0/348925159 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3c9406a840 msgr2=0x7f3c940b40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:29.937 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.926+0000 7f3c9934f700 1 --2- 172.21.15.67:0/348925159 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3c9406a840 0x7f3c940b40c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:29.937 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.926+0000 7f3c9934f700 1 -- 172.21.15.67:0/348925159 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c900094a0 con 0x7f3c940cd950 2023-10-14T10:36:29.937 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.926+0000 7f3c9934f700 1 --2- 172.21.15.67:0/348925159 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c940cd950 0x7f3c940af010 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3c8400e900 tx=0x7f3c8400ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:29.937 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.926+0000 7f3c8a7fc700 1 -- 172.21.15.67:0/348925159 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c84013680 con 0x7f3c940cd950 2023-10-14T10:36:29.938 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.926+0000 7f3c8a7fc700 1 -- 172.21.15.67:0/348925159 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c840137e0 con 0x7f3c940cd950 2023-10-14T10:36:29.938 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.926+0000 7f3c9bdb4700 1 -- 172.21.15.67:0/348925159 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c940b4c80 con 0x7f3c940cd950 2023-10-14T10:36:29.938 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.926+0000 7f3c8a7fc700 1 -- 172.21.15.67:0/348925159 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c84013b50 con 0x7f3c940cd950 2023-10-14T10:36:29.938 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.926+0000 7f3c9bdb4700 1 -- 172.21.15.67:0/348925159 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c940b5240 con 0x7f3c940cd950 2023-10-14T10:36:29.938 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.927+0000 7f3c9bdb4700 1 -- 172.21.15.67:0/348925159 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c940b5460 con 0x7f3c940cd950 2023-10-14T10:36:29.939 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.928+0000 7f3c8a7fc700 1 -- 172.21.15.67:0/348925159 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f3c84013cb0 con 0x7f3c940cd950 2023-10-14T10:36:29.939 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.928+0000 7f3c8a7fc700 1 --2- 172.21.15.67:0/348925159 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f3c7806af00 0x7f3c7806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:29.939 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.928+0000 7f3c8a7fc700 1 -- 172.21.15.67:0/348925159 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3c84089db0 con 0x7f3c940cd950 2023-10-14T10:36:29.939 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.929+0000 7f3c99b50700 1 --2- 172.21.15.67:0/348925159 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f3c7806af00 0x7f3c7806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:29.939 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.929+0000 7f3c99b50700 1 --2- 172.21.15.67:0/348925159 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f3c7806af00 0x7f3c7806d3c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3c90000c00 tx=0x7f3c90008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:29.940 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:29.932+0000 7f3c8a7fc700 1 -- 172.21.15.67:0/348925159 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3c84054e40 con 0x7f3c940cd950 2023-10-14T10:36:30.135 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.133+0000 7f3c9bdb4700 1 -- 172.21.15.67:0/348925159 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3c94067d20 con 0x7f3c940cd950 2023-10-14T10:36:30.136 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.134+0000 7f3c8a7fc700 1 -- 172.21.15.67:0/348925159 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3c840210a0 con 0x7f3c940cd950 2023-10-14T10:36:30.136 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2023-10-14T10:36:30.138 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.136+0000 7f3c9bdb4700 1 -- 172.21.15.67:0/348925159 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f3c7806af00 msgr2=0x7f3c7806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:30.139 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.136+0000 7f3c9bdb4700 1 --2- 172.21.15.67:0/348925159 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f3c7806af00 0x7f3c7806d3c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3c90000c00 tx=0x7f3c90008030 comp rx=0 tx=0).stop 2023-10-14T10:36:30.139 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.136+0000 7f3c9bdb4700 1 -- 172.21.15.67:0/348925159 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c940cd950 msgr2=0x7f3c940af010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:30.139 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.136+0000 7f3c9bdb4700 1 --2- 172.21.15.67:0/348925159 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c940cd950 0x7f3c940af010 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3c8400e900 tx=0x7f3c8400ec10 comp rx=0 tx=0).stop 2023-10-14T10:36:30.139 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.136+0000 7f3c9bdb4700 1 -- 172.21.15.67:0/348925159 shutdown_connections 2023-10-14T10:36:30.139 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.136+0000 7f3c9bdb4700 1 --2- 172.21.15.67:0/348925159 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f3c7806af00 0x7f3c7806d3c0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:30.140 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.136+0000 7f3c9bdb4700 1 --2- 172.21.15.67:0/348925159 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f3c940cede0 0x7f3c940b5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:30.140 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.136+0000 7f3c9bdb4700 1 --2- 172.21.15.67:0/348925159 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3c9406a840 0x7f3c940b40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:30.140 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.136+0000 7f3c9bdb4700 1 --2- 172.21.15.67:0/348925159 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c940cd950 0x7f3c940af010 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:30.140 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.136+0000 7f3c9bdb4700 1 -- 172.21.15.67:0/348925159 >> 172.21.15.67:0/348925159 conn(0x7f3c9406dae0 msgr2=0x7f3c940aff00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:30.140 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.136+0000 7f3c9bdb4700 1 -- 172.21.15.67:0/348925159 shutdown_connections 2023-10-14T10:36:30.141 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.136+0000 7f3c9bdb4700 1 -- 172.21.15.67:0/348925159 wait complete. 2023-10-14T10:36:30.150 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-14T10:36:30.279 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.276+0000 7f0c8cdd5700 1 Processor -- start 2023-10-14T10:36:30.279 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.276+0000 7f0c8cdd5700 1 -- start start 2023-10-14T10:36:30.281 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.277+0000 7f0c8cdd5700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f0c880d0c70 0x7f0c880cdfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:30.282 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.277+0000 7f0c8cdd5700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f0c880c5110 0x7f0c880ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:30.282 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.277+0000 7f0c8cdd5700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c880d23e0 con 0x7f0c8815b740 2023-10-14T10:36:30.282 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.277+0000 7f0c8cdd5700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c8815bb20 con 0x7f0c880d0c70 2023-10-14T10:36:30.282 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.277+0000 7f0c8cdd5700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c880cf190 con 0x7f0c880c5110 2023-10-14T10:36:30.283 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.278+0000 7f0c85d9b700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f0c880d0c70 0x7f0c880cdfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:30.283 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.278+0000 7f0c85d9b700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f0c880d0c70 0x7f0c880cdfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44738/0 (socket says 172.21.15.67:44738) 2023-10-14T10:36:30.283 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.278+0000 7f0c85d9b700 1 -- 172.21.15.67:0/209872938 learned_addr learned my addr 172.21.15.67:0/209872938 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:30.283 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.278+0000 7f0c86d9d700 1 --2- 172.21.15.67:0/209872938 >> v2:172.21.15.72:3300/0 conn(0x7f0c880c5110 0x7f0c880ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:30.284 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.278+0000 7f0c85d9b700 1 -- 172.21.15.67:0/209872938 >> v2:172.21.15.72:3300/0 conn(0x7f0c880c5110 msgr2=0x7f0c880ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:30.284 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.278+0000 7f0c85d9b700 1 --2- 172.21.15.67:0/209872938 >> v2:172.21.15.72:3300/0 conn(0x7f0c880c5110 0x7f0c880ce510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:30.284 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.278+0000 7f0c85d9b700 1 -- 172.21.15.67:0/209872938 >> v1:172.21.15.72:6789/0 conn(0x7f0c8815b740 legacy=0x7f0c880d1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:30.284 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.278+0000 7f0c85d9b700 1 -- 172.21.15.67:0/209872938 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c880cbd40 con 0x7f0c880d0c70 2023-10-14T10:36:30.284 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.278+0000 7f0c86d9d700 1 --2- 172.21.15.67:0/209872938 >> v2:172.21.15.72:3300/0 conn(0x7f0c880c5110 0x7f0c880ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-14T10:36:30.285 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.278+0000 7f0c85d9b700 1 --2- 172.21.15.67:0/209872938 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c880d0c70 0x7f0c880cdfd0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0c880c8620 tx=0x7f0c70009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db64a55aebb8ced6 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:30.285 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.278+0000 7f0c8559a700 1 -- 172.21.15.67:0/209872938 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c70004230 con 0x7f0c880d0c70 2023-10-14T10:36:30.285 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.278+0000 7f0c8559a700 1 -- 172.21.15.67:0/209872938 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c70004390 con 0x7f0c880d0c70 2023-10-14T10:36:30.285 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.278+0000 7f0c8559a700 1 -- 172.21.15.67:0/209872938 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c700036a0 con 0x7f0c880d0c70 2023-10-14T10:36:30.286 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.279+0000 7f0c8cdd5700 1 -- 172.21.15.67:0/209872938 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c880d0c70 msgr2=0x7f0c880cdfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:30.286 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.279+0000 7f0c8cdd5700 1 --2- 172.21.15.67:0/209872938 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c880d0c70 0x7f0c880cdfd0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0c880c8620 tx=0x7f0c70009bd0 comp rx=0 tx=0).stop 2023-10-14T10:36:30.286 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.279+0000 7f0c8cdd5700 1 -- 172.21.15.67:0/209872938 shutdown_connections 2023-10-14T10:36:30.286 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.279+0000 7f0c8cdd5700 1 --2- 172.21.15.67:0/209872938 >> v2:172.21.15.72:3300/0 conn(0x7f0c880c5110 0x7f0c880ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:30.286 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.279+0000 7f0c8cdd5700 1 --2- 172.21.15.67:0/209872938 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c880d0c70 0x7f0c880cdfd0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:30.287 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.279+0000 7f0c8cdd5700 1 -- 172.21.15.67:0/209872938 >> 172.21.15.67:0/209872938 conn(0x7f0c880665a0 msgr2=0x7f0c88066980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:30.287 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.279+0000 7f0c8cdd5700 1 -- 172.21.15.67:0/209872938 shutdown_connections 2023-10-14T10:36:30.287 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.279+0000 7f0c8cdd5700 1 -- 172.21.15.67:0/209872938 wait complete. 2023-10-14T10:36:30.290 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.280+0000 7f0c8cdd5700 1 Processor -- start 2023-10-14T10:36:30.290 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.280+0000 7f0c8cdd5700 1 -- start start 2023-10-14T10:36:30.290 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.281+0000 7f0c8cdd5700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f0c880c5110 0x7f0c880b6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:30.291 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.281+0000 7f0c8cdd5700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0c880d0c70 0x7f0c881f2130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:30.291 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.281+0000 7f0c8cdd5700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c8815b740 0x7f0c881f2670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:30.291 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.281+0000 7f0c8cdd5700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c881563e0 con 0x7f0c8815b740 2023-10-14T10:36:30.291 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.281+0000 7f0c8cdd5700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c880b4b90 con 0x7f0c880d0c70 2023-10-14T10:36:30.291 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.281+0000 7f0c8cdd5700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c880b3f80 con 0x7f0c880c5110 2023-10-14T10:36:30.292 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.281+0000 7f0c86d9d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c8815b740 0x7f0c881f2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:30.292 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.281+0000 7f0c86d9d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c8815b740 0x7f0c881f2670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44748/0 (socket says 172.21.15.67:44748) 2023-10-14T10:36:30.292 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.281+0000 7f0c86d9d700 1 -- 172.21.15.67:0/771019055 learned_addr learned my addr 172.21.15.67:0/771019055 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:30.292 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.281+0000 7f0c8659c700 1 --2- 172.21.15.67:0/771019055 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f0c880c5110 0x7f0c880b6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:30.292 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.281+0000 7f0c85d9b700 1 --2- 172.21.15.67:0/771019055 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0c880d0c70 0x7f0c881f2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:30.293 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.281+0000 7f0c86d9d700 1 -- 172.21.15.67:0/771019055 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f0c880c5110 msgr2=0x7f0c880b6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:30.293 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.281+0000 7f0c86d9d700 1 --2- 172.21.15.67:0/771019055 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f0c880c5110 0x7f0c880b6920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:30.293 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.281+0000 7f0c86d9d700 1 -- 172.21.15.67:0/771019055 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0c880d0c70 msgr2=0x7f0c881f2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:30.293 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.281+0000 7f0c86d9d700 1 --2- 172.21.15.67:0/771019055 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0c880d0c70 0x7f0c881f2130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:30.293 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.281+0000 7f0c86d9d700 1 -- 172.21.15.67:0/771019055 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c70009550 con 0x7f0c8815b740 2023-10-14T10:36:30.294 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.282+0000 7f0c86d9d700 1 --2- 172.21.15.67:0/771019055 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c8815b740 0x7f0c881f2670 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0c8000cef0 tx=0x7f0c8000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:30.294 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.282+0000 7f0c7effd700 1 -- 172.21.15.67:0/771019055 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c8000ae50 con 0x7f0c8815b740 2023-10-14T10:36:30.294 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.282+0000 7f0c7effd700 1 -- 172.21.15.67:0/771019055 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c80015440 con 0x7f0c8815b740 2023-10-14T10:36:30.294 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.282+0000 7f0c8cdd5700 1 -- 172.21.15.67:0/771019055 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c880b4260 con 0x7f0c8815b740 2023-10-14T10:36:30.295 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.282+0000 7f0c7effd700 1 -- 172.21.15.67:0/771019055 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c800157f0 con 0x7f0c8815b740 2023-10-14T10:36:30.295 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.282+0000 7f0c8cdd5700 1 -- 172.21.15.67:0/771019055 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c880b4820 con 0x7f0c8815b740 2023-10-14T10:36:30.295 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.283+0000 7f0c8cdd5700 1 -- 172.21.15.67:0/771019055 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c880cc5c0 con 0x7f0c8815b740 2023-10-14T10:36:30.295 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.283+0000 7f0c7effd700 1 -- 172.21.15.67:0/771019055 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f0c8000d430 con 0x7f0c8815b740 2023-10-14T10:36:30.295 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.284+0000 7f0c7effd700 1 --2- 172.21.15.67:0/771019055 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f0c6806af00 0x7f0c6806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:30.296 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.284+0000 7f0c7effd700 1 -- 172.21.15.67:0/771019055 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0c80089c10 con 0x7f0c8815b740 2023-10-14T10:36:30.296 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.284+0000 7f0c8659c700 1 --2- 172.21.15.67:0/771019055 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f0c6806af00 0x7f0c6806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:30.296 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.284+0000 7f0c8659c700 1 --2- 172.21.15.67:0/771019055 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f0c6806af00 0x7f0c6806d3c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0c7800a7d0 tx=0x7f0c78008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:30.296 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.288+0000 7f0c7effd700 1 -- 172.21.15.67:0/771019055 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0c880cc5c0 con 0x7f0c8815b740 2023-10-14T10:36:30.490 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.488+0000 7f0c8cdd5700 1 -- 172.21.15.67:0/771019055 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0c880cc5c0 con 0x7f0c8815b740 2023-10-14T10:36:30.491 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.488+0000 7f0c7effd700 1 -- 172.21.15.67:0/771019055 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0c880cc5c0 con 0x7f0c8815b740 2023-10-14T10:36:30.491 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2023-10-14T10:36:30.493 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.490+0000 7f0c8cdd5700 1 -- 172.21.15.67:0/771019055 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f0c6806af00 msgr2=0x7f0c6806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:30.493 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.490+0000 7f0c8cdd5700 1 --2- 172.21.15.67:0/771019055 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f0c6806af00 0x7f0c6806d3c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0c7800a7d0 tx=0x7f0c78008040 comp rx=0 tx=0).stop 2023-10-14T10:36:30.493 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.490+0000 7f0c8cdd5700 1 -- 172.21.15.67:0/771019055 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c8815b740 msgr2=0x7f0c881f2670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:30.493 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.490+0000 7f0c8cdd5700 1 --2- 172.21.15.67:0/771019055 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c8815b740 0x7f0c881f2670 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0c8000cef0 tx=0x7f0c8000a340 comp rx=0 tx=0).stop 2023-10-14T10:36:30.493 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.490+0000 7f0c8cdd5700 1 -- 172.21.15.67:0/771019055 shutdown_connections 2023-10-14T10:36:30.494 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.490+0000 7f0c8cdd5700 1 --2- 172.21.15.67:0/771019055 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f0c6806af00 0x7f0c6806d3c0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:30.494 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.490+0000 7f0c8cdd5700 1 --2- 172.21.15.67:0/771019055 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c8815b740 0x7f0c881f2670 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:30.494 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.490+0000 7f0c8cdd5700 1 --2- 172.21.15.67:0/771019055 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f0c880c5110 0x7f0c880b6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:30.494 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.490+0000 7f0c8cdd5700 1 --2- 172.21.15.67:0/771019055 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0c880d0c70 0x7f0c881f2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:30.494 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.490+0000 7f0c8cdd5700 1 -- 172.21.15.67:0/771019055 >> 172.21.15.67:0/771019055 conn(0x7f0c880665a0 msgr2=0x7f0c88159d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:30.495 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.490+0000 7f0c8cdd5700 1 -- 172.21.15.67:0/771019055 shutdown_connections 2023-10-14T10:36:30.495 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.490+0000 7f0c8cdd5700 1 -- 172.21.15.67:0/771019055 wait complete. 2023-10-14T10:36:30.505 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-14T10:36:30.629 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.627+0000 7fac17cd2700 1 Processor -- start 2023-10-14T10:36:30.630 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.627+0000 7fac17cd2700 1 -- start start 2023-10-14T10:36:30.632 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.628+0000 7fac17cd2700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fac1006e750 0x7fac100ceec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:30.632 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.628+0000 7fac17cd2700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fac1006eb30 0x7fac100cf400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:30.632 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.628+0000 7fac17cd2700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac100b5c70 con 0x7fac100b54f0 2023-10-14T10:36:30.633 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.628+0000 7fac17cd2700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac100b5990 con 0x7fac1006e750 2023-10-14T10:36:30.633 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.628+0000 7fac17cd2700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac1015b2b0 con 0x7fac1006eb30 2023-10-14T10:36:30.633 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.628+0000 7fac15a6e700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fac1006e750 0x7fac100ceec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:30.633 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.629+0000 7fac1526d700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fac1006eb30 0x7fac100cf400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:30.633 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.629+0000 7fac1626f700 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7fac100b54f0 0x7fac1015e700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.67:60764/0 (socket says 172.21.15.67:60764) 2023-10-14T10:36:30.634 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.629+0000 7fac15a6e700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fac1006e750 0x7fac100ceec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.67:39542/0 (socket says 172.21.15.67:39542) 2023-10-14T10:36:30.634 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.629+0000 7fac1626f700 1 -- 172.21.15.67:0/3723439047 learned_addr learned my addr 172.21.15.67:0/3723439047 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:36:30.634 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.629+0000 7fac15a6e700 1 -- 172.21.15.67:0/3723439047 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fac1006eb30 msgr2=0x7fac100cf400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:30.634 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.629+0000 7fac15a6e700 1 --2- 172.21.15.67:0/3723439047 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fac1006eb30 0x7fac100cf400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:30.635 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.629+0000 7fac15a6e700 1 -- 172.21.15.67:0/3723439047 >> v1:172.21.15.72:6789/0 conn(0x7fac100b54f0 legacy=0x7fac1015e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:30.635 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.629+0000 7fac15a6e700 1 -- 172.21.15.67:0/3723439047 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac1015b590 con 0x7fac1006e750 2023-10-14T10:36:30.635 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.629+0000 7fac14a6c700 1 -- 172.21.15.67:0/3723439047 <== mon.1 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1904168322 0 0) 0x7fac100b5c70 con 0x7fac100b54f0 2023-10-14T10:36:30.635 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.629+0000 7fac15a6e700 1 --2- 172.21.15.67:0/3723439047 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fac1006e750 0x7fac100ceec0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fac00009840 tx=0x7fac00009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34287b9bcaf547e5 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:30.635 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.629+0000 7fac14a6c700 1 -- 172.21.15.67:0/3723439047 <== mon.1 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac00011830 con 0x7fac1006e750 2023-10-14T10:36:30.636 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.629+0000 7fac14a6c700 1 -- 172.21.15.67:0/3723439047 <== mon.1 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac00011990 con 0x7fac1006e750 2023-10-14T10:36:30.636 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.629+0000 7fac14a6c700 1 -- 172.21.15.67:0/3723439047 <== mon.1 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac00011d20 con 0x7fac1006e750 2023-10-14T10:36:30.636 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.630+0000 7fac17cd2700 1 -- 172.21.15.67:0/3723439047 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fac1006e750 msgr2=0x7fac100ceec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:30.636 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.630+0000 7fac17cd2700 1 --2- 172.21.15.67:0/3723439047 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fac1006e750 0x7fac100ceec0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fac00009840 tx=0x7fac00009b50 comp rx=0 tx=0).stop 2023-10-14T10:36:30.636 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.630+0000 7fac17cd2700 1 -- 172.21.15.67:0/3723439047 shutdown_connections 2023-10-14T10:36:30.637 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.630+0000 7fac17cd2700 1 --2- 172.21.15.67:0/3723439047 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fac1006e750 0x7fac100ceec0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:30.637 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.630+0000 7fac17cd2700 1 --2- 172.21.15.67:0/3723439047 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fac1006eb30 0x7fac100cf400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:30.637 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.630+0000 7fac17cd2700 1 -- 172.21.15.67:0/3723439047 >> 172.21.15.67:0/3723439047 conn(0x7fac100cc6f0 msgr2=0x7fac100ccb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:30.637 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.630+0000 7fac17cd2700 1 -- 172.21.15.67:0/3723439047 shutdown_connections 2023-10-14T10:36:30.637 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.630+0000 7fac17cd2700 1 -- 172.21.15.67:0/3723439047 wait complete. 2023-10-14T10:36:30.640 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.631+0000 7fac17cd2700 1 Processor -- start 2023-10-14T10:36:30.640 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.631+0000 7fac17cd2700 1 -- start start 2023-10-14T10:36:30.641 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.632+0000 7fac17cd2700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fac1006e750 0x7fac1014edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:30.641 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.632+0000 7fac17cd2700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fac1006eb30 0x7fac100c63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:30.641 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.632+0000 7fac17cd2700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fac100b54f0 0x7fac10158d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:30.641 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.632+0000 7fac17cd2700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac100c6d40 con 0x7fac1006eb30 2023-10-14T10:36:30.641 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.632+0000 7fac17cd2700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac1015a950 con 0x7fac100b54f0 2023-10-14T10:36:30.642 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.632+0000 7fac17cd2700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac10153e20 con 0x7fac1006e750 2023-10-14T10:36:30.642 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.632+0000 7fac1526d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fac1006eb30 0x7fac100c63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:30.642 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.632+0000 7fac1526d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fac1006eb30 0x7fac100c63e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44762/0 (socket says 172.21.15.67:44762) 2023-10-14T10:36:30.642 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.632+0000 7fac1526d700 1 -- 172.21.15.67:0/2729004669 learned_addr learned my addr 172.21.15.67:0/2729004669 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:30.643 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.632+0000 7fac1626f700 1 --2- 172.21.15.67:0/2729004669 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fac100b54f0 0x7fac10158d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:30.643 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.632+0000 7fac15a6e700 1 --2- 172.21.15.67:0/2729004669 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fac1006e750 0x7fac1014edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:30.643 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.632+0000 7fac1526d700 1 -- 172.21.15.67:0/2729004669 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fac1006e750 msgr2=0x7fac1014edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:30.643 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.632+0000 7fac1526d700 1 --2- 172.21.15.67:0/2729004669 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fac1006e750 0x7fac1014edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:30.643 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.632+0000 7fac1526d700 1 -- 172.21.15.67:0/2729004669 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fac100b54f0 msgr2=0x7fac10158d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:30.644 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.632+0000 7fac1526d700 1 --2- 172.21.15.67:0/2729004669 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fac100b54f0 0x7fac10158d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:30.644 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.632+0000 7fac1526d700 1 -- 172.21.15.67:0/2729004669 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac000094a0 con 0x7fac1006eb30 2023-10-14T10:36:30.644 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.633+0000 7fac1526d700 1 --2- 172.21.15.67:0/2729004669 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fac1006eb30 0x7fac100c63e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fac0c00e980 tx=0x7fac0c00ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:30.644 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.633+0000 7fac067fc700 1 -- 172.21.15.67:0/2729004669 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac0c009710 con 0x7fac1006eb30 2023-10-14T10:36:30.644 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.633+0000 7fac067fc700 1 -- 172.21.15.67:0/2729004669 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac0c00c690 con 0x7fac1006eb30 2023-10-14T10:36:30.645 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.633+0000 7fac17cd2700 1 -- 172.21.15.67:0/2729004669 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac10154100 con 0x7fac1006eb30 2023-10-14T10:36:30.645 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.633+0000 7fac067fc700 1 -- 172.21.15.67:0/2729004669 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac0c00ca40 con 0x7fac1006eb30 2023-10-14T10:36:30.645 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.633+0000 7fac17cd2700 1 -- 172.21.15.67:0/2729004669 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac101546c0 con 0x7fac1006eb30 2023-10-14T10:36:30.645 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.634+0000 7fac17cd2700 1 -- 172.21.15.67:0/2729004669 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac100c6eb0 con 0x7fac1006eb30 2023-10-14T10:36:30.645 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.634+0000 7fac067fc700 1 -- 172.21.15.67:0/2729004669 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fac0c00cba0 con 0x7fac1006eb30 2023-10-14T10:36:30.645 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.635+0000 7fac067fc700 1 --2- 172.21.15.67:0/2729004669 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fabf406af00 0x7fabf406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:30.646 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.635+0000 7fac067fc700 1 -- 172.21.15.67:0/2729004669 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fac0c089660 con 0x7fac1006eb30 2023-10-14T10:36:30.646 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.635+0000 7fac15a6e700 1 --2- 172.21.15.67:0/2729004669 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fabf406af00 0x7fabf406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:30.646 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.635+0000 7fac15a6e700 1 --2- 172.21.15.67:0/2729004669 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fabf406af00 0x7fabf406d3c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fac00009fd0 tx=0x7fac00008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:30.647 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.639+0000 7fac067fc700 1 -- 172.21.15.67:0/2729004669 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fac100c6eb0 con 0x7fac1006eb30 2023-10-14T10:36:30.845 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.843+0000 7fac17cd2700 1 -- 172.21.15.67:0/2729004669 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fac100c6e80 con 0x7fac1006eb30 2023-10-14T10:36:30.846 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.844+0000 7fac067fc700 1 -- 172.21.15.67:0/2729004669 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16457 (secure 0 0 0) 0x7fac0c054670 con 0x7fac1006eb30 2023-10-14T10:36:30.846 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-14T10:36:30.847 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":22,"fsid":"37f9ab46-3082-4b71-8b24-08d39897b008","created":"2023-10-14T10:35:46.067554+0000","modified":"2023-10-14T10:36:04.837296+0000","last_up_change":"2023-10-14T10:35:56.668142+0000","last_in_change":"2023-10-14T10:35:48.551490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-14T10:35:57.809566+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-14T10:36:01.667146+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-14T10:36:02.219700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2b59a07e-11ef-440c-b766-9f64ac885838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6801","nonce":1287255499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6803","nonce":1287255499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6807","nonce":1287255499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6805","nonce":1287255499}]},"public_addr":"172.21.15.67:6801/1287255499","cluster_addr":"172.21.15.67:6803/1287255499","heartbeat_back_addr":"172.21.15.67:6807/1287255499","heartbeat_front_addr":"172.21.15.67:6805/1287255499","state":["exists","up"]},{"osd":1,"uuid":"6f813bd6-fa0b-487d-a204-431eb1276a60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6818","nonce":2435538787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6822","nonce":2435538787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6830","nonce":2435538787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6826","nonce":2435538787}]},"public_addr":"172.21.15.67:6818/2435538787","cluster_addr":"172.21.15.67:6822/2435538787","heartbeat_back_addr":"172.21.15.67:6830/2435538787","heartbeat_front_addr":"172.21.15.67:6826/2435538787","state":["exists","up"]},{"osd":2,"uuid":"6af42c6a-1b59-4669-9c04-b6876788446d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6809","nonce":2827491791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6811","nonce":2827491791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6815","nonce":2827491791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6813","nonce":2827491791}]},"public_addr":"172.21.15.67:6809/2827491791","cluster_addr":"172.21.15.67:6811/2827491791","heartbeat_back_addr":"172.21.15.67:6815/2827491791","heartbeat_front_addr":"172.21.15.67:6813/2827491791","state":["exists","up"]},{"osd":3,"uuid":"1d889d76-a646-499d-bfff-ed5bae5434ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6819","nonce":1848001176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6821","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6823","nonce":1848001176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6831","nonce":1848001176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6825","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6827","nonce":1848001176}]},"public_addr":"172.21.15.67:6819/1848001176","cluster_addr":"172.21.15.67:6823/1848001176","heartbeat_back_addr":"172.21.15.67:6831/1848001176","heartbeat_front_addr":"172.21.15.67:6827/1848001176","state":["exists","up"]},{"osd":4,"uuid":"c3271dff-b437-4fdd-b13b-6e49a35473af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6804","nonce":1891230785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6808","nonce":1891230785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6815","nonce":1891230785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6811","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6813","nonce":1891230785}]},"public_addr":"172.21.15.72:6804/1891230785","cluster_addr":"172.21.15.72:6808/1891230785","heartbeat_back_addr":"172.21.15.72:6815/1891230785","heartbeat_front_addr":"172.21.15.72:6813/1891230785","state":["exists","up"]},{"osd":5,"uuid":"0337bb18-4b68-4eff-b528-a667275e6926","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6801","nonce":569686991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6803","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6805","nonce":569686991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6812","nonce":569686991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6807","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6809","nonce":569686991}]},"public_addr":"172.21.15.72:6801/569686991","cluster_addr":"172.21.15.72:6805/569686991","heartbeat_back_addr":"172.21.15.72:6812/569686991","heartbeat_front_addr":"172.21.15.72:6809/569686991","state":["exists","up"]},{"osd":6,"uuid":"8e92d9cf-0c8c-46eb-89a7-db2023ef1509","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6817","nonce":3767685934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6819","nonce":3767685934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6823","nonce":3767685934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6821","nonce":3767685934}]},"public_addr":"172.21.15.72:6817/3767685934","cluster_addr":"172.21.15.72:6819/3767685934","heartbeat_back_addr":"172.21.15.72:6823/3767685934","heartbeat_front_addr":"172.21.15.72:6821/3767685934","state":["exists","up"]},{"osd":7,"uuid":"3fd6aa7d-50a9-45c9-9561-5902a044a119","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6825","nonce":3567465926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6827","nonce":3567465926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6831","nonce":3567465926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6829","nonce":3567465926}]},"public_addr":"172.21.15.72:6825/3567465926","cluster_addr":"172.21.15.72:6827/3567465926","heartbeat_back_addr":"172.21.15.72:6831/3567465926","heartbeat_front_addr":"172.21.15.72:6829/3567465926","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.511466+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.795107+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.858674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.728895+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.794842+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.750064+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.732122+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.765185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-14T10:36:30.848 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.846+0000 7fac17cd2700 1 -- 172.21.15.67:0/2729004669 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fabf406af00 msgr2=0x7fabf406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:30.848 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.846+0000 7fac17cd2700 1 --2- 172.21.15.67:0/2729004669 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fabf406af00 0x7fabf406d3c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fac00009fd0 tx=0x7fac00008030 comp rx=0 tx=0).stop 2023-10-14T10:36:30.849 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.846+0000 7fac17cd2700 1 -- 172.21.15.67:0/2729004669 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fac1006eb30 msgr2=0x7fac100c63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:30.849 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.846+0000 7fac17cd2700 1 --2- 172.21.15.67:0/2729004669 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fac1006eb30 0x7fac100c63e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fac0c00e980 tx=0x7fac0c00ec90 comp rx=0 tx=0).stop 2023-10-14T10:36:30.849 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.846+0000 7fac17cd2700 1 -- 172.21.15.67:0/2729004669 shutdown_connections 2023-10-14T10:36:30.849 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.846+0000 7fac17cd2700 1 --2- 172.21.15.67:0/2729004669 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fabf406af00 0x7fabf406d3c0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:30.849 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.846+0000 7fac17cd2700 1 --2- 172.21.15.67:0/2729004669 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fac100b54f0 0x7fac10158d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:30.850 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.846+0000 7fac17cd2700 1 --2- 172.21.15.67:0/2729004669 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fac1006e750 0x7fac1014edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:30.850 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.846+0000 7fac17cd2700 1 --2- 172.21.15.67:0/2729004669 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fac1006eb30 0x7fac100c63e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:30.850 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.846+0000 7fac17cd2700 1 -- 172.21.15.67:0/2729004669 >> 172.21.15.67:0/2729004669 conn(0x7fac100cc6f0 msgr2=0x7fac100ceec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:30.850 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.846+0000 7fac17cd2700 1 -- 172.21.15.67:0/2729004669 shutdown_connections 2023-10-14T10:36:30.851 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.846+0000 7fac17cd2700 1 -- 172.21.15.67:0/2729004669 wait complete. 2023-10-14T10:36:30.860 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-14T10:35:57.809566+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-14T10:36:01.667146+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-14T10:36:02.219700+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-14T10:36:30.861 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-14T10:36:30.989 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.987+0000 7f5732ba6700 1 Processor -- start 2023-10-14T10:36:30.990 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.987+0000 7f5732ba6700 1 -- start start 2023-10-14T10:36:30.992 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.988+0000 7f5732ba6700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f572c15e030 0x7f572c0cb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:30.992 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.988+0000 7f5732ba6700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f572c0ce280 0x7f572c06d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:30.992 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.988+0000 7f5732ba6700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f572c06adb0 con 0x7f572c06a840 2023-10-14T10:36:30.993 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.988+0000 7f5732ba6700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f572c06b090 con 0x7f572c0ce280 2023-10-14T10:36:30.993 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.988+0000 7f5732ba6700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f572c06b370 con 0x7f572c15e030 2023-10-14T10:36:30.993 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.988+0000 7f5731143700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f572c0ce280 0x7f572c06d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:30.993 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.988+0000 7f572bfff700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f572c06a840 0x7f572c0cdb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:52534/0 (socket says 172.21.15.67:52534) 2023-10-14T10:36:30.993 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.988+0000 7f572bfff700 1 -- 172.21.15.67:0/148189688 learned_addr learned my addr 172.21.15.67:0/148189688 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:36:30.994 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.989+0000 7f5730942700 1 --2- 172.21.15.67:0/148189688 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f572c15e030 0x7f572c0cb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:30.994 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.989+0000 7f5731143700 1 -- 172.21.15.67:0/148189688 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f572c15e030 msgr2=0x7f572c0cb3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:30.994 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.989+0000 7f5731143700 1 --2- 172.21.15.67:0/148189688 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f572c15e030 0x7f572c0cb3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:30.994 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.989+0000 7f5731143700 1 -- 172.21.15.67:0/148189688 >> v1:172.21.15.67:6789/0 conn(0x7f572c06a840 legacy=0x7f572c0cdb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:30.994 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.989+0000 7f572b7fe700 1 -- 172.21.15.67:0/148189688 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1805346896 0 0) 0x7f572c06adb0 con 0x7f572c06a840 2023-10-14T10:36:30.995 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.989+0000 7f5731143700 1 -- 172.21.15.67:0/148189688 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f572c06b650 con 0x7f572c0ce280 2023-10-14T10:36:30.995 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.989+0000 7f5730942700 1 --2- 172.21.15.67:0/148189688 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f572c15e030 0x7f572c0cb3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-14T10:36:30.995 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.989+0000 7f5731143700 1 --2- 172.21.15.67:0/148189688 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f572c0ce280 0x7f572c06d3d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f571c009840 tx=0x7f571c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be525c1928f27c49 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:30.995 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.989+0000 7f572b7fe700 1 -- 172.21.15.67:0/148189688 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f571c011830 con 0x7f572c0ce280 2023-10-14T10:36:30.995 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.989+0000 7f572b7fe700 1 -- 172.21.15.67:0/148189688 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f571c011990 con 0x7f572c0ce280 2023-10-14T10:36:30.996 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.989+0000 7f572b7fe700 1 -- 172.21.15.67:0/148189688 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f571c011d20 con 0x7f572c0ce280 2023-10-14T10:36:30.996 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.990+0000 7f5732ba6700 1 -- 172.21.15.67:0/148189688 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f572c0ce280 msgr2=0x7f572c06d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:30.996 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.990+0000 7f5732ba6700 1 --2- 172.21.15.67:0/148189688 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f572c0ce280 0x7f572c06d3d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f571c009840 tx=0x7f571c009b50 comp rx=0 tx=0).stop 2023-10-14T10:36:30.996 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.990+0000 7f5732ba6700 1 -- 172.21.15.67:0/148189688 shutdown_connections 2023-10-14T10:36:30.996 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.990+0000 7f5732ba6700 1 --2- 172.21.15.67:0/148189688 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f572c0ce280 0x7f572c06d3d0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:30.997 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.990+0000 7f5732ba6700 1 --2- 172.21.15.67:0/148189688 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f572c15e030 0x7f572c0cb3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:30.997 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.990+0000 7f5732ba6700 1 -- 172.21.15.67:0/148189688 >> 172.21.15.67:0/148189688 conn(0x7f572c0cc6f0 msgr2=0x7f572c0ccb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:30.997 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.990+0000 7f5732ba6700 1 -- 172.21.15.67:0/148189688 shutdown_connections 2023-10-14T10:36:30.997 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.990+0000 7f5732ba6700 1 -- 172.21.15.67:0/148189688 wait complete. 2023-10-14T10:36:31.000 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.991+0000 7f5732ba6700 1 Processor -- start 2023-10-14T10:36:31.000 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.991+0000 7f5732ba6700 1 -- start start 2023-10-14T10:36:31.000 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.992+0000 7f5732ba6700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f572c06a840 0x7f572c06d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:31.001 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.992+0000 7f5732ba6700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f572c0ce280 0x7f572c0b33b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:31.001 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.992+0000 7f5732ba6700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f572c15e030 0x7f572c160070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:31.001 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.992+0000 7f5732ba6700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f572c160bc0 con 0x7f572c06a840 2023-10-14T10:36:31.001 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.992+0000 7f5732ba6700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f572c0c6a90 con 0x7f572c15e030 2023-10-14T10:36:31.001 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.992+0000 7f5732ba6700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f572c15e540 con 0x7f572c0ce280 2023-10-14T10:36:31.002 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.992+0000 7f5730942700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f572c06a840 0x7f572c06d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:31.002 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.992+0000 7f5730942700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f572c06a840 0x7f572c06d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44770/0 (socket says 172.21.15.67:44770) 2023-10-14T10:36:31.002 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.992+0000 7f5730942700 1 -- 172.21.15.67:0/3083809726 learned_addr learned my addr 172.21.15.67:0/3083809726 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:31.002 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.992+0000 7f5731143700 1 --2- 172.21.15.67:0/3083809726 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f572c15e030 0x7f572c160070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:31.002 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.992+0000 7f572bfff700 1 --2- 172.21.15.67:0/3083809726 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f572c0ce280 0x7f572c0b33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:31.003 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.992+0000 7f5730942700 1 -- 172.21.15.67:0/3083809726 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f572c0ce280 msgr2=0x7f572c0b33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:31.003 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.992+0000 7f5730942700 1 --2- 172.21.15.67:0/3083809726 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f572c0ce280 0x7f572c0b33b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:31.003 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.992+0000 7f5730942700 1 -- 172.21.15.67:0/3083809726 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f572c15e030 msgr2=0x7f572c160070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:31.003 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.992+0000 7f5730942700 1 --2- 172.21.15.67:0/3083809726 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f572c15e030 0x7f572c160070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:31.003 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.992+0000 7f5730942700 1 -- 172.21.15.67:0/3083809726 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f571c0094a0 con 0x7f572c06a840 2023-10-14T10:36:31.004 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.992+0000 7f572bfff700 1 --2- 172.21.15.67:0/3083809726 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f572c0ce280 0x7f572c0b33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-14T10:36:31.004 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.992+0000 7f5731143700 1 --2- 172.21.15.67:0/3083809726 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f572c15e030 0x7f572c160070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-14T10:36:31.004 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.992+0000 7f5730942700 1 --2- 172.21.15.67:0/3083809726 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f572c06a840 0x7f572c06d640 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5724009fa0 tx=0x7f572400d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:31.004 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.992+0000 7f57297fa700 1 -- 172.21.15.67:0/3083809726 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5724003a20 con 0x7f572c06a840 2023-10-14T10:36:31.004 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.992+0000 7f57297fa700 1 -- 172.21.15.67:0/3083809726 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f572400f040 con 0x7f572c06a840 2023-10-14T10:36:31.004 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.992+0000 7f5732ba6700 1 -- 172.21.15.67:0/3083809726 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f572c15e820 con 0x7f572c06a840 2023-10-14T10:36:31.005 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.992+0000 7f57297fa700 1 -- 172.21.15.67:0/3083809726 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5724003de0 con 0x7f572c06a840 2023-10-14T10:36:31.005 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.993+0000 7f5732ba6700 1 -- 172.21.15.67:0/3083809726 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f572c15ede0 con 0x7f572c06a840 2023-10-14T10:36:31.005 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.993+0000 7f5732ba6700 1 -- 172.21.15.67:0/3083809726 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f572c15f150 con 0x7f572c06a840 2023-10-14T10:36:31.005 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.994+0000 7f57297fa700 1 -- 172.21.15.67:0/3083809726 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f5724004030 con 0x7f572c06a840 2023-10-14T10:36:31.006 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.994+0000 7f57297fa700 1 --2- 172.21.15.67:0/3083809726 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f570c06af00 0x7f570c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:31.006 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.994+0000 7f57297fa700 1 -- 172.21.15.67:0/3083809726 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5724089590 con 0x7f572c06a840 2023-10-14T10:36:31.006 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.995+0000 7f572bfff700 1 --2- 172.21.15.67:0/3083809726 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f570c06af00 0x7f570c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:31.006 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.995+0000 7f572bfff700 1 --2- 172.21.15.67:0/3083809726 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f570c06af00 0x7f570c06d3c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5714007fd0 tx=0x7f5714009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:31.007 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:30.999+0000 7f57297fa700 1 -- 172.21.15.67:0/3083809726 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5724054780 con 0x7f572c06a840 2023-10-14T10:36:31.200 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.198+0000 7f5732ba6700 1 -- 172.21.15.67:0/3083809726 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f572c067d20 con 0x7f572c06a840 2023-10-14T10:36:31.200 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.198+0000 7f57297fa700 1 -- 172.21.15.67:0/3083809726 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5724057da0 con 0x7f572c06a840 2023-10-14T10:36:31.201 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2023-10-14T10:36:31.202 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.200+0000 7f5732ba6700 1 -- 172.21.15.67:0/3083809726 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f570c06af00 msgr2=0x7f570c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:31.203 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.200+0000 7f5732ba6700 1 --2- 172.21.15.67:0/3083809726 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f570c06af00 0x7f570c06d3c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5714007fd0 tx=0x7f5714009040 comp rx=0 tx=0).stop 2023-10-14T10:36:31.203 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.200+0000 7f5732ba6700 1 -- 172.21.15.67:0/3083809726 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f572c06a840 msgr2=0x7f572c06d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:31.203 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.200+0000 7f5732ba6700 1 --2- 172.21.15.67:0/3083809726 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f572c06a840 0x7f572c06d640 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5724009fa0 tx=0x7f572400d460 comp rx=0 tx=0).stop 2023-10-14T10:36:31.203 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.200+0000 7f5732ba6700 1 -- 172.21.15.67:0/3083809726 shutdown_connections 2023-10-14T10:36:31.203 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.200+0000 7f5732ba6700 1 --2- 172.21.15.67:0/3083809726 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f570c06af00 0x7f570c06d3c0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:31.204 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.200+0000 7f5732ba6700 1 --2- 172.21.15.67:0/3083809726 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f572c15e030 0x7f572c160070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:31.204 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.200+0000 7f5732ba6700 1 --2- 172.21.15.67:0/3083809726 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f572c06a840 0x7f572c06d640 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:31.204 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.200+0000 7f5732ba6700 1 --2- 172.21.15.67:0/3083809726 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f572c0ce280 0x7f572c0b33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:31.204 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.200+0000 7f5732ba6700 1 -- 172.21.15.67:0/3083809726 >> 172.21.15.67:0/3083809726 conn(0x7f572c0cc6f0 msgr2=0x7f572c0cdb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:31.204 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.200+0000 7f5732ba6700 1 -- 172.21.15.67:0/3083809726 shutdown_connections 2023-10-14T10:36:31.205 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.200+0000 7f5732ba6700 1 -- 172.21.15.67:0/3083809726 wait complete. 2023-10-14T10:36:31.215 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-14T10:36:31.343 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.341+0000 7f662963e700 1 Processor -- start 2023-10-14T10:36:31.343 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.341+0000 7f662963e700 1 -- start start 2023-10-14T10:36:31.347 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.342+0000 7f662963e700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f66240cee90 0x7f662415dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:31.347 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.342+0000 7f662963e700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f662415e6e0 con 0x7f6624157d40 2023-10-14T10:36:31.347 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.342+0000 7f662963e700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6624158120 con 0x7f662415e290 2023-10-14T10:36:31.347 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.342+0000 7f662963e700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f662406e560 con 0x7f66240cee90 2023-10-14T10:36:31.347 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.342+0000 7f6622ffd700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f6624157d40 0x7f662406de50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:52546/0 (socket says 172.21.15.67:52546) 2023-10-14T10:36:31.348 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.342+0000 7f6622ffd700 1 -- 172.21.15.67:0/4184619509 learned_addr learned my addr 172.21.15.67:0/4184619509 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:36:31.348 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.342+0000 7f66227fc700 1 --2- 172.21.15.67:0/4184619509 >> v2:172.21.15.72:3300/0 conn(0x7f66240cee90 0x7f662415dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:31.348 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.342+0000 7f6621ffb700 1 -- 172.21.15.67:0/4184619509 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 315681318 0 0) 0x7f6614001110 con 0x7f6624157d40 2023-10-14T10:36:31.348 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.342+0000 7f6621ffb700 1 -- 172.21.15.67:0/4184619509 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f660c0035b0 con 0x7f6624157d40 2023-10-14T10:36:31.348 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.343+0000 7f6621ffb700 1 -- 172.21.15.67:0/4184619509 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2037400062 0 0) 0x7f660c0035b0 con 0x7f6624157d40 2023-10-14T10:36:31.349 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.343+0000 7f6621ffb700 1 -- 172.21.15.67:0/4184619509 >> v2:172.21.15.72:3300/0 conn(0x7f66240cee90 msgr2=0x7f662415dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:31.349 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.343+0000 7f6621ffb700 1 --2- 172.21.15.67:0/4184619509 >> v2:172.21.15.72:3300/0 conn(0x7f66240cee90 0x7f662415dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:31.349 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.343+0000 7f6621ffb700 1 -- 172.21.15.67:0/4184619509 >> v1:172.21.15.72:6789/0 conn(0x7f662415e290 legacy=0x7f66240cb420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:31.349 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.343+0000 7f6621ffb700 1 -- 172.21.15.67:0/4184619509 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f662406a840 con 0x7f6624157d40 2023-10-14T10:36:31.349 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.343+0000 7f6621ffb700 1 -- 172.21.15.67:0/4184619509 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3180043755 0 0) 0x7f6614001800 con 0x7f6624157d40 2023-10-14T10:36:31.350 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.343+0000 7f6621ffb700 1 -- 172.21.15.67:0/4184619509 >> v1:172.21.15.67:6789/0 conn(0x7f6624157d40 legacy=0x7f662406de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:31.350 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.343+0000 7f6621ffb700 1 --2- 172.21.15.67:0/4184619509 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f660c003a00 0x7f660c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:31.350 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.343+0000 7f6621ffb700 1 --2- 172.21.15.67:0/4184619509 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f660c004850 0x7f660c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:31.350 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.343+0000 7f6621ffb700 1 --2- 172.21.15.67:0/4184619509 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f660c005210 0x7f660c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:31.350 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.343+0000 7f6621ffb700 1 -- 172.21.15.67:0/4184619509 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f660c00d190 con 0x7f660c004850 2023-10-14T10:36:31.351 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.343+0000 7f6621ffb700 1 -- 172.21.15.67:0/4184619509 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f660c00c080 con 0x7f660c005210 2023-10-14T10:36:31.351 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.343+0000 7f6621ffb700 1 -- 172.21.15.67:0/4184619509 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f660c00c240 con 0x7f660c003a00 2023-10-14T10:36:31.351 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.343+0000 7f66227fc700 1 --2- 172.21.15.67:0/4184619509 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f660c004850 0x7f660c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:31.351 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.343+0000 7f6622ffd700 1 --2- 172.21.15.67:0/4184619509 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f660c003a00 0x7f660c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:31.351 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.343+0000 7f66237fe700 1 --2- 172.21.15.67:0/4184619509 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f660c005210 0x7f660c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:31.352 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.343+0000 7f66227fc700 1 -- 172.21.15.67:0/4184619509 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f660c003a00 msgr2=0x7f660c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:31.352 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.343+0000 7f66227fc700 1 --2- 172.21.15.67:0/4184619509 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f660c003a00 0x7f660c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:31.352 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.343+0000 7f66227fc700 1 -- 172.21.15.67:0/4184619509 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f660c005210 msgr2=0x7f660c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:31.352 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.343+0000 7f66227fc700 1 --2- 172.21.15.67:0/4184619509 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f660c005210 0x7f660c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:31.352 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.343+0000 7f66227fc700 1 -- 172.21.15.67:0/4184619509 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f660c0035b0 con 0x7f660c004850 2023-10-14T10:36:31.353 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.343+0000 7f66237fe700 1 --2- 172.21.15.67:0/4184619509 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f660c005210 0x7f660c00ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-14T10:36:31.353 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.343+0000 7f66237fe700 1 -- 172.21.15.67:0/4184619509 reap_dead start 2023-10-14T10:36:31.353 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.343+0000 7f66227fc700 1 --2- 172.21.15.67:0/4184619509 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f660c004850 0x7f660c004cd0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6618009b60 tx=0x7f660c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebddf4a6425d6e0f server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:31.353 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.343+0000 7f6621ffb700 1 -- 172.21.15.67:0/4184619509 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f661800e6c0 con 0x7f660c004850 2023-10-14T10:36:31.354 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.343+0000 7f6621ffb700 1 -- 172.21.15.67:0/4184619509 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f661800e820 con 0x7f660c004850 2023-10-14T10:36:31.354 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.343+0000 7f6621ffb700 1 -- 172.21.15.67:0/4184619509 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f661800eb90 con 0x7f660c004850 2023-10-14T10:36:31.354 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.344+0000 7f662963e700 1 -- 172.21.15.67:0/4184619509 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f660c004850 msgr2=0x7f660c004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:31.354 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.344+0000 7f662963e700 1 --2- 172.21.15.67:0/4184619509 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f660c004850 0x7f660c004cd0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6618009b60 tx=0x7f660c005880 comp rx=0 tx=0).stop 2023-10-14T10:36:31.354 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.344+0000 7f662963e700 1 -- 172.21.15.67:0/4184619509 shutdown_connections 2023-10-14T10:36:31.355 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.344+0000 7f662963e700 1 --2- 172.21.15.67:0/4184619509 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f660c004850 0x7f660c004cd0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:31.355 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.344+0000 7f662963e700 1 -- 172.21.15.67:0/4184619509 >> 172.21.15.67:0/4184619509 conn(0x7f66240cc6f0 msgr2=0x7f66240ccb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:31.355 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.344+0000 7f662963e700 1 -- 172.21.15.67:0/4184619509 shutdown_connections 2023-10-14T10:36:31.355 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.344+0000 7f662963e700 1 -- 172.21.15.67:0/4184619509 wait complete. 2023-10-14T10:36:31.359 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.346+0000 7f662963e700 1 Processor -- start 2023-10-14T10:36:31.359 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.346+0000 7f662963e700 1 -- start start 2023-10-14T10:36:31.360 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.347+0000 7f662963e700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f660c004850 0x7f6624071d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:31.360 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.347+0000 7f662963e700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f66240c8d60 0x7f66240c91c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:31.360 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.347+0000 7f662963e700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f66240c5460 0x7f66240c58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:31.361 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.347+0000 7f662963e700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6624144360 con 0x7f660c004850 2023-10-14T10:36:31.361 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.347+0000 7f662963e700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6624072310 con 0x7f66240c8d60 2023-10-14T10:36:31.361 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.347+0000 7f662963e700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66240c35b0 con 0x7f66240c5460 2023-10-14T10:36:31.362 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.347+0000 7f6622ffd700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f660c004850 0x7f6624071d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:31.362 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.347+0000 7f6622ffd700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f660c004850 0x7f6624071d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44790/0 (socket says 172.21.15.67:44790) 2023-10-14T10:36:31.362 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.347+0000 7f6622ffd700 1 -- 172.21.15.67:0/632010630 learned_addr learned my addr 172.21.15.67:0/632010630 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:31.362 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.347+0000 7f66227fc700 1 --2- 172.21.15.67:0/632010630 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f66240c8d60 0x7f66240c91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:31.363 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.347+0000 7f66237fe700 1 --2- 172.21.15.67:0/632010630 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f66240c5460 0x7f66240c58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:31.363 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.347+0000 7f6622ffd700 1 -- 172.21.15.67:0/632010630 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f66240c5460 msgr2=0x7f66240c58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:31.363 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.347+0000 7f6622ffd700 1 --2- 172.21.15.67:0/632010630 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f66240c5460 0x7f66240c58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:31.363 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.347+0000 7f6622ffd700 1 -- 172.21.15.67:0/632010630 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f66240c8d60 msgr2=0x7f66240c91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:31.364 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.347+0000 7f6622ffd700 1 --2- 172.21.15.67:0/632010630 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f66240c8d60 0x7f66240c91c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:31.364 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.347+0000 7f6622ffd700 1 -- 172.21.15.67:0/632010630 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66240c3850 con 0x7f660c004850 2023-10-14T10:36:31.364 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.347+0000 7f66227fc700 1 --2- 172.21.15.67:0/632010630 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f66240c8d60 0x7f66240c91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-14T10:36:31.364 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.347+0000 7f6622ffd700 1 --2- 172.21.15.67:0/632010630 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f660c004850 0x7f6624071d00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f66140065f0 tx=0x7f66140139c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:31.364 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.348+0000 7f660b7fe700 1 -- 172.21.15.67:0/632010630 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6614023630 con 0x7f660c004850 2023-10-14T10:36:31.365 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.348+0000 7f660b7fe700 1 -- 172.21.15.67:0/632010630 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6614023790 con 0x7f660c004850 2023-10-14T10:36:31.365 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.348+0000 7f662963e700 1 -- 172.21.15.67:0/632010630 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f662415b930 con 0x7f660c004850 2023-10-14T10:36:31.365 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.348+0000 7f660b7fe700 1 -- 172.21.15.67:0/632010630 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6614023b40 con 0x7f660c004850 2023-10-14T10:36:31.365 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.348+0000 7f662963e700 1 -- 172.21.15.67:0/632010630 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66240c9700 con 0x7f660c004850 2023-10-14T10:36:31.365 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.349+0000 7f662963e700 1 -- 172.21.15.67:0/632010630 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f662415bb70 con 0x7f660c004850 2023-10-14T10:36:31.365 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.349+0000 7f660b7fe700 1 -- 172.21.15.67:0/632010630 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f6614019400 con 0x7f660c004850 2023-10-14T10:36:31.366 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.349+0000 7f660b7fe700 1 --2- 172.21.15.67:0/632010630 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f661006af30 0x7f661006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:31.366 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.350+0000 7f660b7fe700 1 -- 172.21.15.67:0/632010630 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6614011040 con 0x7f660c004850 2023-10-14T10:36:31.366 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.350+0000 7f66227fc700 1 --2- 172.21.15.67:0/632010630 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f661006af30 0x7f661006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:31.366 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.350+0000 7f66227fc700 1 --2- 172.21.15.67:0/632010630 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f661006af30 0x7f661006d3f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f66180090b0 tx=0x7f6618008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:31.366 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.354+0000 7f660b7fe700 1 -- 172.21.15.67:0/632010630 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f662415bb70 con 0x7f660c004850 2023-10-14T10:36:31.554 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.552+0000 7f662963e700 1 -- 172.21.15.67:0/632010630 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f662415e6e0 con 0x7f660c004850 2023-10-14T10:36:31.555 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.552+0000 7f660b7fe700 1 -- 172.21.15.67:0/632010630 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f662415e6e0 con 0x7f660c004850 2023-10-14T10:36:31.555 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2023-10-14T10:36:31.557 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.554+0000 7f662963e700 1 -- 172.21.15.67:0/632010630 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f661006af30 msgr2=0x7f661006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:31.557 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.554+0000 7f662963e700 1 --2- 172.21.15.67:0/632010630 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f661006af30 0x7f661006d3f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f66180090b0 tx=0x7f6618008030 comp rx=0 tx=0).stop 2023-10-14T10:36:31.557 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.554+0000 7f662963e700 1 -- 172.21.15.67:0/632010630 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f660c004850 msgr2=0x7f6624071d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:31.557 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.554+0000 7f662963e700 1 --2- 172.21.15.67:0/632010630 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f660c004850 0x7f6624071d00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f66140065f0 tx=0x7f66140139c0 comp rx=0 tx=0).stop 2023-10-14T10:36:31.558 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.554+0000 7f662963e700 1 -- 172.21.15.67:0/632010630 shutdown_connections 2023-10-14T10:36:31.558 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.554+0000 7f662963e700 1 --2- 172.21.15.67:0/632010630 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f661006af30 0x7f661006d3f0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:31.558 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.554+0000 7f662963e700 1 --2- 172.21.15.67:0/632010630 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f66240c5460 0x7f66240c58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:31.558 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.554+0000 7f662963e700 1 --2- 172.21.15.67:0/632010630 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f660c004850 0x7f6624071d00 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:31.559 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.554+0000 7f662963e700 1 --2- 172.21.15.67:0/632010630 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f66240c8d60 0x7f66240c91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:31.559 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.554+0000 7f662963e700 1 -- 172.21.15.67:0/632010630 >> 172.21.15.67:0/632010630 conn(0x7f66240cc6f0 msgr2=0x7f66240ccad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:31.559 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.554+0000 7f662963e700 1 -- 172.21.15.67:0/632010630 shutdown_connections 2023-10-14T10:36:31.559 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.554+0000 7f662963e700 1 -- 172.21.15.67:0/632010630 wait complete. 2023-10-14T10:36:31.569 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-14T10:36:31.696 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.694+0000 7f1711cb1700 1 Processor -- start 2023-10-14T10:36:31.697 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.694+0000 7f1711cb1700 1 -- start start 2023-10-14T10:36:31.707 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.695+0000 7f1711cb1700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f170c15b740 0x7f170c0d1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:31.707 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.695+0000 7f1711cb1700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f170c0d0c70 0x7f170c0cdfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:31.708 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.695+0000 7f1711cb1700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f170c0cf190 con 0x7f170c0c5110 2023-10-14T10:36:31.708 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.695+0000 7f1711cb1700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f170c15bb20 con 0x7f170c0d0c70 2023-10-14T10:36:31.708 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.695+0000 7f1711cb1700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f170c0cbd40 con 0x7f170c15b740 2023-10-14T10:36:31.708 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.695+0000 7f170bfff700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f170c0c5110 0x7f170c0cf690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:52570/0 (socket says 172.21.15.67:52570) 2023-10-14T10:36:31.708 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.695+0000 7f170bfff700 1 -- 172.21.15.67:0/2060950974 learned_addr learned my addr 172.21.15.67:0/2060950974 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:36:31.709 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.695+0000 7f170affd700 1 --2- 172.21.15.67:0/2060950974 >> v2:172.21.15.72:3300/0 conn(0x7f170c0d0c70 0x7f170c0cdfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:31.709 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.695+0000 7f170b7fe700 1 --2- 172.21.15.67:0/2060950974 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f170c15b740 0x7f170c0d1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:31.709 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.695+0000 7f170a7fc700 1 -- 172.21.15.67:0/2060950974 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2647599678 0 0) 0x7f170c0cf190 con 0x7f170c0c5110 2023-10-14T10:36:31.709 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.695+0000 7f170a7fc700 1 -- 172.21.15.67:0/2060950974 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f16f40035b0 con 0x7f170c0c5110 2023-10-14T10:36:31.709 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.695+0000 7f170affd700 1 -- 172.21.15.67:0/2060950974 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f170c15b740 msgr2=0x7f170c0d1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:31.709 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.695+0000 7f170affd700 1 --2- 172.21.15.67:0/2060950974 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f170c15b740 0x7f170c0d1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:31.710 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.695+0000 7f170affd700 1 -- 172.21.15.67:0/2060950974 >> v1:172.21.15.67:6789/0 conn(0x7f170c0c5110 legacy=0x7f170c0cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:31.710 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.695+0000 7f170a7fc700 1 -- 172.21.15.67:0/2060950974 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 670480119 0 0) 0x7f16f40035b0 con 0x7f170c0c5110 2023-10-14T10:36:31.710 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.695+0000 7f170affd700 1 -- 172.21.15.67:0/2060950974 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f170c0cc020 con 0x7f170c0d0c70 2023-10-14T10:36:31.710 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.696+0000 7f170affd700 1 --2- 172.21.15.67:0/2060950974 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f170c0d0c70 0x7f170c0cdfd0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1700009840 tx=0x7f1700009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96141a1e2b2f8dce server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:31.710 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.696+0000 7f170a7fc700 1 -- 172.21.15.67:0/2060950974 <== mon.1 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17040019f0 con 0x7f170c0d0c70 2023-10-14T10:36:31.711 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.696+0000 7f170a7fc700 1 -- 172.21.15.67:0/2060950974 <== mon.1 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1700004030 con 0x7f170c0d0c70 2023-10-14T10:36:31.711 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.696+0000 7f170a7fc700 1 -- 172.21.15.67:0/2060950974 <== mon.1 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17000043c0 con 0x7f170c0d0c70 2023-10-14T10:36:31.711 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.696+0000 7f1711cb1700 1 -- 172.21.15.67:0/2060950974 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f170c0d0c70 msgr2=0x7f170c0cdfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:31.711 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.696+0000 7f1711cb1700 1 --2- 172.21.15.67:0/2060950974 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f170c0d0c70 0x7f170c0cdfd0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1700009840 tx=0x7f1700009b50 comp rx=0 tx=0).stop 2023-10-14T10:36:31.711 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.696+0000 7f1711cb1700 1 -- 172.21.15.67:0/2060950974 shutdown_connections 2023-10-14T10:36:31.712 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.696+0000 7f1711cb1700 1 --2- 172.21.15.67:0/2060950974 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f170c15b740 0x7f170c0d1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:31.712 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.696+0000 7f1711cb1700 1 --2- 172.21.15.67:0/2060950974 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f170c0d0c70 0x7f170c0cdfd0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:31.712 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.696+0000 7f1711cb1700 1 -- 172.21.15.67:0/2060950974 >> 172.21.15.67:0/2060950974 conn(0x7f170c0665a0 msgr2=0x7f170c066980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:31.712 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.696+0000 7f1711cb1700 1 -- 172.21.15.67:0/2060950974 shutdown_connections 2023-10-14T10:36:31.712 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.696+0000 7f1711cb1700 1 -- 172.21.15.67:0/2060950974 wait complete. 2023-10-14T10:36:31.712 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.698+0000 7f1711cb1700 1 Processor -- start 2023-10-14T10:36:31.713 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.698+0000 7f1711cb1700 1 -- start start 2023-10-14T10:36:31.713 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.699+0000 7f1711cb1700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f170c0c5110 0x7f170c15c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:31.713 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.699+0000 7f1711cb1700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f170c0d0c70 0x7f170c14ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:31.713 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.699+0000 7f1711cb1700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f170c15b740 0x7f170c0b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:31.713 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.699+0000 7f1711cb1700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f170c15ecd0 con 0x7f170c0d0c70 2023-10-14T10:36:31.714 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.699+0000 7f1711cb1700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f170c14ef70 con 0x7f170c15b740 2023-10-14T10:36:31.714 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.699+0000 7f1711cb1700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f170c06baf0 con 0x7f170c0c5110 2023-10-14T10:36:31.714 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.699+0000 7f170affd700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f170c0d0c70 0x7f170c14ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:31.714 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.699+0000 7f170affd700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f170c0d0c70 0x7f170c14ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44806/0 (socket says 172.21.15.67:44806) 2023-10-14T10:36:31.714 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.699+0000 7f170affd700 1 -- 172.21.15.67:0/2427791973 learned_addr learned my addr 172.21.15.67:0/2427791973 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:31.715 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.699+0000 7f170b7fe700 1 --2- 172.21.15.67:0/2427791973 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f170c0c5110 0x7f170c15c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:31.715 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.699+0000 7f170bfff700 1 --2- 172.21.15.67:0/2427791973 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f170c15b740 0x7f170c0b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:31.715 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.699+0000 7f170affd700 1 -- 172.21.15.67:0/2427791973 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f170c0c5110 msgr2=0x7f170c15c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:31.715 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.699+0000 7f170affd700 1 --2- 172.21.15.67:0/2427791973 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f170c0c5110 0x7f170c15c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:31.715 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.699+0000 7f170affd700 1 -- 172.21.15.67:0/2427791973 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f170c15b740 msgr2=0x7f170c0b43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:31.716 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.699+0000 7f170affd700 1 --2- 172.21.15.67:0/2427791973 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f170c15b740 0x7f170c0b43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:31.716 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.699+0000 7f170affd700 1 -- 172.21.15.67:0/2427791973 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17000094a0 con 0x7f170c0d0c70 2023-10-14T10:36:31.716 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.699+0000 7f170affd700 1 --2- 172.21.15.67:0/2427791973 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f170c0d0c70 0x7f170c14ea30 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f170000a010 tx=0x7f1700004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:31.716 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.699+0000 7f16f3fff700 1 -- 172.21.15.67:0/2427791973 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1700003cf0 con 0x7f170c0d0c70 2023-10-14T10:36:31.716 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.699+0000 7f16f3fff700 1 -- 172.21.15.67:0/2427791973 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f170001f070 con 0x7f170c0d0c70 2023-10-14T10:36:31.716 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.699+0000 7f1711cb1700 1 -- 172.21.15.67:0/2427791973 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f170c06bd90 con 0x7f170c0d0c70 2023-10-14T10:36:31.717 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.699+0000 7f16f3fff700 1 -- 172.21.15.67:0/2427791973 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1700011870 con 0x7f170c0d0c70 2023-10-14T10:36:31.717 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.699+0000 7f1711cb1700 1 -- 172.21.15.67:0/2427791973 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f170c06c350 con 0x7f170c0d0c70 2023-10-14T10:36:31.717 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.700+0000 7f1711cb1700 1 -- 172.21.15.67:0/2427791973 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f170c067d20 con 0x7f170c0d0c70 2023-10-14T10:36:31.717 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.701+0000 7f16f3fff700 1 -- 172.21.15.67:0/2427791973 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f17000119d0 con 0x7f170c0d0c70 2023-10-14T10:36:31.717 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.701+0000 7f16f3fff700 1 --2- 172.21.15.67:0/2427791973 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f16f806af00 0x7f16f806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:31.718 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.701+0000 7f16f3fff700 1 -- 172.21.15.67:0/2427791973 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1700021070 con 0x7f170c0d0c70 2023-10-14T10:36:31.718 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.701+0000 7f170b7fe700 1 --2- 172.21.15.67:0/2427791973 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f16f806af00 0x7f16f806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:31.718 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.702+0000 7f170b7fe700 1 --2- 172.21.15.67:0/2427791973 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f16f806af00 0x7f16f806d3c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f170c0d0a10 tx=0x7f16fc009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:31.718 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.706+0000 7f16f3fff700 1 -- 172.21.15.67:0/2427791973 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1700091080 con 0x7f170c0d0c70 2023-10-14T10:36:31.908 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.905+0000 7f1711cb1700 1 -- 172.21.15.67:0/2427791973 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f170c0c6110 con 0x7f170c0d0c70 2023-10-14T10:36:31.908 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.906+0000 7f16f3fff700 1 -- 172.21.15.67:0/2427791973 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f170002b020 con 0x7f170c0d0c70 2023-10-14T10:36:31.908 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2023-10-14T10:36:31.910 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.908+0000 7f1711cb1700 1 -- 172.21.15.67:0/2427791973 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f16f806af00 msgr2=0x7f16f806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:31.910 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.908+0000 7f1711cb1700 1 --2- 172.21.15.67:0/2427791973 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f16f806af00 0x7f16f806d3c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f170c0d0a10 tx=0x7f16fc009210 comp rx=0 tx=0).stop 2023-10-14T10:36:31.910 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.908+0000 7f1711cb1700 1 -- 172.21.15.67:0/2427791973 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f170c0d0c70 msgr2=0x7f170c14ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:31.910 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.908+0000 7f1711cb1700 1 --2- 172.21.15.67:0/2427791973 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f170c0d0c70 0x7f170c14ea30 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f170000a010 tx=0x7f1700004750 comp rx=0 tx=0).stop 2023-10-14T10:36:31.911 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.908+0000 7f1711cb1700 1 -- 172.21.15.67:0/2427791973 shutdown_connections 2023-10-14T10:36:31.911 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.908+0000 7f1711cb1700 1 --2- 172.21.15.67:0/2427791973 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f16f806af00 0x7f16f806d3c0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:31.911 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.908+0000 7f1711cb1700 1 --2- 172.21.15.67:0/2427791973 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f170c15b740 0x7f170c0b43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:31.911 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.908+0000 7f1711cb1700 1 --2- 172.21.15.67:0/2427791973 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f170c0c5110 0x7f170c15c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:31.911 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.908+0000 7f1711cb1700 1 --2- 172.21.15.67:0/2427791973 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f170c0d0c70 0x7f170c14ea30 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:31.912 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.908+0000 7f1711cb1700 1 -- 172.21.15.67:0/2427791973 >> 172.21.15.67:0/2427791973 conn(0x7f170c0665a0 msgr2=0x7f170c0cdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:31.912 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.908+0000 7f1711cb1700 1 -- 172.21.15.67:0/2427791973 shutdown_connections 2023-10-14T10:36:31.912 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:31.908+0000 7f1711cb1700 1 -- 172.21.15.67:0/2427791973 wait complete. 2023-10-14T10:36:31.922 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-14T10:36:32.048 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.046+0000 7f28675d8700 1 Processor -- start 2023-10-14T10:36:32.049 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.047+0000 7f28675d8700 1 -- start start 2023-10-14T10:36:32.053 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.048+0000 7f28675d8700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f28600d3fc0 0x7f286015dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:32.053 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.048+0000 7f28675d8700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f286015e4e0 con 0x7f286015dbc0 2023-10-14T10:36:32.053 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.048+0000 7f28675d8700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28600d28e0 con 0x7f28600b7050 2023-10-14T10:36:32.053 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.048+0000 7f28675d8700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28600b7560 con 0x7f28600d3fc0 2023-10-14T10:36:32.053 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.048+0000 7f2865374700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f286015dbc0 0x7f28600d1ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:52580/0 (socket says 172.21.15.67:52580) 2023-10-14T10:36:32.054 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.048+0000 7f2865374700 1 -- 172.21.15.67:0/3896263704 learned_addr learned my addr 172.21.15.67:0/3896263704 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:36:32.054 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.048+0000 7f2857fff700 1 -- 172.21.15.67:0/3896263704 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 634564801 0 0) 0x7f286015e4e0 con 0x7f286015dbc0 2023-10-14T10:36:32.054 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.048+0000 7f2857fff700 1 -- 172.21.15.67:0/3896263704 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f284c0035b0 con 0x7f286015dbc0 2023-10-14T10:36:32.054 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.048+0000 7f2864b73700 1 --2- 172.21.15.67:0/3896263704 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f28600d3fc0 0x7f286015dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:32.054 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.048+0000 7f2857fff700 1 -- 172.21.15.67:0/3896263704 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 379897251 0 0) 0x7f284c0035b0 con 0x7f286015dbc0 2023-10-14T10:36:32.055 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.048+0000 7f2857fff700 1 -- 172.21.15.67:0/3896263704 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f28600d3fc0 msgr2=0x7f286015dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:32.055 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.048+0000 7f2857fff700 1 --2- 172.21.15.67:0/3896263704 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f28600d3fc0 0x7f286015dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:32.055 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.048+0000 7f2857fff700 1 -- 172.21.15.67:0/3896263704 >> v1:172.21.15.72:6789/0 conn(0x7f28600b7050 legacy=0x7f28600d21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:32.055 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.048+0000 7f2857fff700 1 -- 172.21.15.67:0/3896263704 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28600b7840 con 0x7f286015dbc0 2023-10-14T10:36:32.057 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.048+0000 7f2857fff700 1 -- 172.21.15.67:0/3896263704 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3180043755 0 0) 0x7f285c0017f0 con 0x7f286015dbc0 2023-10-14T10:36:32.057 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.049+0000 7f2857fff700 1 -- 172.21.15.67:0/3896263704 >> v1:172.21.15.67:6789/0 conn(0x7f286015dbc0 legacy=0x7f28600d1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:32.058 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.049+0000 7f2857fff700 1 --2- 172.21.15.67:0/3896263704 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f28600d3fc0 0x7f284c003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:32.058 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.049+0000 7f2857fff700 1 --2- 172.21.15.67:0/3896263704 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f284c003fb0 0x7f284c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:32.058 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.049+0000 7f2857fff700 1 --2- 172.21.15.67:0/3896263704 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f284c004d90 0x7f284c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:32.058 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.049+0000 7f2857fff700 1 -- 172.21.15.67:0/3896263704 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28600b7560 con 0x7f284c004d90 2023-10-14T10:36:32.058 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.049+0000 7f2857fff700 1 -- 172.21.15.67:0/3896263704 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f284c00baf0 con 0x7f284c003fb0 2023-10-14T10:36:32.059 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.049+0000 7f2857fff700 1 -- 172.21.15.67:0/3896263704 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f284c00bcb0 con 0x7f28600d3fc0 2023-10-14T10:36:32.059 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.049+0000 7f2865b75700 1 --2- 172.21.15.67:0/3896263704 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f284c004d90 0x7f284c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:32.059 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.049+0000 7f2865374700 1 --2- 172.21.15.67:0/3896263704 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f28600d3fc0 0x7f284c003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:32.059 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.049+0000 7f2864b73700 1 --2- 172.21.15.67:0/3896263704 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f284c003fb0 0x7f284c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:32.059 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.049+0000 7f2865b75700 1 -- 172.21.15.67:0/3896263704 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f28600d3fc0 msgr2=0x7f284c003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:32.060 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.049+0000 7f2865b75700 1 --2- 172.21.15.67:0/3896263704 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f28600d3fc0 0x7f284c003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:32.060 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.049+0000 7f2865b75700 1 -- 172.21.15.67:0/3896263704 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f284c003fb0 msgr2=0x7f284c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:32.060 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.049+0000 7f2865b75700 1 --2- 172.21.15.67:0/3896263704 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f284c003fb0 0x7f284c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:32.060 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.049+0000 7f2865b75700 1 -- 172.21.15.67:0/3896263704 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f284c0035b0 con 0x7f284c004d90 2023-10-14T10:36:32.060 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.049+0000 7f2865374700 1 --2- 172.21.15.67:0/3896263704 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f28600d3fc0 0x7f284c003a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-14T10:36:32.061 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.049+0000 7f2864b73700 1 --2- 172.21.15.67:0/3896263704 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f284c003fb0 0x7f284c004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-14T10:36:32.061 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.049+0000 7f2865b75700 1 --2- 172.21.15.67:0/3896263704 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f284c004d90 0x7f284c005210 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f285800c160 tx=0x7f284c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e76316501713229b server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:32.061 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.049+0000 7f2857fff700 1 -- 172.21.15.67:0/3896263704 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2858001db0 con 0x7f284c004d90 2023-10-14T10:36:32.061 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.049+0000 7f2857fff700 1 -- 172.21.15.67:0/3896263704 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f285800cca0 con 0x7f284c004d90 2023-10-14T10:36:32.061 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.049+0000 7f2857fff700 1 -- 172.21.15.67:0/3896263704 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2858003c40 con 0x7f284c004d90 2023-10-14T10:36:32.061 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.050+0000 7f28675d8700 1 -- 172.21.15.67:0/3896263704 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f284c004d90 msgr2=0x7f284c005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:32.062 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.050+0000 7f28675d8700 1 --2- 172.21.15.67:0/3896263704 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f284c004d90 0x7f284c005210 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f285800c160 tx=0x7f284c005880 comp rx=0 tx=0).stop 2023-10-14T10:36:32.062 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.050+0000 7f2865b75700 1 -- 172.21.15.67:0/3896263704 reap_dead start 2023-10-14T10:36:32.062 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.050+0000 7f28675d8700 1 -- 172.21.15.67:0/3896263704 shutdown_connections 2023-10-14T10:36:32.063 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.050+0000 7f28675d8700 1 -- 172.21.15.67:0/3896263704 >> 172.21.15.67:0/3896263704 conn(0x7f286009cb40 msgr2=0x7f28600d3820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:32.063 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.050+0000 7f28675d8700 1 -- 172.21.15.67:0/3896263704 shutdown_connections 2023-10-14T10:36:32.063 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.050+0000 7f28675d8700 1 -- 172.21.15.67:0/3896263704 wait complete. 2023-10-14T10:36:32.066 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.052+0000 7f28675d8700 1 Processor -- start 2023-10-14T10:36:32.066 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.052+0000 7f28675d8700 1 -- start start 2023-10-14T10:36:32.066 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.053+0000 7f28675d8700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f286009f9d0 0x7f28601f3090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:32.066 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.053+0000 7f28675d8700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f286009fdb0 0x7f28601f4250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:32.067 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.053+0000 7f28675d8700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f286015a3b0 0x7f286015b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:32.067 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.053+0000 7f28675d8700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f286009d5e0 con 0x7f286009f9d0 2023-10-14T10:36:32.067 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.053+0000 7f28675d8700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28601f4790 con 0x7f286015a3b0 2023-10-14T10:36:32.067 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.053+0000 7f28675d8700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28601f36e0 con 0x7f286009fdb0 2023-10-14T10:36:32.067 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.053+0000 7f2865374700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f286009f9d0 0x7f28601f3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:32.068 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.053+0000 7f2865374700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f286009f9d0 0x7f28601f3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44824/0 (socket says 172.21.15.67:44824) 2023-10-14T10:36:32.068 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.053+0000 7f2865374700 1 -- 172.21.15.67:0/1756378057 learned_addr learned my addr 172.21.15.67:0/1756378057 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:32.068 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.053+0000 7f2864b73700 1 --2- 172.21.15.67:0/1756378057 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f286009fdb0 0x7f28601f4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:32.068 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.053+0000 7f2865b75700 1 --2- 172.21.15.67:0/1756378057 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f286015a3b0 0x7f286015b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:32.068 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.053+0000 7f2865374700 1 -- 172.21.15.67:0/1756378057 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f286009fdb0 msgr2=0x7f28601f4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:32.069 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.053+0000 7f2865374700 1 --2- 172.21.15.67:0/1756378057 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f286009fdb0 0x7f28601f4250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:32.069 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.053+0000 7f2865374700 1 -- 172.21.15.67:0/1756378057 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f286015a3b0 msgr2=0x7f286015b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:32.069 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.053+0000 7f2865374700 1 --2- 172.21.15.67:0/1756378057 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f286015a3b0 0x7f286015b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:32.069 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.053+0000 7f2865374700 1 -- 172.21.15.67:0/1756378057 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28601f39c0 con 0x7f286009f9d0 2023-10-14T10:36:32.069 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.053+0000 7f2865b75700 1 --2- 172.21.15.67:0/1756378057 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f286015a3b0 0x7f286015b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-14T10:36:32.070 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.053+0000 7f2865374700 1 --2- 172.21.15.67:0/1756378057 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f286009f9d0 0x7f28601f3090 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f285c001880 tx=0x7f285c00d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:32.070 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.053+0000 7f2855ffb700 1 -- 172.21.15.67:0/1756378057 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f285c00da40 con 0x7f286009f9d0 2023-10-14T10:36:32.070 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.053+0000 7f2855ffb700 1 -- 172.21.15.67:0/1756378057 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f285c00dba0 con 0x7f286009f9d0 2023-10-14T10:36:32.070 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.053+0000 7f28675d8700 1 -- 172.21.15.67:0/1756378057 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28601f3ca0 con 0x7f286009f9d0 2023-10-14T10:36:32.070 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.053+0000 7f2855ffb700 1 -- 172.21.15.67:0/1756378057 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f285c0070d0 con 0x7f286009f9d0 2023-10-14T10:36:32.071 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.054+0000 7f28675d8700 1 -- 172.21.15.67:0/1756378057 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f286015a860 con 0x7f286009f9d0 2023-10-14T10:36:32.071 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.054+0000 7f28675d8700 1 -- 172.21.15.67:0/1756378057 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f286004ebb0 con 0x7f286009f9d0 2023-10-14T10:36:32.071 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.055+0000 7f2855ffb700 1 -- 172.21.15.67:0/1756378057 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f285c007230 con 0x7f286009f9d0 2023-10-14T10:36:32.071 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.055+0000 7f2855ffb700 1 --2- 172.21.15.67:0/1756378057 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f284406af00 0x7f284406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:32.071 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.055+0000 7f2855ffb700 1 -- 172.21.15.67:0/1756378057 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f285c089810 con 0x7f286009f9d0 2023-10-14T10:36:32.072 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.056+0000 7f2864b73700 1 --2- 172.21.15.67:0/1756378057 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f284406af00 0x7f284406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:32.072 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.056+0000 7f2864b73700 1 --2- 172.21.15.67:0/1756378057 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f284406af00 0x7f284406d3c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f286009e7e0 tx=0x7f2850008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:32.072 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.060+0000 7f2855ffb700 1 -- 172.21.15.67:0/1756378057 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f285c054950 con 0x7f286009f9d0 2023-10-14T10:36:32.261 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.259+0000 7f28675d8700 1 -- 172.21.15.67:0/1756378057 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f286008d890 con 0x7f286009f9d0 2023-10-14T10:36:32.262 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.260+0000 7f2855ffb700 1 -- 172.21.15.67:0/1756378057 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16457 (secure 0 0 0) 0x7f285c057f70 con 0x7f286009f9d0 2023-10-14T10:36:32.262 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-14T10:36:32.263 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":22,"fsid":"37f9ab46-3082-4b71-8b24-08d39897b008","created":"2023-10-14T10:35:46.067554+0000","modified":"2023-10-14T10:36:04.837296+0000","last_up_change":"2023-10-14T10:35:56.668142+0000","last_in_change":"2023-10-14T10:35:48.551490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-14T10:35:57.809566+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-14T10:36:01.667146+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-14T10:36:02.219700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2b59a07e-11ef-440c-b766-9f64ac885838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6801","nonce":1287255499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6803","nonce":1287255499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6807","nonce":1287255499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6805","nonce":1287255499}]},"public_addr":"172.21.15.67:6801/1287255499","cluster_addr":"172.21.15.67:6803/1287255499","heartbeat_back_addr":"172.21.15.67:6807/1287255499","heartbeat_front_addr":"172.21.15.67:6805/1287255499","state":["exists","up"]},{"osd":1,"uuid":"6f813bd6-fa0b-487d-a204-431eb1276a60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6818","nonce":2435538787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6822","nonce":2435538787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6830","nonce":2435538787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6826","nonce":2435538787}]},"public_addr":"172.21.15.67:6818/2435538787","cluster_addr":"172.21.15.67:6822/2435538787","heartbeat_back_addr":"172.21.15.67:6830/2435538787","heartbeat_front_addr":"172.21.15.67:6826/2435538787","state":["exists","up"]},{"osd":2,"uuid":"6af42c6a-1b59-4669-9c04-b6876788446d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6809","nonce":2827491791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6811","nonce":2827491791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6815","nonce":2827491791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6813","nonce":2827491791}]},"public_addr":"172.21.15.67:6809/2827491791","cluster_addr":"172.21.15.67:6811/2827491791","heartbeat_back_addr":"172.21.15.67:6815/2827491791","heartbeat_front_addr":"172.21.15.67:6813/2827491791","state":["exists","up"]},{"osd":3,"uuid":"1d889d76-a646-499d-bfff-ed5bae5434ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6819","nonce":1848001176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6821","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6823","nonce":1848001176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6831","nonce":1848001176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6825","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6827","nonce":1848001176}]},"public_addr":"172.21.15.67:6819/1848001176","cluster_addr":"172.21.15.67:6823/1848001176","heartbeat_back_addr":"172.21.15.67:6831/1848001176","heartbeat_front_addr":"172.21.15.67:6827/1848001176","state":["exists","up"]},{"osd":4,"uuid":"c3271dff-b437-4fdd-b13b-6e49a35473af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6804","nonce":1891230785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6808","nonce":1891230785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6815","nonce":1891230785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6811","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6813","nonce":1891230785}]},"public_addr":"172.21.15.72:6804/1891230785","cluster_addr":"172.21.15.72:6808/1891230785","heartbeat_back_addr":"172.21.15.72:6815/1891230785","heartbeat_front_addr":"172.21.15.72:6813/1891230785","state":["exists","up"]},{"osd":5,"uuid":"0337bb18-4b68-4eff-b528-a667275e6926","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6801","nonce":569686991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6803","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6805","nonce":569686991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6812","nonce":569686991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6807","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6809","nonce":569686991}]},"public_addr":"172.21.15.72:6801/569686991","cluster_addr":"172.21.15.72:6805/569686991","heartbeat_back_addr":"172.21.15.72:6812/569686991","heartbeat_front_addr":"172.21.15.72:6809/569686991","state":["exists","up"]},{"osd":6,"uuid":"8e92d9cf-0c8c-46eb-89a7-db2023ef1509","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6817","nonce":3767685934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6819","nonce":3767685934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6823","nonce":3767685934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6821","nonce":3767685934}]},"public_addr":"172.21.15.72:6817/3767685934","cluster_addr":"172.21.15.72:6819/3767685934","heartbeat_back_addr":"172.21.15.72:6823/3767685934","heartbeat_front_addr":"172.21.15.72:6821/3767685934","state":["exists","up"]},{"osd":7,"uuid":"3fd6aa7d-50a9-45c9-9561-5902a044a119","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6825","nonce":3567465926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6827","nonce":3567465926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6831","nonce":3567465926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6829","nonce":3567465926}]},"public_addr":"172.21.15.72:6825/3567465926","cluster_addr":"172.21.15.72:6827/3567465926","heartbeat_back_addr":"172.21.15.72:6831/3567465926","heartbeat_front_addr":"172.21.15.72:6829/3567465926","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.511466+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.795107+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.858674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.728895+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.794842+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.750064+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.732122+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.765185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-14T10:36:32.264 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.262+0000 7f28675d8700 1 -- 172.21.15.67:0/1756378057 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f284406af00 msgr2=0x7f284406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:32.265 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.262+0000 7f28675d8700 1 --2- 172.21.15.67:0/1756378057 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f284406af00 0x7f284406d3c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f286009e7e0 tx=0x7f2850008040 comp rx=0 tx=0).stop 2023-10-14T10:36:32.265 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.262+0000 7f28675d8700 1 -- 172.21.15.67:0/1756378057 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f286009f9d0 msgr2=0x7f28601f3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:32.265 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.262+0000 7f28675d8700 1 --2- 172.21.15.67:0/1756378057 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f286009f9d0 0x7f28601f3090 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f285c001880 tx=0x7f285c00d430 comp rx=0 tx=0).stop 2023-10-14T10:36:32.265 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.262+0000 7f28675d8700 1 -- 172.21.15.67:0/1756378057 shutdown_connections 2023-10-14T10:36:32.266 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.262+0000 7f28675d8700 1 --2- 172.21.15.67:0/1756378057 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f284406af00 0x7f284406d3c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:32.266 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.262+0000 7f28675d8700 1 --2- 172.21.15.67:0/1756378057 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f286015a3b0 0x7f286015b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:32.266 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.262+0000 7f28675d8700 1 --2- 172.21.15.67:0/1756378057 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f286009f9d0 0x7f28601f3090 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:32.266 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.262+0000 7f28675d8700 1 --2- 172.21.15.67:0/1756378057 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f286009fdb0 0x7f28601f4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:32.266 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.262+0000 7f28675d8700 1 -- 172.21.15.67:0/1756378057 >> 172.21.15.67:0/1756378057 conn(0x7f286009cb40 msgr2=0x7f28600d00d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:32.267 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.262+0000 7f28675d8700 1 -- 172.21.15.67:0/1756378057 shutdown_connections 2023-10-14T10:36:32.267 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.262+0000 7f28675d8700 1 -- 172.21.15.67:0/1756378057 wait complete. 2023-10-14T10:36:32.277 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-14T10:35:57.809566+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-14T10:36:01.667146+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-14T10:36:02.219700+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-14T10:36:32.278 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-14T10:36:32.408 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.406+0000 7fb54e084700 1 Processor -- start 2023-10-14T10:36:32.408 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.406+0000 7fb54e084700 1 -- start start 2023-10-14T10:36:32.411 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.406+0000 7fb54e084700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fb54815f1f0 0x7fb5480cf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:32.411 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.406+0000 7fb54e084700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fb5480cfc60 0x7fb5480cbf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:32.411 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.406+0000 7fb54e084700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb54815e7a0 con 0x7fb54815ab10 2023-10-14T10:36:32.411 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.406+0000 7fb54e084700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5480d00d0 con 0x7fb54815f1f0 2023-10-14T10:36:32.411 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.406+0000 7fb54e084700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5480b29a0 con 0x7fb5480cfc60 2023-10-14T10:36:32.412 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.407+0000 7fb5477fe700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fb54815ab10 0x7fb5480b1f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:52600/0 (socket says 172.21.15.67:52600) 2023-10-14T10:36:32.412 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.407+0000 7fb5477fe700 1 -- 172.21.15.67:0/3259096875 learned_addr learned my addr 172.21.15.67:0/3259096875 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:36:32.412 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.407+0000 7fb547fff700 1 --2- 172.21.15.67:0/3259096875 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fb5480cfc60 0x7fb5480cbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:32.412 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.407+0000 7fb546ffd700 1 --2- 172.21.15.67:0/3259096875 >> v2:172.21.15.72:3300/0 conn(0x7fb54815f1f0 0x7fb5480cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:32.412 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.407+0000 7fb5467fc700 1 -- 172.21.15.67:0/3259096875 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4123360782 0 0) 0x7fb54815e7a0 con 0x7fb54815ab10 2023-10-14T10:36:32.413 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.407+0000 7fb5467fc700 1 -- 172.21.15.67:0/3259096875 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5300035b0 con 0x7fb54815ab10 2023-10-14T10:36:32.413 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.407+0000 7fb546ffd700 1 -- 172.21.15.67:0/3259096875 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fb5480cfc60 msgr2=0x7fb5480cbf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:32.413 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.407+0000 7fb546ffd700 1 --2- 172.21.15.67:0/3259096875 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fb5480cfc60 0x7fb5480cbf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:32.413 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.407+0000 7fb546ffd700 1 -- 172.21.15.67:0/3259096875 >> v1:172.21.15.67:6789/0 conn(0x7fb54815ab10 legacy=0x7fb5480b1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:32.413 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.407+0000 7fb546ffd700 1 -- 172.21.15.67:0/3259096875 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5480b2c80 con 0x7fb54815f1f0 2023-10-14T10:36:32.414 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.407+0000 7fb547fff700 1 --2- 172.21.15.67:0/3259096875 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fb5480cfc60 0x7fb5480cbf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-14T10:36:32.414 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.407+0000 7fb546ffd700 1 --2- 172.21.15.67:0/3259096875 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb54815f1f0 0x7fb5480cf720 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb53c009820 tx=0x7fb53c009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43e77f719723378f server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:32.414 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.407+0000 7fb5467fc700 1 -- 172.21.15.67:0/3259096875 <== mon.1 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb53c005e30 con 0x7fb54815f1f0 2023-10-14T10:36:32.414 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.408+0000 7fb5467fc700 1 -- 172.21.15.67:0/3259096875 <== mon.1 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb53c006030 con 0x7fb54815f1f0 2023-10-14T10:36:32.414 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.408+0000 7fb5467fc700 1 -- 172.21.15.67:0/3259096875 <== mon.1 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb53c0063c0 con 0x7fb54815f1f0 2023-10-14T10:36:32.415 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.408+0000 7fb54e084700 1 -- 172.21.15.67:0/3259096875 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb54815f1f0 msgr2=0x7fb5480cf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:32.416 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.408+0000 7fb54e084700 1 --2- 172.21.15.67:0/3259096875 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb54815f1f0 0x7fb5480cf720 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb53c009820 tx=0x7fb53c009b30 comp rx=0 tx=0).stop 2023-10-14T10:36:32.416 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.408+0000 7fb54e084700 1 -- 172.21.15.67:0/3259096875 shutdown_connections 2023-10-14T10:36:32.416 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.408+0000 7fb54e084700 1 --2- 172.21.15.67:0/3259096875 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fb5480cfc60 0x7fb5480cbf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:32.416 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.408+0000 7fb54e084700 1 --2- 172.21.15.67:0/3259096875 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb54815f1f0 0x7fb5480cf720 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:32.417 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.408+0000 7fb54e084700 1 -- 172.21.15.67:0/3259096875 >> 172.21.15.67:0/3259096875 conn(0x7fb54806dae0 msgr2=0x7fb54806def0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:32.417 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.408+0000 7fb54e084700 1 -- 172.21.15.67:0/3259096875 shutdown_connections 2023-10-14T10:36:32.417 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.408+0000 7fb54e084700 1 -- 172.21.15.67:0/3259096875 wait complete. 2023-10-14T10:36:32.420 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.410+0000 7fb54e084700 1 Processor -- start 2023-10-14T10:36:32.421 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.410+0000 7fb54e084700 1 -- start start 2023-10-14T10:36:32.421 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.411+0000 7fb54e084700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb5480cfc60 0x7fb548152910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:32.421 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.411+0000 7fb54e084700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fb54815ab10 0x7fb5481f0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:32.421 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.411+0000 7fb54e084700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb54815f1f0 0x7fb5480c75a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:32.421 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.411+0000 7fb54e084700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5481f14f0 con 0x7fb5480cfc60 2023-10-14T10:36:32.422 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.411+0000 7fb54e084700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb548152ef0 con 0x7fb54815f1f0 2023-10-14T10:36:32.422 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.411+0000 7fb54e084700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5481531d0 con 0x7fb54815ab10 2023-10-14T10:36:32.422 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.411+0000 7fb5477fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb5480cfc60 0x7fb548152910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:32.422 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.411+0000 7fb5477fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb5480cfc60 0x7fb548152910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44838/0 (socket says 172.21.15.67:44838) 2023-10-14T10:36:32.422 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.411+0000 7fb5477fe700 1 -- 172.21.15.67:0/2168880485 learned_addr learned my addr 172.21.15.67:0/2168880485 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:32.423 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.411+0000 7fb547fff700 1 --2- 172.21.15.67:0/2168880485 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb54815f1f0 0x7fb5480c75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:32.423 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.411+0000 7fb546ffd700 1 --2- 172.21.15.67:0/2168880485 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fb54815ab10 0x7fb5481f0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:32.423 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.411+0000 7fb5477fe700 1 -- 172.21.15.67:0/2168880485 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fb54815ab10 msgr2=0x7fb5481f0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:32.423 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.411+0000 7fb5477fe700 1 --2- 172.21.15.67:0/2168880485 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fb54815ab10 0x7fb5481f0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:32.423 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.411+0000 7fb5477fe700 1 -- 172.21.15.67:0/2168880485 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb54815f1f0 msgr2=0x7fb5480c75a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:32.424 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.411+0000 7fb5477fe700 1 --2- 172.21.15.67:0/2168880485 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb54815f1f0 0x7fb5480c75a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:32.424 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.411+0000 7fb5477fe700 1 -- 172.21.15.67:0/2168880485 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb540009a70 con 0x7fb5480cfc60 2023-10-14T10:36:32.424 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.411+0000 7fb547fff700 1 --2- 172.21.15.67:0/2168880485 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb54815f1f0 0x7fb5480c75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-14T10:36:32.424 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.411+0000 7fb546ffd700 1 --2- 172.21.15.67:0/2168880485 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fb54815ab10 0x7fb5481f0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-14T10:36:32.424 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.411+0000 7fb5477fe700 1 --2- 172.21.15.67:0/2168880485 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb5480cfc60 0x7fb548152910 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb538001eb0 tx=0x7fb53800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:32.425 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.412+0000 7fb52ffff700 1 -- 172.21.15.67:0/2168880485 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb538005d70 con 0x7fb5480cfc60 2023-10-14T10:36:32.425 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.412+0000 7fb52ffff700 1 -- 172.21.15.67:0/2168880485 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5380070d0 con 0x7fb5480cfc60 2023-10-14T10:36:32.425 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.412+0000 7fb54e084700 1 -- 172.21.15.67:0/2168880485 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb53c009430 con 0x7fb5480cfc60 2023-10-14T10:36:32.425 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.412+0000 7fb52ffff700 1 -- 172.21.15.67:0/2168880485 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb538007460 con 0x7fb5480cfc60 2023-10-14T10:36:32.425 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.412+0000 7fb54e084700 1 -- 172.21.15.67:0/2168880485 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb548153790 con 0x7fb5480cfc60 2023-10-14T10:36:32.426 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.413+0000 7fb54e084700 1 -- 172.21.15.67:0/2168880485 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb548067d20 con 0x7fb5480cfc60 2023-10-14T10:36:32.426 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.413+0000 7fb52ffff700 1 -- 172.21.15.67:0/2168880485 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fb5380075c0 con 0x7fb5480cfc60 2023-10-14T10:36:32.426 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.413+0000 7fb52ffff700 1 --2- 172.21.15.67:0/2168880485 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fb53406af00 0x7fb53406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:32.426 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.414+0000 7fb52ffff700 1 -- 172.21.15.67:0/2168880485 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb538088a90 con 0x7fb5480cfc60 2023-10-14T10:36:32.426 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.414+0000 7fb546ffd700 1 --2- 172.21.15.67:0/2168880485 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fb53406af00 0x7fb53406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:32.426 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.414+0000 7fb546ffd700 1 --2- 172.21.15.67:0/2168880485 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fb53406af00 0x7fb53406d3c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb53c013040 tx=0x7fb53c008070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:32.427 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.418+0000 7fb52ffff700 1 -- 172.21.15.67:0/2168880485 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb538053bd0 con 0x7fb5480cfc60 2023-10-14T10:36:32.620 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.617+0000 7fb54e084700 1 -- 172.21.15.67:0/2168880485 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb54815e090 con 0x7fb5480cfc60 2023-10-14T10:36:32.620 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.618+0000 7fb52ffff700 1 -- 172.21.15.67:0/2168880485 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb5380170a0 con 0x7fb5480cfc60 2023-10-14T10:36:32.620 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2023-10-14T10:36:32.622 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.620+0000 7fb54e084700 1 -- 172.21.15.67:0/2168880485 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fb53406af00 msgr2=0x7fb53406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:32.623 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.620+0000 7fb54e084700 1 --2- 172.21.15.67:0/2168880485 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fb53406af00 0x7fb53406d3c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb53c013040 tx=0x7fb53c008070 comp rx=0 tx=0).stop 2023-10-14T10:36:32.623 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.620+0000 7fb54e084700 1 -- 172.21.15.67:0/2168880485 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb5480cfc60 msgr2=0x7fb548152910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:32.623 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.620+0000 7fb54e084700 1 --2- 172.21.15.67:0/2168880485 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb5480cfc60 0x7fb548152910 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb538001eb0 tx=0x7fb53800c860 comp rx=0 tx=0).stop 2023-10-14T10:36:32.623 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.620+0000 7fb54e084700 1 -- 172.21.15.67:0/2168880485 shutdown_connections 2023-10-14T10:36:32.623 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.620+0000 7fb54e084700 1 --2- 172.21.15.67:0/2168880485 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fb53406af00 0x7fb53406d3c0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:32.624 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.620+0000 7fb54e084700 1 --2- 172.21.15.67:0/2168880485 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb54815f1f0 0x7fb5480c75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:32.624 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.620+0000 7fb54e084700 1 --2- 172.21.15.67:0/2168880485 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb5480cfc60 0x7fb548152910 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:32.624 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.620+0000 7fb54e084700 1 --2- 172.21.15.67:0/2168880485 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fb54815ab10 0x7fb5481f0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:32.624 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.620+0000 7fb54e084700 1 -- 172.21.15.67:0/2168880485 >> 172.21.15.67:0/2168880485 conn(0x7fb54806dae0 msgr2=0x7fb548159960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:32.624 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.620+0000 7fb54e084700 1 -- 172.21.15.67:0/2168880485 shutdown_connections 2023-10-14T10:36:32.625 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.620+0000 7fb54e084700 1 -- 172.21.15.67:0/2168880485 wait complete. 2023-10-14T10:36:32.634 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-14T10:36:32.762 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.760+0000 7f1be5cfa700 1 Processor -- start 2023-10-14T10:36:32.762 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.760+0000 7f1be5cfa700 1 -- start start 2023-10-14T10:36:32.764 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.760+0000 7f1be5cfa700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f1be015e460 0x7f1be01650e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:32.764 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.760+0000 7f1be5cfa700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1be0158120 con 0x7f1be0157d40 2023-10-14T10:36:32.765 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.760+0000 7f1be5cfa700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1be006a840 con 0x7f1be00cee90 2023-10-14T10:36:32.765 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.760+0000 7f1be5cfa700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1be006e560 con 0x7f1be015e460 2023-10-14T10:36:32.765 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.761+0000 7f1bdffff700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f1be015e460 0x7f1be01650e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:32.765 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.761+0000 7f1bdf7fe700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f1be0157d40 0x7f1be006de50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:52614/0 (socket says 172.21.15.67:52614) 2023-10-14T10:36:32.765 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.761+0000 7f1bdf7fe700 1 -- 172.21.15.67:0/2185363354 learned_addr learned my addr 172.21.15.67:0/2185363354 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:36:32.766 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.761+0000 7f1bdffff700 1 -- 172.21.15.67:0/2185363354 >> v1:172.21.15.72:6789/0 conn(0x7f1be00cee90 legacy=0x7f1be015dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:32.766 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.761+0000 7f1bdffff700 1 -- 172.21.15.67:0/2185363354 >> v1:172.21.15.67:6789/0 conn(0x7f1be0157d40 legacy=0x7f1be006de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:32.766 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.761+0000 7f1bdffff700 1 -- 172.21.15.67:0/2185363354 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1be00caef0 con 0x7f1be015e460 2023-10-14T10:36:32.766 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.761+0000 7f1bdffff700 1 --2- 172.21.15.67:0/2185363354 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1be015e460 0x7f1be01650e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1bd8009d90 tx=0x7f1bd800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b1bac6ddf3d7570 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:32.766 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.761+0000 7f1bde7fc700 1 -- 172.21.15.67:0/2185363354 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bd800bb90 con 0x7f1be015e460 2023-10-14T10:36:32.767 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.761+0000 7f1bde7fc700 1 -- 172.21.15.67:0/2185363354 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bd800bcf0 con 0x7f1be015e460 2023-10-14T10:36:32.767 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.761+0000 7f1bde7fc700 1 -- 172.21.15.67:0/2185363354 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bd8003c60 con 0x7f1be015e460 2023-10-14T10:36:32.767 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.762+0000 7f1be5cfa700 1 -- 172.21.15.67:0/2185363354 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1be015e460 msgr2=0x7f1be01650e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:32.767 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.762+0000 7f1be5cfa700 1 --2- 172.21.15.67:0/2185363354 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1be015e460 0x7f1be01650e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1bd8009d90 tx=0x7f1bd800b330 comp rx=0 tx=0).stop 2023-10-14T10:36:32.767 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.762+0000 7f1be5cfa700 1 -- 172.21.15.67:0/2185363354 shutdown_connections 2023-10-14T10:36:32.768 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.762+0000 7f1be5cfa700 1 --2- 172.21.15.67:0/2185363354 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1be015e460 0x7f1be01650e0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:32.768 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.762+0000 7f1be5cfa700 1 -- 172.21.15.67:0/2185363354 >> 172.21.15.67:0/2185363354 conn(0x7f1be00cc6f0 msgr2=0x7f1be00ccb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:32.768 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.762+0000 7f1be5cfa700 1 -- 172.21.15.67:0/2185363354 shutdown_connections 2023-10-14T10:36:32.768 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.762+0000 7f1be5cfa700 1 -- 172.21.15.67:0/2185363354 wait complete. 2023-10-14T10:36:32.771 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.763+0000 7f1be5cfa700 1 Processor -- start 2023-10-14T10:36:32.771 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.764+0000 7f1be5cfa700 1 -- start start 2023-10-14T10:36:32.771 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.765+0000 7f1be5cfa700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1be00cee90 0x7f1be0146ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:32.771 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.765+0000 7f1be5cfa700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1be0157d40 0x7f1be014a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:32.772 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.765+0000 7f1be5cfa700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f1be015e460 0x7f1be00c8f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:32.772 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.765+0000 7f1be5cfa700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1be00c6bb0 con 0x7f1be00cee90 2023-10-14T10:36:32.772 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.765+0000 7f1be5cfa700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1be0147530 con 0x7f1be0157d40 2023-10-14T10:36:32.772 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.765+0000 7f1be5cfa700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1be00c6720 con 0x7f1be015e460 2023-10-14T10:36:32.772 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.765+0000 7f1bdf7fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1be00cee90 0x7f1be0146ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:32.773 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.765+0000 7f1bdf7fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1be00cee90 0x7f1be0146ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44852/0 (socket says 172.21.15.67:44852) 2023-10-14T10:36:32.773 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.765+0000 7f1bdf7fe700 1 -- 172.21.15.67:0/3508803492 learned_addr learned my addr 172.21.15.67:0/3508803492 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:32.773 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.765+0000 7f1bdeffd700 1 --2- 172.21.15.67:0/3508803492 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1be0157d40 0x7f1be014a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:32.773 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.765+0000 7f1bdffff700 1 --2- 172.21.15.67:0/3508803492 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f1be015e460 0x7f1be00c8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:32.774 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.765+0000 7f1bdf7fe700 1 -- 172.21.15.67:0/3508803492 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f1be015e460 msgr2=0x7f1be00c8f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:32.774 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.765+0000 7f1bdf7fe700 1 --2- 172.21.15.67:0/3508803492 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f1be015e460 0x7f1be00c8f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:32.774 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.765+0000 7f1bdf7fe700 1 -- 172.21.15.67:0/3508803492 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1be0157d40 msgr2=0x7f1be014a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:32.774 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.765+0000 7f1bdf7fe700 1 --2- 172.21.15.67:0/3508803492 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1be0157d40 0x7f1be014a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:32.774 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.765+0000 7f1bdf7fe700 1 -- 172.21.15.67:0/3508803492 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bd8009a70 con 0x7f1be00cee90 2023-10-14T10:36:32.775 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.765+0000 7f1bdf7fe700 1 --2- 172.21.15.67:0/3508803492 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1be00cee90 0x7f1be0146ff0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1bd000c490 tx=0x7f1bd000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:32.775 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.765+0000 7f1bc7fff700 1 -- 172.21.15.67:0/3508803492 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bd0005d70 con 0x7f1be00cee90 2023-10-14T10:36:32.775 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.765+0000 7f1bc7fff700 1 -- 172.21.15.67:0/3508803492 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bd00070d0 con 0x7f1be00cee90 2023-10-14T10:36:32.775 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.765+0000 7f1bc7fff700 1 -- 172.21.15.67:0/3508803492 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bd0007460 con 0x7f1be00cee90 2023-10-14T10:36:32.775 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.765+0000 7f1be5cfa700 1 -- 172.21.15.67:0/3508803492 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1be014a7a0 con 0x7f1be00cee90 2023-10-14T10:36:32.776 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.765+0000 7f1be5cfa700 1 -- 172.21.15.67:0/3508803492 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1be00c60d0 con 0x7f1be00cee90 2023-10-14T10:36:32.776 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.766+0000 7f1be5cfa700 1 -- 172.21.15.67:0/3508803492 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1be0067d20 con 0x7f1be00cee90 2023-10-14T10:36:32.776 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.767+0000 7f1bc7fff700 1 -- 172.21.15.67:0/3508803492 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f1bd00075c0 con 0x7f1be00cee90 2023-10-14T10:36:32.776 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.767+0000 7f1bc7fff700 1 --2- 172.21.15.67:0/3508803492 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f1bcc06af00 0x7f1bcc06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:32.776 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.767+0000 7f1bc7fff700 1 -- 172.21.15.67:0/3508803492 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1bd0088ad0 con 0x7f1be00cee90 2023-10-14T10:36:32.777 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.768+0000 7f1bdeffd700 1 --2- 172.21.15.67:0/3508803492 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f1bcc06af00 0x7f1bcc06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:32.777 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.768+0000 7f1bdeffd700 1 --2- 172.21.15.67:0/3508803492 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f1bcc06af00 0x7f1bcc06d3c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1be0158fe0 tx=0x7f1bd4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:32.777 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.772+0000 7f1bc7fff700 1 -- 172.21.15.67:0/3508803492 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1bd0053b80 con 0x7f1be00cee90 2023-10-14T10:36:32.975 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.973+0000 7f1be5cfa700 1 -- 172.21.15.67:0/3508803492 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1be006d6e0 con 0x7f1be00cee90 2023-10-14T10:36:32.976 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.973+0000 7f1bc7fff700 1 -- 172.21.15.67:0/3508803492 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1bd00170a0 con 0x7f1be00cee90 2023-10-14T10:36:32.976 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2023-10-14T10:36:32.978 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.975+0000 7f1be5cfa700 1 -- 172.21.15.67:0/3508803492 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f1bcc06af00 msgr2=0x7f1bcc06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:32.978 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.976+0000 7f1be5cfa700 1 --2- 172.21.15.67:0/3508803492 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f1bcc06af00 0x7f1bcc06d3c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1be0158fe0 tx=0x7f1bd4009040 comp rx=0 tx=0).stop 2023-10-14T10:36:32.978 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.976+0000 7f1be5cfa700 1 -- 172.21.15.67:0/3508803492 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1be00cee90 msgr2=0x7f1be0146ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:32.979 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.976+0000 7f1be5cfa700 1 --2- 172.21.15.67:0/3508803492 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1be00cee90 0x7f1be0146ff0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1bd000c490 tx=0x7f1bd000c850 comp rx=0 tx=0).stop 2023-10-14T10:36:32.979 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.976+0000 7f1be5cfa700 1 -- 172.21.15.67:0/3508803492 shutdown_connections 2023-10-14T10:36:32.979 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.976+0000 7f1be5cfa700 1 --2- 172.21.15.67:0/3508803492 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f1bcc06af00 0x7f1bcc06d3c0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:32.979 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.976+0000 7f1be5cfa700 1 --2- 172.21.15.67:0/3508803492 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f1be015e460 0x7f1be00c8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:32.979 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.976+0000 7f1be5cfa700 1 --2- 172.21.15.67:0/3508803492 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1be00cee90 0x7f1be0146ff0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:32.980 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.976+0000 7f1be5cfa700 1 --2- 172.21.15.67:0/3508803492 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1be0157d40 0x7f1be014a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:32.980 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.976+0000 7f1be5cfa700 1 -- 172.21.15.67:0/3508803492 >> 172.21.15.67:0/3508803492 conn(0x7f1be00cc6f0 msgr2=0x7f1be01572a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:32.980 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.976+0000 7f1be5cfa700 1 -- 172.21.15.67:0/3508803492 shutdown_connections 2023-10-14T10:36:32.980 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:32.976+0000 7f1be5cfa700 1 -- 172.21.15.67:0/3508803492 wait complete. 2023-10-14T10:36:32.990 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-14T10:36:33.119 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.116+0000 7f4dbe59b700 1 Processor -- start 2023-10-14T10:36:33.119 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.116+0000 7f4dbe59b700 1 -- start start 2023-10-14T10:36:33.121 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.117+0000 7f4dbe59b700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f4db815f1f0 0x7f4db80cf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:33.121 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.117+0000 7f4dbe59b700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4db815e7a0 con 0x7f4db80cfc60 2023-10-14T10:36:33.122 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.117+0000 7f4dbe59b700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4db80d0090 con 0x7f4db815ab10 2023-10-14T10:36:33.122 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.117+0000 7f4dbe59b700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4db80b29a0 con 0x7f4db815f1f0 2023-10-14T10:36:33.122 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.117+0000 7f4dbcb38700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f4db80cfc60 0x7f4db80cbf40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:52632/0 (socket says 172.21.15.67:52632) 2023-10-14T10:36:33.122 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.117+0000 7f4dbcb38700 1 -- 172.21.15.67:0/43753596 learned_addr learned my addr 172.21.15.67:0/43753596 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:36:33.122 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.118+0000 7f4db77fe700 1 --2- 172.21.15.67:0/43753596 >> v2:172.21.15.72:3300/0 conn(0x7f4db815f1f0 0x7f4db80cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:33.123 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.118+0000 7f4db6ffd700 1 -- 172.21.15.67:0/43753596 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4256393194 0 0) 0x7f4db815e7a0 con 0x7f4db80cfc60 2023-10-14T10:36:33.123 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.118+0000 7f4db6ffd700 1 -- 172.21.15.67:0/43753596 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4da40035b0 con 0x7f4db80cfc60 2023-10-14T10:36:33.123 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.118+0000 7f4db6ffd700 1 -- 172.21.15.67:0/43753596 <== mon.1 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 228067358 0 0) 0x7f4db80d0090 con 0x7f4db815ab10 2023-10-14T10:36:33.123 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.118+0000 7f4db6ffd700 1 -- 172.21.15.67:0/43753596 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4db815e7a0 con 0x7f4db815ab10 2023-10-14T10:36:33.123 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.118+0000 7f4db77fe700 1 -- 172.21.15.67:0/43753596 >> v1:172.21.15.72:6789/0 conn(0x7f4db815ab10 legacy=0x7f4db80b1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:33.124 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.118+0000 7f4db6ffd700 1 -- 172.21.15.67:0/43753596 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4190105741 0 0) 0x7f4da40035b0 con 0x7f4db80cfc60 2023-10-14T10:36:33.124 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.118+0000 7f4db77fe700 1 -- 172.21.15.67:0/43753596 >> v1:172.21.15.67:6789/0 conn(0x7f4db80cfc60 legacy=0x7f4db80cbf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:33.124 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.118+0000 7f4db77fe700 1 -- 172.21.15.67:0/43753596 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4db80b2c80 con 0x7f4db815f1f0 2023-10-14T10:36:33.124 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.118+0000 7f4db77fe700 1 --2- 172.21.15.67:0/43753596 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4db815f1f0 0x7f4db80cf720 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4da0009820 tx=0x7f4da0009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a41ab7e29bd4c47e server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:33.124 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.118+0000 7f4db6ffd700 1 -- 172.21.15.67:0/43753596 <== mon.1 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4db00019f0 con 0x7f4db815f1f0 2023-10-14T10:36:33.125 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.118+0000 7f4db6ffd700 1 -- 172.21.15.67:0/43753596 <== mon.1 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4da0005c30 con 0x7f4db815f1f0 2023-10-14T10:36:33.125 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.118+0000 7f4db6ffd700 1 -- 172.21.15.67:0/43753596 <== mon.1 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4da0004030 con 0x7f4db815f1f0 2023-10-14T10:36:33.125 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.119+0000 7f4dbe59b700 1 -- 172.21.15.67:0/43753596 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4db815f1f0 msgr2=0x7f4db80cf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:33.125 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.119+0000 7f4dbe59b700 1 --2- 172.21.15.67:0/43753596 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4db815f1f0 0x7f4db80cf720 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4da0009820 tx=0x7f4da0009b30 comp rx=0 tx=0).stop 2023-10-14T10:36:33.125 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.119+0000 7f4dbe59b700 1 -- 172.21.15.67:0/43753596 shutdown_connections 2023-10-14T10:36:33.125 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.119+0000 7f4dbe59b700 1 --2- 172.21.15.67:0/43753596 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4db815f1f0 0x7f4db80cf720 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:33.126 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.119+0000 7f4dbe59b700 1 -- 172.21.15.67:0/43753596 >> 172.21.15.67:0/43753596 conn(0x7f4db806dae0 msgr2=0x7f4db806def0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:33.126 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.119+0000 7f4dbe59b700 1 -- 172.21.15.67:0/43753596 shutdown_connections 2023-10-14T10:36:33.126 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.119+0000 7f4dbe59b700 1 -- 172.21.15.67:0/43753596 wait complete. 2023-10-14T10:36:33.129 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.120+0000 7f4dbe59b700 1 Processor -- start 2023-10-14T10:36:33.129 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.120+0000 7f4dbe59b700 1 -- start start 2023-10-14T10:36:33.129 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.121+0000 7f4dbe59b700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4db80cfc60 0x7f4db8154880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:33.130 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.121+0000 7f4dbe59b700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4db815ab10 0x7f4db80c8f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:33.130 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.121+0000 7f4dbe59b700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f4db815f1f0 0x7f4db81f4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:33.130 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.121+0000 7f4dbe59b700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4db80c9810 con 0x7f4db815ab10 2023-10-14T10:36:33.130 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.121+0000 7f4dbe59b700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4db815aef0 con 0x7f4db80cfc60 2023-10-14T10:36:33.130 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.121+0000 7f4dbe59b700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4db815ba30 con 0x7f4db815f1f0 2023-10-14T10:36:33.131 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.121+0000 7f4db77fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4db815ab10 0x7f4db80c8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:33.131 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.121+0000 7f4db77fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4db815ab10 0x7f4db80c8f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44868/0 (socket says 172.21.15.67:44868) 2023-10-14T10:36:33.131 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.121+0000 7f4db77fe700 1 -- 172.21.15.67:0/3106803466 learned_addr learned my addr 172.21.15.67:0/3106803466 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:33.131 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.122+0000 7f4dbcb38700 1 --2- 172.21.15.67:0/3106803466 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f4db815f1f0 0x7f4db81f4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:33.131 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.122+0000 7f4db77fe700 1 -- 172.21.15.67:0/3106803466 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f4db815f1f0 msgr2=0x7f4db81f4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:33.132 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.122+0000 7f4db7fff700 1 --2- 172.21.15.67:0/3106803466 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4db80cfc60 0x7f4db8154880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:33.132 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.122+0000 7f4db77fe700 1 --2- 172.21.15.67:0/3106803466 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f4db815f1f0 0x7f4db81f4740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:33.132 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.122+0000 7f4db77fe700 1 -- 172.21.15.67:0/3106803466 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4db80cfc60 msgr2=0x7f4db8154880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:33.132 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.122+0000 7f4db77fe700 1 --2- 172.21.15.67:0/3106803466 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4db80cfc60 0x7f4db8154880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:33.132 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.122+0000 7f4db77fe700 1 -- 172.21.15.67:0/3106803466 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4da0009430 con 0x7f4db815ab10 2023-10-14T10:36:33.133 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.122+0000 7f4db7fff700 1 --2- 172.21.15.67:0/3106803466 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4db80cfc60 0x7f4db8154880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-14T10:36:33.133 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.122+0000 7f4db77fe700 1 --2- 172.21.15.67:0/3106803466 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4db815ab10 0x7f4db80c8f90 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4da0009e80 tx=0x7f4da0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:33.133 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.122+0000 7f4db4ff9700 1 -- 172.21.15.67:0/3106803466 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4da0003bb0 con 0x7f4db815ab10 2023-10-14T10:36:33.133 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.122+0000 7f4db4ff9700 1 -- 172.21.15.67:0/3106803466 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4da0003d10 con 0x7f4db815ab10 2023-10-14T10:36:33.133 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.122+0000 7f4db4ff9700 1 -- 172.21.15.67:0/3106803466 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4da001d650 con 0x7f4db815ab10 2023-10-14T10:36:33.134 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.122+0000 7f4dbe59b700 1 -- 172.21.15.67:0/3106803466 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4db815b100 con 0x7f4db815ab10 2023-10-14T10:36:33.134 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.122+0000 7f4dbe59b700 1 -- 172.21.15.67:0/3106803466 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4db815b6c0 con 0x7f4db815ab10 2023-10-14T10:36:33.134 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.123+0000 7f4dbe59b700 1 -- 172.21.15.67:0/3106803466 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4db8067d20 con 0x7f4db815ab10 2023-10-14T10:36:33.134 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.123+0000 7f4db4ff9700 1 -- 172.21.15.67:0/3106803466 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f4da001d7b0 con 0x7f4db815ab10 2023-10-14T10:36:33.134 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.124+0000 7f4db4ff9700 1 --2- 172.21.15.67:0/3106803466 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f4d9806af00 0x7f4d9806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:33.135 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.124+0000 7f4db4ff9700 1 -- 172.21.15.67:0/3106803466 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4da008a550 con 0x7f4db815ab10 2023-10-14T10:36:33.135 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.124+0000 7f4db7fff700 1 --2- 172.21.15.67:0/3106803466 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f4d9806af00 0x7f4d9806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:33.135 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.125+0000 7f4db7fff700 1 --2- 172.21.15.67:0/3106803466 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f4d9806af00 0x7f4d9806d3c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4da80011e0 tx=0x7f4da8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:33.135 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.128+0000 7f4db4ff9700 1 -- 172.21.15.67:0/3106803466 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4da0055690 con 0x7f4db815ab10 2023-10-14T10:36:33.331 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.329+0000 7f4dbe59b700 1 -- 172.21.15.67:0/3106803466 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4db815bb70 con 0x7f4db815ab10 2023-10-14T10:36:33.331 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.329+0000 7f4db4ff9700 1 -- 172.21.15.67:0/3106803466 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4da0058cb0 con 0x7f4db815ab10 2023-10-14T10:36:33.332 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2023-10-14T10:36:33.334 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.331+0000 7f4dbe59b700 1 -- 172.21.15.67:0/3106803466 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f4d9806af00 msgr2=0x7f4d9806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:33.334 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.331+0000 7f4dbe59b700 1 --2- 172.21.15.67:0/3106803466 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f4d9806af00 0x7f4d9806d3c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4da80011e0 tx=0x7f4da8009040 comp rx=0 tx=0).stop 2023-10-14T10:36:33.334 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.331+0000 7f4dbe59b700 1 -- 172.21.15.67:0/3106803466 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4db815ab10 msgr2=0x7f4db80c8f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:33.334 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.331+0000 7f4dbe59b700 1 --2- 172.21.15.67:0/3106803466 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4db815ab10 0x7f4db80c8f90 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4da0009e80 tx=0x7f4da0004750 comp rx=0 tx=0).stop 2023-10-14T10:36:33.335 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.331+0000 7f4dbe59b700 1 -- 172.21.15.67:0/3106803466 shutdown_connections 2023-10-14T10:36:33.335 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.331+0000 7f4dbe59b700 1 --2- 172.21.15.67:0/3106803466 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f4d9806af00 0x7f4d9806d3c0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:33.335 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.331+0000 7f4dbe59b700 1 --2- 172.21.15.67:0/3106803466 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f4db815f1f0 0x7f4db81f4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:33.335 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.331+0000 7f4dbe59b700 1 --2- 172.21.15.67:0/3106803466 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4db80cfc60 0x7f4db8154880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:33.335 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.331+0000 7f4dbe59b700 1 --2- 172.21.15.67:0/3106803466 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4db815ab10 0x7f4db80c8f90 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:33.336 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.331+0000 7f4dbe59b700 1 -- 172.21.15.67:0/3106803466 >> 172.21.15.67:0/3106803466 conn(0x7f4db806dae0 msgr2=0x7f4db8159cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:33.336 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.332+0000 7f4dbe59b700 1 -- 172.21.15.67:0/3106803466 shutdown_connections 2023-10-14T10:36:33.336 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.332+0000 7f4dbe59b700 1 -- 172.21.15.67:0/3106803466 wait complete. 2023-10-14T10:36:33.445 INFO:tasks.cephfs_test_runner:Starting test: test_volume_create (tasks.cephfs.test_volumes.TestVolumes) That the volume can be created and then cleans up 2023-10-14T10:36:33.446 INFO:tasks.cephfs_test_runner:test_volume_create (tasks.cephfs.test_volumes.TestVolumes) 2023-10-14T10:36:33.447 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-14T10:36:33.573 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.570+0000 7f995bb53700 1 Processor -- start 2023-10-14T10:36:33.573 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.571+0000 7f995bb53700 1 -- start start 2023-10-14T10:36:33.575 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.571+0000 7f995bb53700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f99540d0c70 0x7f99540cdfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:33.575 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.571+0000 7f995bb53700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99540d23e0 con 0x7f995415b740 2023-10-14T10:36:33.575 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.571+0000 7f995bb53700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99540cf190 con 0x7f99540c5110 2023-10-14T10:36:33.575 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.571+0000 7f995bb53700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f995415bb20 con 0x7f99540d0c70 2023-10-14T10:36:33.576 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.572+0000 7f99590ee700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f99540d0c70 0x7f99540cdfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:33.576 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.572+0000 7f99598ef700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f995415b740 0x7f99540d1cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:52642/0 (socket says 172.21.15.67:52642) 2023-10-14T10:36:33.576 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.572+0000 7f99598ef700 1 -- 172.21.15.67:0/2904060879 learned_addr learned my addr 172.21.15.67:0/2904060879 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:36:33.576 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.572+0000 7f99590ee700 1 -- 172.21.15.67:0/2904060879 >> v1:172.21.15.72:6789/0 conn(0x7f99540c5110 legacy=0x7f99540cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:33.577 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.572+0000 7f99590ee700 1 -- 172.21.15.67:0/2904060879 >> v1:172.21.15.67:6789/0 conn(0x7f995415b740 legacy=0x7f99540d1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:33.577 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.572+0000 7f99590ee700 1 -- 172.21.15.67:0/2904060879 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99540cbd40 con 0x7f99540d0c70 2023-10-14T10:36:33.577 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.572+0000 7f99588ed700 1 -- 172.21.15.67:0/2904060879 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3894693284 0 0) 0x7f9948001110 con 0x7f995415b740 2023-10-14T10:36:33.577 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.572+0000 7f99590ee700 1 --2- 172.21.15.67:0/2904060879 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99540d0c70 0x7f99540cdfd0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9950009840 tx=0x7f9950009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fcf822f819ebf0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:33.577 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.572+0000 7f99588ed700 1 -- 172.21.15.67:0/2904060879 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9950011830 con 0x7f99540d0c70 2023-10-14T10:36:33.578 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.572+0000 7f99588ed700 1 -- 172.21.15.67:0/2904060879 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9950011990 con 0x7f99540d0c70 2023-10-14T10:36:33.578 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.572+0000 7f99588ed700 1 -- 172.21.15.67:0/2904060879 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9950011d20 con 0x7f99540d0c70 2023-10-14T10:36:33.578 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.573+0000 7f995bb53700 1 -- 172.21.15.67:0/2904060879 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99540d0c70 msgr2=0x7f99540cdfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:33.578 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.573+0000 7f995bb53700 1 --2- 172.21.15.67:0/2904060879 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99540d0c70 0x7f99540cdfd0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9950009840 tx=0x7f9950009b50 comp rx=0 tx=0).stop 2023-10-14T10:36:33.578 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.573+0000 7f995bb53700 1 -- 172.21.15.67:0/2904060879 shutdown_connections 2023-10-14T10:36:33.579 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.573+0000 7f995bb53700 1 --2- 172.21.15.67:0/2904060879 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99540d0c70 0x7f99540cdfd0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:33.579 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.573+0000 7f995bb53700 1 -- 172.21.15.67:0/2904060879 >> 172.21.15.67:0/2904060879 conn(0x7f99540665a0 msgr2=0x7f9954066980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:33.579 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.573+0000 7f995bb53700 1 -- 172.21.15.67:0/2904060879 shutdown_connections 2023-10-14T10:36:33.579 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.573+0000 7f995bb53700 1 -- 172.21.15.67:0/2904060879 wait complete. 2023-10-14T10:36:33.582 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.574+0000 7f995bb53700 1 Processor -- start 2023-10-14T10:36:33.582 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.574+0000 7f995bb53700 1 -- start start 2023-10-14T10:36:33.582 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.575+0000 7f995bb53700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99540c5110 0x7f99540b6540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:33.583 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.575+0000 7f995bb53700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f99540d0c70 0x7f99541f62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:33.583 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.575+0000 7f995bb53700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f995415b740 0x7f99541f6830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:33.583 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.575+0000 7f995bb53700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99540b4ac0 con 0x7f99540c5110 2023-10-14T10:36:33.583 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.575+0000 7f995bb53700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99540b3eb0 con 0x7f995415b740 2023-10-14T10:36:33.583 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.575+0000 7f995bb53700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99540b4190 con 0x7f99540d0c70 2023-10-14T10:36:33.584 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.575+0000 7f99598ef700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99540c5110 0x7f99540b6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:33.584 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.575+0000 7f99598ef700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99540c5110 0x7f99540b6540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44880/0 (socket says 172.21.15.67:44880) 2023-10-14T10:36:33.584 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.575+0000 7f99598ef700 1 -- 172.21.15.67:0/3472812786 learned_addr learned my addr 172.21.15.67:0/3472812786 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:33.584 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.575+0000 7f99590ee700 1 --2- 172.21.15.67:0/3472812786 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f99540d0c70 0x7f99541f62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:33.584 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.575+0000 7f995a0f0700 1 --2- 172.21.15.67:0/3472812786 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f995415b740 0x7f99541f6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:33.585 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.575+0000 7f99598ef700 1 -- 172.21.15.67:0/3472812786 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f99540d0c70 msgr2=0x7f99541f62f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:33.585 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.575+0000 7f99598ef700 1 --2- 172.21.15.67:0/3472812786 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f99540d0c70 0x7f99541f62f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:33.585 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.575+0000 7f99598ef700 1 -- 172.21.15.67:0/3472812786 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f995415b740 msgr2=0x7f99541f6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:33.585 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.575+0000 7f99598ef700 1 --2- 172.21.15.67:0/3472812786 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f995415b740 0x7f99541f6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:33.585 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.575+0000 7f99598ef700 1 -- 172.21.15.67:0/3472812786 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99500094a0 con 0x7f99540c5110 2023-10-14T10:36:33.586 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.576+0000 7f99598ef700 1 --2- 172.21.15.67:0/3472812786 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99540c5110 0x7f99540b6540 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f994800c520 tx=0x7f994800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:33.586 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.576+0000 7f99467fc700 1 -- 172.21.15.67:0/3472812786 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9948004c80 con 0x7f99540c5110 2023-10-14T10:36:33.586 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.576+0000 7f99467fc700 1 -- 172.21.15.67:0/3472812786 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9948004e20 con 0x7f99540c5110 2023-10-14T10:36:33.586 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.576+0000 7f995bb53700 1 -- 172.21.15.67:0/3472812786 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99540b4470 con 0x7f99540c5110 2023-10-14T10:36:33.586 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.576+0000 7f99467fc700 1 -- 172.21.15.67:0/3472812786 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9948007310 con 0x7f99540c5110 2023-10-14T10:36:33.587 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.576+0000 7f995bb53700 1 -- 172.21.15.67:0/3472812786 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99541f3a00 con 0x7f99540c5110 2023-10-14T10:36:33.587 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.577+0000 7f995bb53700 1 -- 172.21.15.67:0/3472812786 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99541f3bf0 con 0x7f99540c5110 2023-10-14T10:36:33.587 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.577+0000 7f99467fc700 1 -- 172.21.15.67:0/3472812786 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f9948007470 con 0x7f99540c5110 2023-10-14T10:36:33.587 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.578+0000 7f99467fc700 1 --2- 172.21.15.67:0/3472812786 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f993806af00 0x7f993806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:33.587 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.578+0000 7f99467fc700 1 -- 172.21.15.67:0/3472812786 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9948089140 con 0x7f99540c5110 2023-10-14T10:36:33.588 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.578+0000 7f99590ee700 1 --2- 172.21.15.67:0/3472812786 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f993806af00 0x7f993806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:33.588 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.579+0000 7f99590ee700 1 --2- 172.21.15.67:0/3472812786 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f993806af00 0x7f993806d3c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f99500090b0 tx=0x7f9950008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:33.588 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.582+0000 7f99467fc700 1 -- 172.21.15.67:0/3472812786 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f99480541f0 con 0x7f99540c5110 2023-10-14T10:36:33.785 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.783+0000 7f995bb53700 1 -- 172.21.15.67:0/3472812786 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f99540d23e0 con 0x7f99540c5110 2023-10-14T10:36:33.786 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.785+0000 7f99467fc700 1 -- 172.21.15.67:0/3472812786 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16457 (secure 0 0 0) 0x7f99540d23e0 con 0x7f99540c5110 2023-10-14T10:36:33.787 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-14T10:36:33.787 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":22,"fsid":"37f9ab46-3082-4b71-8b24-08d39897b008","created":"2023-10-14T10:35:46.067554+0000","modified":"2023-10-14T10:36:04.837296+0000","last_up_change":"2023-10-14T10:35:56.668142+0000","last_in_change":"2023-10-14T10:35:48.551490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-14T10:35:57.809566+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-14T10:36:01.667146+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-14T10:36:02.219700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2b59a07e-11ef-440c-b766-9f64ac885838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6801","nonce":1287255499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6803","nonce":1287255499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6807","nonce":1287255499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6805","nonce":1287255499}]},"public_addr":"172.21.15.67:6801/1287255499","cluster_addr":"172.21.15.67:6803/1287255499","heartbeat_back_addr":"172.21.15.67:6807/1287255499","heartbeat_front_addr":"172.21.15.67:6805/1287255499","state":["exists","up"]},{"osd":1,"uuid":"6f813bd6-fa0b-487d-a204-431eb1276a60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6818","nonce":2435538787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6822","nonce":2435538787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6830","nonce":2435538787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6826","nonce":2435538787}]},"public_addr":"172.21.15.67:6818/2435538787","cluster_addr":"172.21.15.67:6822/2435538787","heartbeat_back_addr":"172.21.15.67:6830/2435538787","heartbeat_front_addr":"172.21.15.67:6826/2435538787","state":["exists","up"]},{"osd":2,"uuid":"6af42c6a-1b59-4669-9c04-b6876788446d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6809","nonce":2827491791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6811","nonce":2827491791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6815","nonce":2827491791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6813","nonce":2827491791}]},"public_addr":"172.21.15.67:6809/2827491791","cluster_addr":"172.21.15.67:6811/2827491791","heartbeat_back_addr":"172.21.15.67:6815/2827491791","heartbeat_front_addr":"172.21.15.67:6813/2827491791","state":["exists","up"]},{"osd":3,"uuid":"1d889d76-a646-499d-bfff-ed5bae5434ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6819","nonce":1848001176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6821","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6823","nonce":1848001176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6831","nonce":1848001176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6825","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6827","nonce":1848001176}]},"public_addr":"172.21.15.67:6819/1848001176","cluster_addr":"172.21.15.67:6823/1848001176","heartbeat_back_addr":"172.21.15.67:6831/1848001176","heartbeat_front_addr":"172.21.15.67:6827/1848001176","state":["exists","up"]},{"osd":4,"uuid":"c3271dff-b437-4fdd-b13b-6e49a35473af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6804","nonce":1891230785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6808","nonce":1891230785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6815","nonce":1891230785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6811","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6813","nonce":1891230785}]},"public_addr":"172.21.15.72:6804/1891230785","cluster_addr":"172.21.15.72:6808/1891230785","heartbeat_back_addr":"172.21.15.72:6815/1891230785","heartbeat_front_addr":"172.21.15.72:6813/1891230785","state":["exists","up"]},{"osd":5,"uuid":"0337bb18-4b68-4eff-b528-a667275e6926","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6801","nonce":569686991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6803","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6805","nonce":569686991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6812","nonce":569686991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6807","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6809","nonce":569686991}]},"public_addr":"172.21.15.72:6801/569686991","cluster_addr":"172.21.15.72:6805/569686991","heartbeat_back_addr":"172.21.15.72:6812/569686991","heartbeat_front_addr":"172.21.15.72:6809/569686991","state":["exists","up"]},{"osd":6,"uuid":"8e92d9cf-0c8c-46eb-89a7-db2023ef1509","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6817","nonce":3767685934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6819","nonce":3767685934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6823","nonce":3767685934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6821","nonce":3767685934}]},"public_addr":"172.21.15.72:6817/3767685934","cluster_addr":"172.21.15.72:6819/3767685934","heartbeat_back_addr":"172.21.15.72:6823/3767685934","heartbeat_front_addr":"172.21.15.72:6821/3767685934","state":["exists","up"]},{"osd":7,"uuid":"3fd6aa7d-50a9-45c9-9561-5902a044a119","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6825","nonce":3567465926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6827","nonce":3567465926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6831","nonce":3567465926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6829","nonce":3567465926}]},"public_addr":"172.21.15.72:6825/3567465926","cluster_addr":"172.21.15.72:6827/3567465926","heartbeat_back_addr":"172.21.15.72:6831/3567465926","heartbeat_front_addr":"172.21.15.72:6829/3567465926","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.511466+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.795107+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.858674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.728895+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.794842+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.750064+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.732122+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.765185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-14T10:36:33.789 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.786+0000 7f995bb53700 1 -- 172.21.15.67:0/3472812786 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f993806af00 msgr2=0x7f993806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:33.789 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.786+0000 7f995bb53700 1 --2- 172.21.15.67:0/3472812786 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f993806af00 0x7f993806d3c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f99500090b0 tx=0x7f9950008030 comp rx=0 tx=0).stop 2023-10-14T10:36:33.789 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.787+0000 7f995bb53700 1 -- 172.21.15.67:0/3472812786 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99540c5110 msgr2=0x7f99540b6540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:33.789 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.787+0000 7f995bb53700 1 --2- 172.21.15.67:0/3472812786 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99540c5110 0x7f99540b6540 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f994800c520 tx=0x7f994800c8e0 comp rx=0 tx=0).stop 2023-10-14T10:36:33.790 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.787+0000 7f995bb53700 1 -- 172.21.15.67:0/3472812786 shutdown_connections 2023-10-14T10:36:33.790 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.787+0000 7f995bb53700 1 --2- 172.21.15.67:0/3472812786 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f993806af00 0x7f993806d3c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:33.790 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.787+0000 7f995bb53700 1 --2- 172.21.15.67:0/3472812786 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f995415b740 0x7f99541f6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:33.790 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.787+0000 7f995bb53700 1 --2- 172.21.15.67:0/3472812786 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99540c5110 0x7f99540b6540 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:33.790 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.787+0000 7f995bb53700 1 --2- 172.21.15.67:0/3472812786 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f99540d0c70 0x7f99541f62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:33.791 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.787+0000 7f995bb53700 1 -- 172.21.15.67:0/3472812786 >> 172.21.15.67:0/3472812786 conn(0x7f99540665a0 msgr2=0x7f99540ca6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:33.791 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.787+0000 7f995bb53700 1 -- 172.21.15.67:0/3472812786 shutdown_connections 2023-10-14T10:36:33.791 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.787+0000 7f995bb53700 1 -- 172.21.15.67:0/3472812786 wait complete. 2023-10-14T10:36:33.801 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-14T10:35:57.809566+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-14T10:36:01.667146+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-14T10:36:02.219700+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-14T10:36:33.802 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-14T10:36:33.929 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.927+0000 7f1771a18700 1 Processor -- start 2023-10-14T10:36:33.930 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.927+0000 7f1771a18700 1 -- start start 2023-10-14T10:36:33.932 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.928+0000 7f1771a18700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f176c0d3fc0 0x7f176c15dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:33.932 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.928+0000 7f1771a18700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f176c0b7050 0x7f176c0d21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:33.933 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.928+0000 7f1771a18700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f176c15e4e0 con 0x7f176c15dbc0 2023-10-14T10:36:33.933 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.928+0000 7f1771a18700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f176c0b7560 con 0x7f176c0d3fc0 2023-10-14T10:36:33.933 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.928+0000 7f1771a18700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f176c0b7840 con 0x7f176c0b7050 2023-10-14T10:36:33.933 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.928+0000 7f176b7fe700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f176c0b7050 0x7f176c0d21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:33.934 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.928+0000 7f176a7fc700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f176c0d3fc0 0x7f176c15dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:33.934 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.928+0000 7f176a7fc700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f176c0d3fc0 0x7f176c15dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.67:39680/0 (socket says 172.21.15.67:39680) 2023-10-14T10:36:33.934 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.928+0000 7f176b7fe700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f176c0b7050 0x7f176c0d21d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3301/0 says I am v2:172.21.15.67:32942/0 (socket says 172.21.15.67:32942) 2023-10-14T10:36:33.934 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.928+0000 7f176affd700 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f176c15dbc0 0x7f176c0d1ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.67:60896/0 (socket says 172.21.15.67:60896) 2023-10-14T10:36:33.934 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.928+0000 7f176b7fe700 1 -- 172.21.15.67:0/1577690920 learned_addr learned my addr 172.21.15.67:0/1577690920 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:33.935 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.928+0000 7f1769ffb700 1 -- 172.21.15.67:0/1577690920 <== mon.1 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2965305301 0 0) 0x7f176c15e4e0 con 0x7f176c15dbc0 2023-10-14T10:36:33.935 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.928+0000 7f1769ffb700 1 -- 172.21.15.67:0/1577690920 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f17540035b0 con 0x7f176c15dbc0 2023-10-14T10:36:33.935 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.929+0000 7f176b7fe700 1 -- 172.21.15.67:0/1577690920 >> v2:172.21.15.72:3300/0 conn(0x7f176c0d3fc0 msgr2=0x7f176c15dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:33.935 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.929+0000 7f176b7fe700 1 --2- 172.21.15.67:0/1577690920 >> v2:172.21.15.72:3300/0 conn(0x7f176c0d3fc0 0x7f176c15dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:33.935 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.929+0000 7f176b7fe700 1 -- 172.21.15.67:0/1577690920 >> v1:172.21.15.72:6789/0 conn(0x7f176c15dbc0 legacy=0x7f176c0d1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:33.936 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.929+0000 7f176b7fe700 1 -- 172.21.15.67:0/1577690920 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f176c0b7b20 con 0x7f176c0b7050 2023-10-14T10:36:33.936 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.929+0000 7f176a7fc700 1 --2- 172.21.15.67:0/1577690920 >> v2:172.21.15.72:3300/0 conn(0x7f176c0d3fc0 0x7f176c15dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-14T10:36:33.936 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.929+0000 7f176b7fe700 1 --2- 172.21.15.67:0/1577690920 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f176c0b7050 0x7f176c0d21d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1764009e50 tx=0x7f176400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43d232e8aad0fcdf server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:33.936 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.929+0000 7f1769ffb700 1 -- 172.21.15.67:0/1577690920 <== mon.2 v2:172.21.15.72:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f176400e070 con 0x7f176c0b7050 2023-10-14T10:36:33.936 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.929+0000 7f1769ffb700 1 -- 172.21.15.67:0/1577690920 <== mon.2 v2:172.21.15.72:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f176400b910 con 0x7f176c0b7050 2023-10-14T10:36:33.937 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.929+0000 7f1769ffb700 1 -- 172.21.15.67:0/1577690920 <== mon.2 v2:172.21.15.72:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f176400bcc0 con 0x7f176c0b7050 2023-10-14T10:36:33.937 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.929+0000 7f1771a18700 1 -- 172.21.15.67:0/1577690920 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f176c0b7050 msgr2=0x7f176c0d21d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:33.937 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.929+0000 7f1771a18700 1 --2- 172.21.15.67:0/1577690920 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f176c0b7050 0x7f176c0d21d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1764009e50 tx=0x7f176400b330 comp rx=0 tx=0).stop 2023-10-14T10:36:33.937 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.929+0000 7f1771a18700 1 -- 172.21.15.67:0/1577690920 shutdown_connections 2023-10-14T10:36:33.937 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.929+0000 7f1771a18700 1 --2- 172.21.15.67:0/1577690920 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f176c0b7050 0x7f176c0d21d0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:33.938 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.929+0000 7f1771a18700 1 --2- 172.21.15.67:0/1577690920 >> v2:172.21.15.72:3300/0 conn(0x7f176c0d3fc0 0x7f176c15dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:33.938 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.929+0000 7f1771a18700 1 -- 172.21.15.67:0/1577690920 >> 172.21.15.67:0/1577690920 conn(0x7f176c09cb40 msgr2=0x7f176c0d3820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:33.938 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.930+0000 7f1771a18700 1 -- 172.21.15.67:0/1577690920 shutdown_connections 2023-10-14T10:36:33.938 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.930+0000 7f1771a18700 1 -- 172.21.15.67:0/1577690920 wait complete. 2023-10-14T10:36:33.938 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.931+0000 7f1771a18700 1 Processor -- start 2023-10-14T10:36:33.941 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.931+0000 7f1771a18700 1 -- start start 2023-10-14T10:36:33.941 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.932+0000 7f1771a18700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f176c0b7050 0x7f176c0c94a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:33.941 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.932+0000 7f1771a18700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f176c0d3fc0 0x7f176c158f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:33.942 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.932+0000 7f1771a18700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f176c15dbc0 0x7f176c147410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:33.942 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.932+0000 7f1771a18700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f176c15ad10 con 0x7f176c15dbc0 2023-10-14T10:36:33.942 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.932+0000 7f1771a18700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f176c0c99e0 con 0x7f176c0d3fc0 2023-10-14T10:36:33.942 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.932+0000 7f1771a18700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f176c159e40 con 0x7f176c0b7050 2023-10-14T10:36:33.943 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.932+0000 7f176b7fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f176c15dbc0 0x7f176c147410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:33.943 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.932+0000 7f176b7fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f176c15dbc0 0x7f176c147410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44898/0 (socket says 172.21.15.67:44898) 2023-10-14T10:36:33.943 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.932+0000 7f176b7fe700 1 -- 172.21.15.67:0/2878405403 learned_addr learned my addr 172.21.15.67:0/2878405403 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:33.943 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.932+0000 7f176affd700 1 --2- 172.21.15.67:0/2878405403 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f176c0b7050 0x7f176c0c94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:33.943 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.932+0000 7f176a7fc700 1 --2- 172.21.15.67:0/2878405403 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f176c0d3fc0 0x7f176c158f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:33.943 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.932+0000 7f176b7fe700 1 -- 172.21.15.67:0/2878405403 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f176c0b7050 msgr2=0x7f176c0c94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:33.944 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.932+0000 7f176b7fe700 1 --2- 172.21.15.67:0/2878405403 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f176c0b7050 0x7f176c0c94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:33.944 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.932+0000 7f176b7fe700 1 -- 172.21.15.67:0/2878405403 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f176c0d3fc0 msgr2=0x7f176c158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:33.944 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.932+0000 7f176b7fe700 1 --2- 172.21.15.67:0/2878405403 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f176c0d3fc0 0x7f176c158f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:33.944 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.932+0000 7f176b7fe700 1 -- 172.21.15.67:0/2878405403 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f175c0094a0 con 0x7f176c15dbc0 2023-10-14T10:36:33.944 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.932+0000 7f176b7fe700 1 --2- 172.21.15.67:0/2878405403 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f176c15dbc0 0x7f176c147410 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f176c0b7980 tx=0x7f1764004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:33.945 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.932+0000 7f1753fff700 1 -- 172.21.15.67:0/2878405403 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1764004800 con 0x7f176c15dbc0 2023-10-14T10:36:33.945 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.932+0000 7f1753fff700 1 -- 172.21.15.67:0/2878405403 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1764004960 con 0x7f176c15dbc0 2023-10-14T10:36:33.945 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.932+0000 7f1771a18700 1 -- 172.21.15.67:0/2878405403 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1764009b30 con 0x7f176c15dbc0 2023-10-14T10:36:33.945 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.932+0000 7f1753fff700 1 -- 172.21.15.67:0/2878405403 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f176401c680 con 0x7f176c15dbc0 2023-10-14T10:36:33.945 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.932+0000 7f1771a18700 1 -- 172.21.15.67:0/2878405403 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f176c15a090 con 0x7f176c15dbc0 2023-10-14T10:36:33.946 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.933+0000 7f1771a18700 1 -- 172.21.15.67:0/2878405403 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f176c15ae50 con 0x7f176c15dbc0 2023-10-14T10:36:33.946 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.934+0000 7f1753fff700 1 -- 172.21.15.67:0/2878405403 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f176401c7e0 con 0x7f176c15dbc0 2023-10-14T10:36:33.946 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.934+0000 7f1753fff700 1 --2- 172.21.15.67:0/2878405403 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f175806af00 0x7f175806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:33.946 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.934+0000 7f1753fff700 1 -- 172.21.15.67:0/2878405403 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f176408b080 con 0x7f176c15dbc0 2023-10-14T10:36:33.946 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.934+0000 7f176affd700 1 --2- 172.21.15.67:0/2878405403 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f175806af00 0x7f175806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:33.947 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.935+0000 7f176affd700 1 --2- 172.21.15.67:0/2878405403 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f175806af00 0x7f175806d3c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1760001230 tx=0x7f1760007db0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:33.947 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:33.939+0000 7f1753fff700 1 -- 172.21.15.67:0/2878405403 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f176c15ae50 con 0x7f176c15dbc0 2023-10-14T10:36:34.143 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.141+0000 7f1771a18700 1 -- 172.21.15.67:0/2878405403 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f176c15ae50 con 0x7f176c15dbc0 2023-10-14T10:36:34.144 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.142+0000 7f1753fff700 1 -- 172.21.15.67:0/2878405403 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f176c15ae50 con 0x7f176c15dbc0 2023-10-14T10:36:34.144 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2023-10-14T10:36:34.146 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.144+0000 7f1771a18700 1 -- 172.21.15.67:0/2878405403 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f175806af00 msgr2=0x7f175806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:34.146 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.144+0000 7f1771a18700 1 --2- 172.21.15.67:0/2878405403 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f175806af00 0x7f175806d3c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1760001230 tx=0x7f1760007db0 comp rx=0 tx=0).stop 2023-10-14T10:36:34.146 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.144+0000 7f1771a18700 1 -- 172.21.15.67:0/2878405403 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f176c15dbc0 msgr2=0x7f176c147410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:34.147 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.144+0000 7f1771a18700 1 --2- 172.21.15.67:0/2878405403 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f176c15dbc0 0x7f176c147410 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f176c0b7980 tx=0x7f1764004ad0 comp rx=0 tx=0).stop 2023-10-14T10:36:34.147 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.144+0000 7f1771a18700 1 -- 172.21.15.67:0/2878405403 shutdown_connections 2023-10-14T10:36:34.147 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.144+0000 7f1771a18700 1 --2- 172.21.15.67:0/2878405403 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f175806af00 0x7f175806d3c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:34.147 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.144+0000 7f1771a18700 1 --2- 172.21.15.67:0/2878405403 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f176c15dbc0 0x7f176c147410 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:34.147 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.144+0000 7f1771a18700 1 --2- 172.21.15.67:0/2878405403 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f176c0b7050 0x7f176c0c94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:34.148 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.144+0000 7f1771a18700 1 --2- 172.21.15.67:0/2878405403 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f176c0d3fc0 0x7f176c158f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:34.148 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.144+0000 7f1771a18700 1 -- 172.21.15.67:0/2878405403 >> 172.21.15.67:0/2878405403 conn(0x7f176c09cb40 msgr2=0x7f176c0cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:34.148 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.144+0000 7f1771a18700 1 -- 172.21.15.67:0/2878405403 shutdown_connections 2023-10-14T10:36:34.148 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.144+0000 7f1771a18700 1 -- 172.21.15.67:0/2878405403 wait complete. 2023-10-14T10:36:34.158 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-14T10:36:34.287 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.284+0000 7f7f136c1700 1 Processor -- start 2023-10-14T10:36:34.287 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.285+0000 7f7f136c1700 1 -- start start 2023-10-14T10:36:34.289 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.285+0000 7f7f136c1700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f7f0c0cd950 0x7f7f0c15e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:34.290 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.285+0000 7f7f136c1700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f7f0c06a840 0x7f7f0c06ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:34.290 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.285+0000 7f7f136c1700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f0c0cf290 con 0x7f7f0c0cede0 2023-10-14T10:36:34.290 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.285+0000 7f7f136c1700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f0c1619a0 con 0x7f7f0c0cd950 2023-10-14T10:36:34.290 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.285+0000 7f7f136c1700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f0c06b1b0 con 0x7f7f0c06a840 2023-10-14T10:36:34.291 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.286+0000 7f7f1145d700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f7f0c0cd950 0x7f7f0c15e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:34.291 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.286+0000 7f7f1145d700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f7f0c0cd950 0x7f7f0c15e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44902/0 (socket says 172.21.15.67:44902) 2023-10-14T10:36:34.291 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.286+0000 7f7f1145d700 1 -- 172.21.15.67:0/1855153087 learned_addr learned my addr 172.21.15.67:0/1855153087 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:34.291 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.286+0000 7f7f11c5e700 1 --2- 172.21.15.67:0/1855153087 >> v2:172.21.15.72:3300/0 conn(0x7f7f0c06a840 0x7f7f0c06ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:34.292 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.286+0000 7f7f1145d700 1 -- 172.21.15.67:0/1855153087 >> v2:172.21.15.72:3300/0 conn(0x7f7f0c06a840 msgr2=0x7f7f0c06ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:34.292 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.286+0000 7f7f1145d700 1 --2- 172.21.15.67:0/1855153087 >> v2:172.21.15.72:3300/0 conn(0x7f7f0c06a840 0x7f7f0c06ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:34.292 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.286+0000 7f7f1145d700 1 -- 172.21.15.67:0/1855153087 >> v1:172.21.15.72:6789/0 conn(0x7f7f0c0cede0 legacy=0x7f7f0c15f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:34.293 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.286+0000 7f7f1145d700 1 -- 172.21.15.67:0/1855153087 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f0c0b2d60 con 0x7f7f0c0cd950 2023-10-14T10:36:34.293 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.286+0000 7f7f1145d700 1 --2- 172.21.15.67:0/1855153087 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7f0c0cd950 0x7f7f0c15e2f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c15e830 tx=0x7f7f04009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e92c19e6f828184 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:34.294 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.286+0000 7f7f0bfff700 1 -- 172.21.15.67:0/1855153087 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f04011830 con 0x7f7f0c0cd950 2023-10-14T10:36:34.294 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.286+0000 7f7f0bfff700 1 -- 172.21.15.67:0/1855153087 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f04011990 con 0x7f7f0c0cd950 2023-10-14T10:36:34.294 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.286+0000 7f7f0bfff700 1 -- 172.21.15.67:0/1855153087 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f04011d20 con 0x7f7f0c0cd950 2023-10-14T10:36:34.294 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.287+0000 7f7f136c1700 1 -- 172.21.15.67:0/1855153087 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7f0c0cd950 msgr2=0x7f7f0c15e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:34.294 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.287+0000 7f7f136c1700 1 --2- 172.21.15.67:0/1855153087 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7f0c0cd950 0x7f7f0c15e2f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c15e830 tx=0x7f7f04009bd0 comp rx=0 tx=0).stop 2023-10-14T10:36:34.295 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.287+0000 7f7f136c1700 1 -- 172.21.15.67:0/1855153087 shutdown_connections 2023-10-14T10:36:34.295 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.287+0000 7f7f136c1700 1 --2- 172.21.15.67:0/1855153087 >> v2:172.21.15.72:3300/0 conn(0x7f7f0c06a840 0x7f7f0c06ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:34.295 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.287+0000 7f7f136c1700 1 --2- 172.21.15.67:0/1855153087 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7f0c0cd950 0x7f7f0c15e2f0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:34.295 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.287+0000 7f7f136c1700 1 -- 172.21.15.67:0/1855153087 >> 172.21.15.67:0/1855153087 conn(0x7f7f0c06dae0 msgr2=0x7f7f0c06def0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:34.295 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.287+0000 7f7f136c1700 1 -- 172.21.15.67:0/1855153087 shutdown_connections 2023-10-14T10:36:34.296 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.287+0000 7f7f136c1700 1 -- 172.21.15.67:0/1855153087 wait complete. 2023-10-14T10:36:34.299 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.289+0000 7f7f136c1700 1 Processor -- start 2023-10-14T10:36:34.299 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.289+0000 7f7f136c1700 1 -- start start 2023-10-14T10:36:34.299 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.290+0000 7f7f136c1700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f7f0c06a840 0x7f7f0c15aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:34.299 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.290+0000 7f7f136c1700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7f0c0cd950 0x7f7f0c1f6c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:34.300 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.290+0000 7f7f136c1700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7f0c0cede0 0x7f7f0c0c6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:34.300 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.290+0000 7f7f136c1700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f0c1f3640 con 0x7f7f0c0cd950 2023-10-14T10:36:34.300 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.290+0000 7f7f136c1700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f0c0c8940 con 0x7f7f0c0cede0 2023-10-14T10:36:34.300 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.290+0000 7f7f136c1700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f0c0c8c20 con 0x7f7f0c06a840 2023-10-14T10:36:34.300 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.290+0000 7f7f10c5c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7f0c0cd950 0x7f7f0c1f6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:34.301 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.290+0000 7f7f10c5c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7f0c0cd950 0x7f7f0c1f6c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44910/0 (socket says 172.21.15.67:44910) 2023-10-14T10:36:34.301 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.290+0000 7f7f10c5c700 1 -- 172.21.15.67:0/956212198 learned_addr learned my addr 172.21.15.67:0/956212198 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:34.301 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.290+0000 7f7f11c5e700 1 --2- 172.21.15.67:0/956212198 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7f0c0cede0 0x7f7f0c0c6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:34.301 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.290+0000 7f7f1145d700 1 --2- 172.21.15.67:0/956212198 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f7f0c06a840 0x7f7f0c15aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:34.301 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.290+0000 7f7f10c5c700 1 -- 172.21.15.67:0/956212198 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f7f0c06a840 msgr2=0x7f7f0c15aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:34.302 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.290+0000 7f7f10c5c700 1 --2- 172.21.15.67:0/956212198 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f7f0c06a840 0x7f7f0c15aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:34.302 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.290+0000 7f7f10c5c700 1 -- 172.21.15.67:0/956212198 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7f0c0cede0 msgr2=0x7f7f0c0c6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:34.302 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.290+0000 7f7f10c5c700 1 --2- 172.21.15.67:0/956212198 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7f0c0cede0 0x7f7f0c0c6150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:34.302 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.290+0000 7f7f10c5c700 1 -- 172.21.15.67:0/956212198 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f04009550 con 0x7f7f0c0cd950 2023-10-14T10:36:34.302 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.290+0000 7f7f10c5c700 1 --2- 172.21.15.67:0/956212198 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7f0c0cd950 0x7f7f0c1f6c40 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7f0000c430 tx=0x7f7f0000c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:34.303 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.290+0000 7f7f09ffb700 1 -- 172.21.15.67:0/956212198 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f000052d0 con 0x7f7f0c0cd950 2023-10-14T10:36:34.303 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.290+0000 7f7f09ffb700 1 -- 172.21.15.67:0/956212198 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f00005450 con 0x7f7f0c0cd950 2023-10-14T10:36:34.303 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.290+0000 7f7f136c1700 1 -- 172.21.15.67:0/956212198 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f0c0c8f00 con 0x7f7f0c0cd950 2023-10-14T10:36:34.303 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.290+0000 7f7f09ffb700 1 -- 172.21.15.67:0/956212198 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f00006af0 con 0x7f7f0c0cd950 2023-10-14T10:36:34.303 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.291+0000 7f7f136c1700 1 -- 172.21.15.67:0/956212198 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f0c15afc0 con 0x7f7f0c0cd950 2023-10-14T10:36:34.304 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.291+0000 7f7f136c1700 1 -- 172.21.15.67:0/956212198 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f0c067d20 con 0x7f7f0c0cd950 2023-10-14T10:36:34.304 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.292+0000 7f7f09ffb700 1 -- 172.21.15.67:0/956212198 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f7f000036d0 con 0x7f7f0c0cd950 2023-10-14T10:36:34.304 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.292+0000 7f7f09ffb700 1 --2- 172.21.15.67:0/956212198 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f7ef806af00 0x7f7ef806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:34.304 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.292+0000 7f7f09ffb700 1 -- 172.21.15.67:0/956212198 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7f000886e0 con 0x7f7f0c0cd950 2023-10-14T10:36:34.304 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.293+0000 7f7f1145d700 1 --2- 172.21.15.67:0/956212198 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f7ef806af00 0x7f7ef806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:34.305 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.293+0000 7f7f1145d700 1 --2- 172.21.15.67:0/956212198 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f7ef806af00 0x7f7ef806d3c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7f04009c10 tx=0x7f7f04008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:34.305 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.297+0000 7f7f09ffb700 1 -- 172.21.15.67:0/956212198 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7f00053790 con 0x7f7f0c0cd950 2023-10-14T10:36:34.499 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.496+0000 7f7f136c1700 1 -- 172.21.15.67:0/956212198 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7f0c0b2630 con 0x7f7f0c0cd950 2023-10-14T10:36:34.499 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.497+0000 7f7f09ffb700 1 -- 172.21.15.67:0/956212198 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7f00056db0 con 0x7f7f0c0cd950 2023-10-14T10:36:34.499 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2023-10-14T10:36:34.501 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.499+0000 7f7f136c1700 1 -- 172.21.15.67:0/956212198 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f7ef806af00 msgr2=0x7f7ef806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:34.502 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.499+0000 7f7f136c1700 1 --2- 172.21.15.67:0/956212198 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f7ef806af00 0x7f7ef806d3c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7f04009c10 tx=0x7f7f04008030 comp rx=0 tx=0).stop 2023-10-14T10:36:34.502 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.499+0000 7f7f136c1700 1 -- 172.21.15.67:0/956212198 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7f0c0cd950 msgr2=0x7f7f0c1f6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:34.502 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.499+0000 7f7f136c1700 1 --2- 172.21.15.67:0/956212198 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7f0c0cd950 0x7f7f0c1f6c40 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7f0000c430 tx=0x7f7f0000c740 comp rx=0 tx=0).stop 2023-10-14T10:36:34.502 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.499+0000 7f7f136c1700 1 -- 172.21.15.67:0/956212198 shutdown_connections 2023-10-14T10:36:34.503 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.499+0000 7f7f136c1700 1 --2- 172.21.15.67:0/956212198 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f7ef806af00 0x7f7ef806d3c0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:34.503 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.499+0000 7f7f136c1700 1 --2- 172.21.15.67:0/956212198 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7f0c0cede0 0x7f7f0c0c6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:34.503 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.499+0000 7f7f136c1700 1 --2- 172.21.15.67:0/956212198 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f7f0c06a840 0x7f7f0c15aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:34.503 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.499+0000 7f7f136c1700 1 --2- 172.21.15.67:0/956212198 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7f0c0cd950 0x7f7f0c1f6c40 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:34.503 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.499+0000 7f7f136c1700 1 -- 172.21.15.67:0/956212198 >> 172.21.15.67:0/956212198 conn(0x7f7f0c06dae0 msgr2=0x7f7f0c0b0500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:34.503 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.499+0000 7f7f136c1700 1 -- 172.21.15.67:0/956212198 shutdown_connections 2023-10-14T10:36:34.504 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.499+0000 7f7f136c1700 1 -- 172.21.15.67:0/956212198 wait complete. 2023-10-14T10:36:34.513 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-14T10:36:34.641 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.639+0000 7ff630672700 1 Processor -- start 2023-10-14T10:36:34.641 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.639+0000 7ff630672700 1 -- start start 2023-10-14T10:36:34.644 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.640+0000 7ff630672700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7ff62815f1f0 0x7ff6280cf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:34.644 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.640+0000 7ff630672700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7ff6280cfc60 0x7ff6280cbf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:34.644 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.640+0000 7ff630672700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff62815e7a0 con 0x7ff62815ab10 2023-10-14T10:36:34.644 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.640+0000 7ff630672700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6280d00d0 con 0x7ff62815f1f0 2023-10-14T10:36:34.645 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.640+0000 7ff630672700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6280b29a0 con 0x7ff6280cfc60 2023-10-14T10:36:34.645 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.640+0000 7ff62dc0d700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7ff62815f1f0 0x7ff6280cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:34.645 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.640+0000 7ff62ec0f700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7ff6280cfc60 0x7ff6280cbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:34.645 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.640+0000 7ff62dc0d700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7ff62815f1f0 0x7ff6280cf720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.67:39706/0 (socket says 172.21.15.67:39706) 2023-10-14T10:36:34.645 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.640+0000 7ff62dc0d700 1 -- 172.21.15.67:0/3563506068 learned_addr learned my addr 172.21.15.67:0/3563506068 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:34.646 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.640+0000 7ff62e40e700 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7ff62815ab10 0x7ff6280b1f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.67:60924/0 (socket says 172.21.15.67:60924) 2023-10-14T10:36:34.646 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.640+0000 7ff62dc0d700 1 -- 172.21.15.67:0/3563506068 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7ff6280cfc60 msgr2=0x7ff6280cbf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:34.646 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.640+0000 7ff62dc0d700 1 --2- 172.21.15.67:0/3563506068 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7ff6280cfc60 0x7ff6280cbf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:34.646 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.640+0000 7ff62dc0d700 1 -- 172.21.15.67:0/3563506068 >> v1:172.21.15.72:6789/0 conn(0x7ff62815ab10 legacy=0x7ff6280b1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:34.646 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.640+0000 7ff62dc0d700 1 -- 172.21.15.67:0/3563506068 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6280b2c80 con 0x7ff62815f1f0 2023-10-14T10:36:34.647 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.641+0000 7ff62dc0d700 1 --2- 172.21.15.67:0/3563506068 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff62815f1f0 0x7ff6280cf720 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff618006fd0 tx=0x7ff61800be70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d42bb2e1b144d34 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:34.647 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.641+0000 7ff62d40c700 1 -- 172.21.15.67:0/3563506068 <== mon.1 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff618005e30 con 0x7ff62815f1f0 2023-10-14T10:36:34.647 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.641+0000 7ff62d40c700 1 -- 172.21.15.67:0/3563506068 <== mon.1 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff618004030 con 0x7ff62815f1f0 2023-10-14T10:36:34.647 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.641+0000 7ff62d40c700 1 -- 172.21.15.67:0/3563506068 <== mon.1 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6180043e0 con 0x7ff62815f1f0 2023-10-14T10:36:34.647 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.641+0000 7ff630672700 1 -- 172.21.15.67:0/3563506068 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff62815f1f0 msgr2=0x7ff6280cf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:34.648 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.641+0000 7ff630672700 1 --2- 172.21.15.67:0/3563506068 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff62815f1f0 0x7ff6280cf720 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff618006fd0 tx=0x7ff61800be70 comp rx=0 tx=0).stop 2023-10-14T10:36:34.648 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.641+0000 7ff630672700 1 -- 172.21.15.67:0/3563506068 shutdown_connections 2023-10-14T10:36:34.648 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.641+0000 7ff630672700 1 --2- 172.21.15.67:0/3563506068 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7ff6280cfc60 0x7ff6280cbf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:34.648 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.641+0000 7ff630672700 1 --2- 172.21.15.67:0/3563506068 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff62815f1f0 0x7ff6280cf720 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:34.649 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.641+0000 7ff630672700 1 -- 172.21.15.67:0/3563506068 >> 172.21.15.67:0/3563506068 conn(0x7ff62806dae0 msgr2=0x7ff62806def0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:34.649 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.641+0000 7ff630672700 1 -- 172.21.15.67:0/3563506068 shutdown_connections 2023-10-14T10:36:34.649 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.641+0000 7ff630672700 1 -- 172.21.15.67:0/3563506068 wait complete. 2023-10-14T10:36:34.652 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.643+0000 7ff630672700 1 Processor -- start 2023-10-14T10:36:34.652 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.643+0000 7ff630672700 1 -- start start 2023-10-14T10:36:34.652 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.643+0000 7ff630672700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff6280cfc60 0x7ff6280ca010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:34.652 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.643+0000 7ff630672700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7ff62815ab10 0x7ff6280acfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:34.652 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.643+0000 7ff630672700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff62815f1f0 0x7ff628158980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:34.653 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.643+0000 7ff630672700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6280ada80 con 0x7ff6280cfc60 2023-10-14T10:36:34.653 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.643+0000 7ff630672700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6280add60 con 0x7ff62815f1f0 2023-10-14T10:36:34.653 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.643+0000 7ff630672700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6280ae040 con 0x7ff62815ab10 2023-10-14T10:36:34.653 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.644+0000 7ff62e40e700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff6280cfc60 0x7ff6280ca010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:34.653 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.644+0000 7ff62e40e700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff6280cfc60 0x7ff6280ca010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44922/0 (socket says 172.21.15.67:44922) 2023-10-14T10:36:34.654 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.644+0000 7ff62e40e700 1 -- 172.21.15.67:0/737526300 learned_addr learned my addr 172.21.15.67:0/737526300 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:34.654 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.644+0000 7ff62ec0f700 1 --2- 172.21.15.67:0/737526300 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff62815f1f0 0x7ff628158980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:34.654 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.644+0000 7ff62dc0d700 1 --2- 172.21.15.67:0/737526300 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7ff62815ab10 0x7ff6280acfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:34.654 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.644+0000 7ff62e40e700 1 -- 172.21.15.67:0/737526300 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7ff62815ab10 msgr2=0x7ff6280acfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:34.654 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.644+0000 7ff62e40e700 1 --2- 172.21.15.67:0/737526300 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7ff62815ab10 0x7ff6280acfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:34.655 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.644+0000 7ff62e40e700 1 -- 172.21.15.67:0/737526300 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff62815f1f0 msgr2=0x7ff628158980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:34.655 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.644+0000 7ff62e40e700 1 --2- 172.21.15.67:0/737526300 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff62815f1f0 0x7ff628158980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:34.655 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.644+0000 7ff62e40e700 1 -- 172.21.15.67:0/737526300 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff61800b7a0 con 0x7ff6280cfc60 2023-10-14T10:36:34.655 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.644+0000 7ff62dc0d700 1 --2- 172.21.15.67:0/737526300 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7ff62815ab10 0x7ff6280acfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-14T10:36:34.655 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.644+0000 7ff62e40e700 1 --2- 172.21.15.67:0/737526300 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff6280cfc60 0x7ff6280ca010 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff62400c500 tx=0x7ff62400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:34.656 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.644+0000 7ff61effd700 1 -- 172.21.15.67:0/737526300 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff624005cb0 con 0x7ff6280cfc60 2023-10-14T10:36:34.656 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.644+0000 7ff61effd700 1 -- 172.21.15.67:0/737526300 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff624005e10 con 0x7ff6280cfc60 2023-10-14T10:36:34.656 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.644+0000 7ff630672700 1 -- 172.21.15.67:0/737526300 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6280ae320 con 0x7ff6280cfc60 2023-10-14T10:36:34.656 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.644+0000 7ff61effd700 1 -- 172.21.15.67:0/737526300 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6240072d0 con 0x7ff6280cfc60 2023-10-14T10:36:34.656 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.644+0000 7ff630672700 1 -- 172.21.15.67:0/737526300 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6280af1f0 con 0x7ff6280cfc60 2023-10-14T10:36:34.657 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.645+0000 7ff630672700 1 -- 172.21.15.67:0/737526300 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff628067d20 con 0x7ff6280cfc60 2023-10-14T10:36:34.657 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.646+0000 7ff61effd700 1 -- 172.21.15.67:0/737526300 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff624007430 con 0x7ff6280cfc60 2023-10-14T10:36:34.657 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.646+0000 7ff61effd700 1 --2- 172.21.15.67:0/737526300 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7ff60c06af00 0x7ff60c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:34.657 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.646+0000 7ff61effd700 1 -- 172.21.15.67:0/737526300 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff624088870 con 0x7ff6280cfc60 2023-10-14T10:36:34.657 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.646+0000 7ff62dc0d700 1 --2- 172.21.15.67:0/737526300 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7ff60c06af00 0x7ff60c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:34.657 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.647+0000 7ff62dc0d700 1 --2- 172.21.15.67:0/737526300 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7ff60c06af00 0x7ff60c06d3c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff61800b340 tx=0x7ff618008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:34.658 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.651+0000 7ff61effd700 1 -- 172.21.15.67:0/737526300 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff6240539d0 con 0x7ff6280cfc60 2023-10-14T10:36:34.850 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.848+0000 7ff630672700 1 -- 172.21.15.67:0/737526300 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff6280b50b0 con 0x7ff6280cfc60 2023-10-14T10:36:34.850 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.848+0000 7ff61effd700 1 -- 172.21.15.67:0/737526300 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff624017020 con 0x7ff6280cfc60 2023-10-14T10:36:34.851 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2023-10-14T10:36:34.853 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.850+0000 7ff630672700 1 -- 172.21.15.67:0/737526300 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7ff60c06af00 msgr2=0x7ff60c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:34.853 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.850+0000 7ff630672700 1 --2- 172.21.15.67:0/737526300 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7ff60c06af00 0x7ff60c06d3c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff61800b340 tx=0x7ff618008040 comp rx=0 tx=0).stop 2023-10-14T10:36:34.853 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.850+0000 7ff630672700 1 -- 172.21.15.67:0/737526300 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff6280cfc60 msgr2=0x7ff6280ca010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:34.853 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.850+0000 7ff630672700 1 --2- 172.21.15.67:0/737526300 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff6280cfc60 0x7ff6280ca010 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff62400c500 tx=0x7ff62400c810 comp rx=0 tx=0).stop 2023-10-14T10:36:34.853 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.850+0000 7ff630672700 1 -- 172.21.15.67:0/737526300 shutdown_connections 2023-10-14T10:36:34.854 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.850+0000 7ff630672700 1 --2- 172.21.15.67:0/737526300 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7ff60c06af00 0x7ff60c06d3c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:34.854 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.850+0000 7ff630672700 1 --2- 172.21.15.67:0/737526300 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff62815f1f0 0x7ff628158980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:34.854 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.850+0000 7ff630672700 1 --2- 172.21.15.67:0/737526300 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff6280cfc60 0x7ff6280ca010 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:34.854 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.850+0000 7ff630672700 1 --2- 172.21.15.67:0/737526300 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7ff62815ab10 0x7ff6280acfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:34.855 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.850+0000 7ff630672700 1 -- 172.21.15.67:0/737526300 >> 172.21.15.67:0/737526300 conn(0x7ff62806dae0 msgr2=0x7ff6280b3030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:34.855 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.850+0000 7ff630672700 1 -- 172.21.15.67:0/737526300 shutdown_connections 2023-10-14T10:36:34.855 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.850+0000 7ff630672700 1 -- 172.21.15.67:0/737526300 wait complete. 2023-10-14T10:36:34.865 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_create' 2023-10-14T10:36:34.991 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.989+0000 7ff8390f7700 1 Processor -- start 2023-10-14T10:36:34.991 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.989+0000 7ff8390f7700 1 -- start start 2023-10-14T10:36:34.993 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.989+0000 7ff8390f7700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7ff834157d40 0x7ff8340cee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:34.994 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.989+0000 7ff8390f7700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7ff83406e2b0 0x7ff8340cb420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:34.994 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.989+0000 7ff8390f7700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff834158120 con 0x7ff83406de50 2023-10-14T10:36:34.994 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.989+0000 7ff8390f7700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff83406a840 con 0x7ff83406e2b0 2023-10-14T10:36:34.994 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.989+0000 7ff8390f7700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8340caef0 con 0x7ff834157d40 2023-10-14T10:36:34.995 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.990+0000 7ff83259c700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7ff83406de50 0x7ff83415dd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:52700/0 (socket says 172.21.15.67:52700) 2023-10-14T10:36:34.995 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.990+0000 7ff83259c700 1 -- 172.21.15.67:0/486550101 learned_addr learned my addr 172.21.15.67:0/486550101 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:36:34.995 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.990+0000 7ff83359e700 1 --2- 172.21.15.67:0/486550101 >> v2:172.21.15.72:3300/0 conn(0x7ff83406e2b0 0x7ff8340cb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:34.995 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.990+0000 7ff832d9d700 1 --2- 172.21.15.67:0/486550101 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7ff834157d40 0x7ff8340cee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:34.995 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.990+0000 7ff831d9b700 1 -- 172.21.15.67:0/486550101 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3508247411 0 0) 0x7ff834158120 con 0x7ff83406de50 2023-10-14T10:36:34.996 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.990+0000 7ff831d9b700 1 -- 172.21.15.67:0/486550101 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8200035b0 con 0x7ff83406de50 2023-10-14T10:36:34.996 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.990+0000 7ff83359e700 1 -- 172.21.15.67:0/486550101 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7ff834157d40 msgr2=0x7ff8340cee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:34.996 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.990+0000 7ff83359e700 1 --2- 172.21.15.67:0/486550101 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7ff834157d40 0x7ff8340cee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:34.996 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.990+0000 7ff83359e700 1 -- 172.21.15.67:0/486550101 >> v1:172.21.15.67:6789/0 conn(0x7ff83406de50 legacy=0x7ff83415dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:34.996 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.990+0000 7ff83359e700 1 -- 172.21.15.67:0/486550101 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff83406e750 con 0x7ff83406e2b0 2023-10-14T10:36:34.997 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.990+0000 7ff83359e700 1 --2- 172.21.15.67:0/486550101 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff83406e2b0 0x7ff8340cb420 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff82c009d90 tx=0x7ff82c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=625180ac2069a7d server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:34.997 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.990+0000 7ff831d9b700 1 -- 172.21.15.67:0/486550101 <== mon.1 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff82c00e070 con 0x7ff83406e2b0 2023-10-14T10:36:34.997 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.990+0000 7ff831d9b700 1 -- 172.21.15.67:0/486550101 <== mon.1 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff82c00bb20 con 0x7ff83406e2b0 2023-10-14T10:36:34.997 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.990+0000 7ff831d9b700 1 -- 172.21.15.67:0/486550101 <== mon.1 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff82c012430 con 0x7ff83406e2b0 2023-10-14T10:36:34.997 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.991+0000 7ff8390f7700 1 -- 172.21.15.67:0/486550101 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff83406e2b0 msgr2=0x7ff8340cb420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:34.998 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.991+0000 7ff8390f7700 1 --2- 172.21.15.67:0/486550101 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff83406e2b0 0x7ff8340cb420 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff82c009d90 tx=0x7ff82c00b330 comp rx=0 tx=0).stop 2023-10-14T10:36:34.998 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.991+0000 7ff8390f7700 1 -- 172.21.15.67:0/486550101 shutdown_connections 2023-10-14T10:36:34.998 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.991+0000 7ff8390f7700 1 --2- 172.21.15.67:0/486550101 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff83406e2b0 0x7ff8340cb420 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:34.998 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.991+0000 7ff8390f7700 1 --2- 172.21.15.67:0/486550101 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7ff834157d40 0x7ff8340cee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:34.998 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.991+0000 7ff8390f7700 1 -- 172.21.15.67:0/486550101 >> 172.21.15.67:0/486550101 conn(0x7ff8340cc6f0 msgr2=0x7ff8340ccb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:34.999 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.991+0000 7ff8390f7700 1 -- 172.21.15.67:0/486550101 shutdown_connections 2023-10-14T10:36:34.999 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.991+0000 7ff8390f7700 1 -- 172.21.15.67:0/486550101 wait complete. 2023-10-14T10:36:35.002 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.992+0000 7ff8390f7700 1 Processor -- start 2023-10-14T10:36:35.002 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.993+0000 7ff8390f7700 1 -- start start 2023-10-14T10:36:35.002 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.993+0000 7ff8390f7700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff83406de50 0x7ff83415b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:35.002 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.993+0000 7ff8390f7700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7ff83406e2b0 0x7ff83414ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:35.003 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.993+0000 7ff8390f7700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff834157d40 0x7ff83414b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:35.003 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.993+0000 7ff8390f7700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff834158e90 con 0x7ff834157d40 2023-10-14T10:36:35.003 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.993+0000 7ff8390f7700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8340c3e70 con 0x7ff83406de50 2023-10-14T10:36:35.003 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.993+0000 7ff8390f7700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8340c3260 con 0x7ff83406e2b0 2023-10-14T10:36:35.004 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.993+0000 7ff83359e700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff834157d40 0x7ff83414b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:35.004 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.993+0000 7ff83359e700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff834157d40 0x7ff83414b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44940/0 (socket says 172.21.15.67:44940) 2023-10-14T10:36:35.004 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.993+0000 7ff83359e700 1 -- 172.21.15.67:0/2211227537 learned_addr learned my addr 172.21.15.67:0/2211227537 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:35.004 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.994+0000 7ff832d9d700 1 --2- 172.21.15.67:0/2211227537 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff83406de50 0x7ff83415b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:35.005 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.994+0000 7ff83259c700 1 --2- 172.21.15.67:0/2211227537 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7ff83406e2b0 0x7ff83414ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:35.005 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.994+0000 7ff83359e700 1 -- 172.21.15.67:0/2211227537 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7ff83406e2b0 msgr2=0x7ff83414ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:35.005 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.994+0000 7ff83359e700 1 --2- 172.21.15.67:0/2211227537 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7ff83406e2b0 0x7ff83414ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:35.005 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.994+0000 7ff83359e700 1 -- 172.21.15.67:0/2211227537 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff83406de50 msgr2=0x7ff83415b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:35.006 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.994+0000 7ff83359e700 1 --2- 172.21.15.67:0/2211227537 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff83406de50 0x7ff83415b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:35.006 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.994+0000 7ff83359e700 1 -- 172.21.15.67:0/2211227537 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff82c009a70 con 0x7ff834157d40 2023-10-14T10:36:35.006 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.994+0000 7ff83259c700 1 --2- 172.21.15.67:0/2211227537 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7ff83406e2b0 0x7ff83414ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-14T10:36:35.006 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.994+0000 7ff832d9d700 1 --2- 172.21.15.67:0/2211227537 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff83406de50 0x7ff83415b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-14T10:36:35.006 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.994+0000 7ff83359e700 1 --2- 172.21.15.67:0/2211227537 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff834157d40 0x7ff83414b0c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff82c014040 tx=0x7ff82c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:35.007 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.994+0000 7ff82b7fe700 1 -- 172.21.15.67:0/2211227537 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff82c0129f0 con 0x7ff834157d40 2023-10-14T10:36:35.007 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.994+0000 7ff82b7fe700 1 -- 172.21.15.67:0/2211227537 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff82c020070 con 0x7ff834157d40 2023-10-14T10:36:35.007 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.994+0000 7ff8390f7700 1 -- 172.21.15.67:0/2211227537 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8340c3540 con 0x7ff834157d40 2023-10-14T10:36:35.007 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.994+0000 7ff82b7fe700 1 -- 172.21.15.67:0/2211227537 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff82c012db0 con 0x7ff834157d40 2023-10-14T10:36:35.007 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.994+0000 7ff8390f7700 1 -- 172.21.15.67:0/2211227537 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8340c3b00 con 0x7ff834157d40 2023-10-14T10:36:35.008 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.995+0000 7ff8390f7700 1 -- 172.21.15.67:0/2211227537 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff834067d20 con 0x7ff834157d40 2023-10-14T10:36:35.008 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.996+0000 7ff82b7fe700 1 -- 172.21.15.67:0/2211227537 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff82c004040 con 0x7ff834157d40 2023-10-14T10:36:35.008 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.996+0000 7ff82b7fe700 1 --2- 172.21.15.67:0/2211227537 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7ff81406af00 0x7ff81406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:35.008 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.996+0000 7ff82b7fe700 1 -- 172.21.15.67:0/2211227537 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff82c022070 con 0x7ff834157d40 2023-10-14T10:36:35.008 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.996+0000 7ff832d9d700 1 --2- 172.21.15.67:0/2211227537 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7ff81406af00 0x7ff81406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:35.009 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:34.997+0000 7ff832d9d700 1 --2- 172.21.15.67:0/2211227537 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7ff81406af00 0x7ff81406d3c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff81c009b90 tx=0x7ff81c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:35.009 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:35.000+0000 7ff82b7fe700 1 -- 172.21.15.67:0/2211227537 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff82c05a020 con 0x7ff834157d40 2023-10-14T10:36:35.291 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:35.289+0000 7ff8390f7700 1 -- 172.21.15.67:0/2211227537 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_create"]} v 0) v1 -- 0x7ff8340c3fb0 con 0x7ff834157d40 2023-10-14T10:36:35.962 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:35.960+0000 7ff82b7fe700 1 -- 172.21.15.67:0/2211227537 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_create"]}]=0 v47) v1 ==== 138+0+0 (secure 0 0 0) 0x7ff82c059bb0 con 0x7ff834157d40 2023-10-14T10:36:35.965 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:35.962+0000 7ff8390f7700 1 -- 172.21.15.67:0/2211227537 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7ff81406af00 msgr2=0x7ff81406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:35.965 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:35.962+0000 7ff8390f7700 1 --2- 172.21.15.67:0/2211227537 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7ff81406af00 0x7ff81406d3c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff81c009b90 tx=0x7ff81c008040 comp rx=0 tx=0).stop 2023-10-14T10:36:35.965 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:35.962+0000 7ff8390f7700 1 -- 172.21.15.67:0/2211227537 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff834157d40 msgr2=0x7ff83414b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:35.965 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:35.962+0000 7ff8390f7700 1 --2- 172.21.15.67:0/2211227537 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff834157d40 0x7ff83414b0c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff82c014040 tx=0x7ff82c004ad0 comp rx=0 tx=0).stop 2023-10-14T10:36:35.965 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:35.962+0000 7ff8390f7700 1 -- 172.21.15.67:0/2211227537 shutdown_connections 2023-10-14T10:36:35.966 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:35.962+0000 7ff8390f7700 1 --2- 172.21.15.67:0/2211227537 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7ff81406af00 0x7ff81406d3c0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:35.966 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:35.962+0000 7ff8390f7700 1 --2- 172.21.15.67:0/2211227537 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff834157d40 0x7ff83414b0c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:35.966 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:35.962+0000 7ff8390f7700 1 --2- 172.21.15.67:0/2211227537 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff83406de50 0x7ff83415b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:35.966 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:35.962+0000 7ff8390f7700 1 --2- 172.21.15.67:0/2211227537 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7ff83406e2b0 0x7ff83414ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:35.967 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:35.962+0000 7ff8390f7700 1 -- 172.21.15.67:0/2211227537 >> 172.21.15.67:0/2211227537 conn(0x7ff8340cc6f0 msgr2=0x7ff83406b2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:35.967 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:35.962+0000 7ff8390f7700 1 -- 172.21.15.67:0/2211227537 shutdown_connections 2023-10-14T10:36:35.967 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:35.963+0000 7ff8390f7700 1 -- 172.21.15.67:0/2211227537 wait complete. 2023-10-14T10:36:35.979 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-14T10:36:36.113 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.111+0000 7f3b1687d700 1 Processor -- start 2023-10-14T10:36:36.113 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.111+0000 7f3b1687d700 1 -- start start 2023-10-14T10:36:36.117 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.112+0000 7f3b1687d700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f3b1015bb50 0x7f3b1006e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:36.117 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.112+0000 7f3b1687d700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f3b100ce280 0x7f3b1006a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:36.118 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.112+0000 7f3b1687d700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b1015bf30 con 0x7f3b1006ed50 2023-10-14T10:36:36.118 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.112+0000 7f3b1687d700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b1006f1d0 con 0x7f3b1015bb50 2023-10-14T10:36:36.118 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.112+0000 7f3b1687d700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b100b2d70 con 0x7f3b100ce280 2023-10-14T10:36:36.118 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.112+0000 7f3b0f7fe700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f3b1006ed50 0x7f3b100cdb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:52714/0 (socket says 172.21.15.67:52714) 2023-10-14T10:36:36.118 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.112+0000 7f3b0f7fe700 1 -- 172.21.15.67:0/1736159240 learned_addr learned my addr 172.21.15.67:0/1736159240 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:36:36.119 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.112+0000 7f3b0ffff700 1 --2- 172.21.15.67:0/1736159240 >> v2:172.21.15.72:3300/0 conn(0x7f3b1015bb50 0x7f3b1006e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:36.119 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.112+0000 7f3b14e1a700 1 --2- 172.21.15.67:0/1736159240 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f3b100ce280 0x7f3b1006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:36.119 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.112+0000 7f3b0effd700 1 -- 172.21.15.67:0/1736159240 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 227938729 0 0) 0x7f3b1015bf30 con 0x7f3b1006ed50 2023-10-14T10:36:36.119 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.112+0000 7f3b0effd700 1 -- 172.21.15.67:0/1736159240 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3af80035b0 con 0x7f3b1006ed50 2023-10-14T10:36:36.120 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.112+0000 7f3b0effd700 1 -- 172.21.15.67:0/1736159240 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1235137793 0 0) 0x7f3af80035b0 con 0x7f3b1006ed50 2023-10-14T10:36:36.120 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.112+0000 7f3b0effd700 1 -- 172.21.15.67:0/1736159240 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f3b100ce280 msgr2=0x7f3b1006a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:36.120 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.112+0000 7f3b0effd700 1 --2- 172.21.15.67:0/1736159240 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f3b100ce280 0x7f3b1006a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:36.120 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.112+0000 7f3b0effd700 1 -- 172.21.15.67:0/1736159240 >> v2:172.21.15.72:3300/0 conn(0x7f3b1015bb50 msgr2=0x7f3b1006e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:36.120 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.112+0000 7f3b0effd700 1 --2- 172.21.15.67:0/1736159240 >> v2:172.21.15.72:3300/0 conn(0x7f3b1015bb50 0x7f3b1006e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:36.120 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.112+0000 7f3b0effd700 1 -- 172.21.15.67:0/1736159240 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b1006d500 con 0x7f3b1006ed50 2023-10-14T10:36:36.121 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.112+0000 7f3b0effd700 1 -- 172.21.15.67:0/1736159240 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3180043755 0 0) 0x7f3b04001670 con 0x7f3b1006ed50 2023-10-14T10:36:36.121 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.112+0000 7f3b0effd700 1 -- 172.21.15.67:0/1736159240 >> v1:172.21.15.67:6789/0 conn(0x7f3b1006ed50 legacy=0x7f3b100cdb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:36.121 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.112+0000 7f3b0effd700 1 --2- 172.21.15.67:0/1736159240 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f3b100ce280 0x7f3af8003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:36.121 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.113+0000 7f3b0effd700 1 --2- 172.21.15.67:0/1736159240 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3af8003f80 0x7f3af8004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:36.121 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.113+0000 7f3b0effd700 1 --2- 172.21.15.67:0/1736159240 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3af8004d90 0x7f3af8005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:36.122 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.113+0000 7f3b0effd700 1 -- 172.21.15.67:0/1736159240 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b100b2d70 con 0x7f3af8003f80 2023-10-14T10:36:36.122 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.113+0000 7f3b0effd700 1 -- 172.21.15.67:0/1736159240 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3af800ba10 con 0x7f3af8004d90 2023-10-14T10:36:36.122 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.113+0000 7f3b0effd700 1 -- 172.21.15.67:0/1736159240 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3af800bb50 con 0x7f3b100ce280 2023-10-14T10:36:36.122 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.113+0000 7f3b0f7fe700 1 --2- 172.21.15.67:0/1736159240 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3af8003f80 0x7f3af8004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:36.122 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.113+0000 7f3b0ffff700 1 --2- 172.21.15.67:0/1736159240 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f3b100ce280 0x7f3af8003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:36.123 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.113+0000 7f3b14e1a700 1 --2- 172.21.15.67:0/1736159240 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3af8004d90 0x7f3af8005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:36.123 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.113+0000 7f3b0f7fe700 1 -- 172.21.15.67:0/1736159240 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f3b100ce280 msgr2=0x7f3af8003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:36.123 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.113+0000 7f3b0f7fe700 1 --2- 172.21.15.67:0/1736159240 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f3b100ce280 0x7f3af8003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:36.123 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.113+0000 7f3b0f7fe700 1 -- 172.21.15.67:0/1736159240 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3af8004d90 msgr2=0x7f3af8005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:36.123 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.113+0000 7f3b0f7fe700 1 --2- 172.21.15.67:0/1736159240 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3af8004d90 0x7f3af8005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:36.124 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.113+0000 7f3b0f7fe700 1 -- 172.21.15.67:0/1736159240 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3af80035b0 con 0x7f3af8003f80 2023-10-14T10:36:36.124 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.113+0000 7f3b0f7fe700 1 --2- 172.21.15.67:0/1736159240 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3af8003f80 0x7f3af8004850 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f3b0400c370 tx=0x7f3af8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e394baaa6486aeb7 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:36.124 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.113+0000 7f3b0effd700 1 -- 172.21.15.67:0/1736159240 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b04007c60 con 0x7f3af8003f80 2023-10-14T10:36:36.124 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.113+0000 7f3b0effd700 1 -- 172.21.15.67:0/1736159240 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b0400ce50 con 0x7f3af8003f80 2023-10-14T10:36:36.125 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.113+0000 7f3b0effd700 1 -- 172.21.15.67:0/1736159240 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b04005ba0 con 0x7f3af8003f80 2023-10-14T10:36:36.125 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.114+0000 7f3b1687d700 1 -- 172.21.15.67:0/1736159240 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3af8003f80 msgr2=0x7f3af8004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:36.125 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.114+0000 7f3b1687d700 1 --2- 172.21.15.67:0/1736159240 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3af8003f80 0x7f3af8004850 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f3b0400c370 tx=0x7f3af8005880 comp rx=0 tx=0).stop 2023-10-14T10:36:36.125 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.114+0000 7f3b14e1a700 1 -- 172.21.15.67:0/1736159240 reap_dead start 2023-10-14T10:36:36.125 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.114+0000 7f3b1687d700 1 -- 172.21.15.67:0/1736159240 shutdown_connections 2023-10-14T10:36:36.126 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.114+0000 7f3b1687d700 1 -- 172.21.15.67:0/1736159240 >> 172.21.15.67:0/1736159240 conn(0x7f3b100cc6f0 msgr2=0x7f3b100ccb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:36.126 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.114+0000 7f3b1687d700 1 -- 172.21.15.67:0/1736159240 shutdown_connections 2023-10-14T10:36:36.126 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.114+0000 7f3b1687d700 1 -- 172.21.15.67:0/1736159240 wait complete. 2023-10-14T10:36:36.126 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.115+0000 7f3b1687d700 1 Processor -- start 2023-10-14T10:36:36.126 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.115+0000 7f3b1687d700 1 -- start start 2023-10-14T10:36:36.129 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.116+0000 7f3b1687d700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f3af8003f80 0x7f3b10150f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:36.129 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.116+0000 7f3b1687d700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3b10156e20 0x7f3b10153e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:36.129 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.116+0000 7f3b1687d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3b10154380 0x7f3b101f35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:36.130 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.116+0000 7f3b1687d700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b10154820 con 0x7f3b10154380 2023-10-14T10:36:36.130 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.116+0000 7f3b1687d700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b10151b20 con 0x7f3b10156e20 2023-10-14T10:36:36.130 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.116+0000 7f3b1687d700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b10151e00 con 0x7f3af8003f80 2023-10-14T10:36:36.130 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.116+0000 7f3b14e1a700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3b10154380 0x7f3b101f35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:36.131 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.116+0000 7f3b14e1a700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3b10154380 0x7f3b101f35d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44960/0 (socket says 172.21.15.67:44960) 2023-10-14T10:36:36.131 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.116+0000 7f3b14e1a700 1 -- 172.21.15.67:0/2456844587 learned_addr learned my addr 172.21.15.67:0/2456844587 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:36.131 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.116+0000 7f3b0ffff700 1 --2- 172.21.15.67:0/2456844587 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f3af8003f80 0x7f3b10150f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:36.131 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.116+0000 7f3b0f7fe700 1 --2- 172.21.15.67:0/2456844587 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3b10156e20 0x7f3b10153e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:36.131 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.116+0000 7f3b14e1a700 1 -- 172.21.15.67:0/2456844587 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f3af8003f80 msgr2=0x7f3b10150f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:36.131 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.116+0000 7f3b14e1a700 1 --2- 172.21.15.67:0/2456844587 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f3af8003f80 0x7f3b10150f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:36.132 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.116+0000 7f3b14e1a700 1 -- 172.21.15.67:0/2456844587 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3b10156e20 msgr2=0x7f3b10153e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:36.132 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.116+0000 7f3b14e1a700 1 --2- 172.21.15.67:0/2456844587 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3b10156e20 0x7f3b10153e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:36.132 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.116+0000 7f3b14e1a700 1 -- 172.21.15.67:0/2456844587 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b101520e0 con 0x7f3b10154380 2023-10-14T10:36:36.132 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.116+0000 7f3b0f7fe700 1 --2- 172.21.15.67:0/2456844587 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3b10156e20 0x7f3b10153e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-14T10:36:36.132 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.117+0000 7f3b14e1a700 1 --2- 172.21.15.67:0/2456844587 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3b10154380 0x7f3b101f35d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f3b100cd680 tx=0x7f3b08011f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:36.133 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.117+0000 7f3b0cff9700 1 -- 172.21.15.67:0/2456844587 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b0800e040 con 0x7f3b10154380 2023-10-14T10:36:36.133 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.117+0000 7f3b0cff9700 1 -- 172.21.15.67:0/2456844587 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b0800f040 con 0x7f3b10154380 2023-10-14T10:36:36.133 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.117+0000 7f3b1687d700 1 -- 172.21.15.67:0/2456844587 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b101523c0 con 0x7f3b10154380 2023-10-14T10:36:36.134 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.117+0000 7f3b0cff9700 1 -- 172.21.15.67:0/2456844587 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b0800fdb0 con 0x7f3b10154380 2023-10-14T10:36:36.134 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.117+0000 7f3b1687d700 1 -- 172.21.15.67:0/2456844587 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b101f3b10 con 0x7f3b10154380 2023-10-14T10:36:36.134 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.118+0000 7f3b1687d700 1 -- 172.21.15.67:0/2456844587 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b10067d20 con 0x7f3b10154380 2023-10-14T10:36:36.134 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.118+0000 7f3b0cff9700 1 -- 172.21.15.67:0/2456844587 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f3b08015430 con 0x7f3b10154380 2023-10-14T10:36:36.134 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.119+0000 7f3b0cff9700 1 --2- 172.21.15.67:0/2456844587 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f3afc06af00 0x7f3afc06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:36.135 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.119+0000 7f3b0cff9700 1 -- 172.21.15.67:0/2456844587 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3b08089a00 con 0x7f3b10154380 2023-10-14T10:36:36.135 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.119+0000 7f3b0ffff700 1 --2- 172.21.15.67:0/2456844587 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f3afc06af00 0x7f3afc06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:36.135 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.120+0000 7f3b0ffff700 1 --2- 172.21.15.67:0/2456844587 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f3afc06af00 0x7f3afc06d3c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3b00009ba0 tx=0x7f3b00008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:36.135 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.124+0000 7f3b0cff9700 1 -- 172.21.15.67:0/2456844587 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3b08054b40 con 0x7f3b10154380 2023-10-14T10:36:36.328 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.326+0000 7f3b1687d700 1 -- 172.21.15.67:0/2456844587 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f3b100668b0 con 0x7f3b10154380 2023-10-14T10:36:36.331 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.329+0000 7f3b0cff9700 1 -- 172.21.15.67:0/2456844587 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f3b08058160 con 0x7f3b10154380 2023-10-14T10:36:36.334 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.331+0000 7f3b1687d700 1 -- 172.21.15.67:0/2456844587 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f3afc06af00 msgr2=0x7f3afc06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:36.334 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.331+0000 7f3b1687d700 1 --2- 172.21.15.67:0/2456844587 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f3afc06af00 0x7f3afc06d3c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3b00009ba0 tx=0x7f3b00008040 comp rx=0 tx=0).stop 2023-10-14T10:36:36.334 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.331+0000 7f3b1687d700 1 -- 172.21.15.67:0/2456844587 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3b10154380 msgr2=0x7f3b101f35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:36.335 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.331+0000 7f3b1687d700 1 --2- 172.21.15.67:0/2456844587 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3b10154380 0x7f3b101f35d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f3b100cd680 tx=0x7f3b08011f80 comp rx=0 tx=0).stop 2023-10-14T10:36:36.335 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.331+0000 7f3b1687d700 1 -- 172.21.15.67:0/2456844587 shutdown_connections 2023-10-14T10:36:36.335 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.331+0000 7f3b1687d700 1 --2- 172.21.15.67:0/2456844587 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f3afc06af00 0x7f3afc06d3c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:36.335 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.331+0000 7f3b1687d700 1 --2- 172.21.15.67:0/2456844587 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3b10154380 0x7f3b101f35d0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:36.335 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.331+0000 7f3b1687d700 1 --2- 172.21.15.67:0/2456844587 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f3af8003f80 0x7f3b10150f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:36.336 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.331+0000 7f3b1687d700 1 --2- 172.21.15.67:0/2456844587 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3b10156e20 0x7f3b10153e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:36.336 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.331+0000 7f3b1687d700 1 -- 172.21.15.67:0/2456844587 >> 172.21.15.67:0/2456844587 conn(0x7f3b100cc6f0 msgr2=0x7f3b1015ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:36.336 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.332+0000 7f3b1687d700 1 -- 172.21.15.67:0/2456844587 shutdown_connections 2023-10-14T10:36:36.336 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.332+0000 7f3b1687d700 1 -- 172.21.15.67:0/2456844587 wait complete. 2023-10-14T10:36:36.346 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-14T10:36:36.346 DEBUG:teuthology.orchestra.run.smithi067:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-14T10:36:36.390 DEBUG:teuthology.orchestra.run.smithi072:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-14T10:36:36.446 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-14T10:36:36.447 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2023-10-14T10:36:36.447 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-14T10:36:36.465 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2023-10-14T10:36:36.465 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-14T10:36:36.522 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2023-10-14T10:36:36.523 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-14T10:36:36.580 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-14T10:36:36.744 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.741+0000 7f6ba8a1b700 1 Processor -- start 2023-10-14T10:36:36.744 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.742+0000 7f6ba8a1b700 1 -- start start 2023-10-14T10:36:36.747 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.742+0000 7f6ba8a1b700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f6ba415e030 0x7f6ba40cb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:36.747 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.742+0000 7f6ba8a1b700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f6ba406a840 0x7f6ba40cdb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:36.747 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.743+0000 7f6ba8a1b700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ba406adb0 con 0x7f6ba406be50 2023-10-14T10:36:36.747 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.743+0000 7f6ba8a1b700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ba406b090 con 0x7f6ba415e030 2023-10-14T10:36:36.748 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.743+0000 7f6ba8a1b700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba406b370 con 0x7f6ba406a840 2023-10-14T10:36:36.748 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.743+0000 7f6ba259c700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f6ba415e030 0x7f6ba40cb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:36.748 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.743+0000 7f6ba259c700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f6ba415e030 0x7f6ba40cb3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44964/0 (socket says 172.21.15.67:44964) 2023-10-14T10:36:36.749 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.743+0000 7f6ba259c700 1 -- 172.21.15.67:0/1263123657 learned_addr learned my addr 172.21.15.67:0/1263123657 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:36.749 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.743+0000 7f6ba259c700 1 -- 172.21.15.67:0/1263123657 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f6ba406a840 msgr2=0x7f6ba40cdb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:36.749 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.743+0000 7f6ba1d9b700 1 --2- 172.21.15.67:0/1263123657 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f6ba406a840 0x7f6ba40cdb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:36.749 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.743+0000 7f6ba259c700 1 --2- 172.21.15.67:0/1263123657 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f6ba406a840 0x7f6ba40cdb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:36.749 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.743+0000 7f6ba259c700 1 -- 172.21.15.67:0/1263123657 >> v1:172.21.15.72:6789/0 conn(0x7f6ba406be50 legacy=0x7f6ba406d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:36.750 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.743+0000 7f6ba259c700 1 -- 172.21.15.67:0/1263123657 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ba406b650 con 0x7f6ba415e030 2023-10-14T10:36:36.750 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.743+0000 7f6ba1d9b700 1 --2- 172.21.15.67:0/1263123657 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f6ba406a840 0x7f6ba40cdb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-14T10:36:36.750 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.743+0000 7f6ba259c700 1 --2- 172.21.15.67:0/1263123657 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6ba415e030 0x7f6ba40cb3a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c0098f0 tx=0x7f6b8c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d06708f677622103 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:36.750 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.743+0000 7f6ba159a700 1 -- 172.21.15.67:0/1263123657 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b8c004230 con 0x7f6ba415e030 2023-10-14T10:36:36.751 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.743+0000 7f6ba159a700 1 -- 172.21.15.67:0/1263123657 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b8c004390 con 0x7f6ba415e030 2023-10-14T10:36:36.751 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.743+0000 7f6ba159a700 1 -- 172.21.15.67:0/1263123657 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b8c0036a0 con 0x7f6ba415e030 2023-10-14T10:36:36.751 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.744+0000 7f6ba8a1b700 1 -- 172.21.15.67:0/1263123657 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6ba415e030 msgr2=0x7f6ba40cb3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:36.751 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.744+0000 7f6ba8a1b700 1 --2- 172.21.15.67:0/1263123657 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6ba415e030 0x7f6ba40cb3a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c0098f0 tx=0x7f6b8c009c00 comp rx=0 tx=0).stop 2023-10-14T10:36:36.752 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.744+0000 7f6ba8a1b700 1 -- 172.21.15.67:0/1263123657 shutdown_connections 2023-10-14T10:36:36.752 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.744+0000 7f6ba8a1b700 1 --2- 172.21.15.67:0/1263123657 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6ba415e030 0x7f6ba40cb3a0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:36.752 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.744+0000 7f6ba8a1b700 1 --2- 172.21.15.67:0/1263123657 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f6ba406a840 0x7f6ba40cdb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:36.752 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.744+0000 7f6ba8a1b700 1 -- 172.21.15.67:0/1263123657 >> 172.21.15.67:0/1263123657 conn(0x7f6ba40cc6f0 msgr2=0x7f6ba40ccb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:36.752 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.744+0000 7f6ba8a1b700 1 -- 172.21.15.67:0/1263123657 shutdown_connections 2023-10-14T10:36:36.753 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.744+0000 7f6ba8a1b700 1 -- 172.21.15.67:0/1263123657 wait complete. 2023-10-14T10:36:36.755 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.745+0000 7f6ba8a1b700 1 Processor -- start 2023-10-14T10:36:36.756 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.745+0000 7f6ba8a1b700 1 -- start start 2023-10-14T10:36:36.756 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.746+0000 7f6ba8a1b700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f6ba406a840 0x7f6ba415f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:36.756 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.746+0000 7f6ba8a1b700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6ba406be50 0x7f6ba41f6be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:36.756 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.746+0000 7f6ba8a1b700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6ba415e030 0x7f6ba40c9090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:36.756 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.746+0000 7f6ba8a1b700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba41f3990 con 0x7f6ba406be50 2023-10-14T10:36:36.757 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.746+0000 7f6ba8a1b700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba41f7430 con 0x7f6ba415e030 2023-10-14T10:36:36.757 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.746+0000 7f6ba8a1b700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba41f7710 con 0x7f6ba406a840 2023-10-14T10:36:36.757 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.746+0000 7f6ba1d9b700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6ba406be50 0x7f6ba41f6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:36.757 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.746+0000 7f6ba1d9b700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6ba406be50 0x7f6ba41f6be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44972/0 (socket says 172.21.15.67:44972) 2023-10-14T10:36:36.757 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.746+0000 7f6ba1d9b700 1 -- 172.21.15.67:0/2285026358 learned_addr learned my addr 172.21.15.67:0/2285026358 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:36.758 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.746+0000 7f6ba259c700 1 --2- 172.21.15.67:0/2285026358 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f6ba406a840 0x7f6ba415f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:36.758 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.746+0000 7f6ba2d9d700 1 --2- 172.21.15.67:0/2285026358 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6ba415e030 0x7f6ba40c9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:36.758 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.746+0000 7f6ba1d9b700 1 -- 172.21.15.67:0/2285026358 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f6ba406a840 msgr2=0x7f6ba415f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:36.758 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.746+0000 7f6ba1d9b700 1 --2- 172.21.15.67:0/2285026358 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f6ba406a840 0x7f6ba415f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:36.758 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.746+0000 7f6ba1d9b700 1 -- 172.21.15.67:0/2285026358 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6ba415e030 msgr2=0x7f6ba40c9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:36.759 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.746+0000 7f6ba1d9b700 1 --2- 172.21.15.67:0/2285026358 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6ba415e030 0x7f6ba40c9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:36.759 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.746+0000 7f6ba1d9b700 1 -- 172.21.15.67:0/2285026358 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b8c009550 con 0x7f6ba406be50 2023-10-14T10:36:36.759 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.746+0000 7f6ba2d9d700 1 --2- 172.21.15.67:0/2285026358 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6ba415e030 0x7f6ba40c9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-14T10:36:36.759 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.746+0000 7f6ba259c700 1 --2- 172.21.15.67:0/2285026358 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f6ba406a840 0x7f6ba415f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-14T10:36:36.759 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.747+0000 7f6ba1d9b700 1 --2- 172.21.15.67:0/2285026358 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6ba406be50 0x7f6ba41f6be0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6b9400c7f0 tx=0x7f6b9400cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:36.760 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.747+0000 7f6b9affd700 1 -- 172.21.15.67:0/2285026358 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b94011800 con 0x7f6ba406be50 2023-10-14T10:36:36.760 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.747+0000 7f6b9affd700 1 -- 172.21.15.67:0/2285026358 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b94011960 con 0x7f6ba406be50 2023-10-14T10:36:36.760 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.747+0000 7f6ba8a1b700 1 -- 172.21.15.67:0/2285026358 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ba41f79f0 con 0x7f6ba406be50 2023-10-14T10:36:36.760 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.747+0000 7f6b9affd700 1 -- 172.21.15.67:0/2285026358 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b94011d10 con 0x7f6ba406be50 2023-10-14T10:36:36.760 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.747+0000 7f6ba8a1b700 1 -- 172.21.15.67:0/2285026358 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ba40c95d0 con 0x7f6ba406be50 2023-10-14T10:36:36.761 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.748+0000 7f6ba8a1b700 1 -- 172.21.15.67:0/2285026358 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ba41f3b40 con 0x7f6ba406be50 2023-10-14T10:36:36.761 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.748+0000 7f6b9affd700 1 -- 172.21.15.67:0/2285026358 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f6b94018360 con 0x7f6ba406be50 2023-10-14T10:36:36.761 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.749+0000 7f6b9affd700 1 --2- 172.21.15.67:0/2285026358 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f6b8406af00 0x7f6b8406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:36.761 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.749+0000 7f6b9affd700 1 -- 172.21.15.67:0/2285026358 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6b94089580 con 0x7f6ba406be50 2023-10-14T10:36:36.761 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.749+0000 7f6ba259c700 1 --2- 172.21.15.67:0/2285026358 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f6b8406af00 0x7f6b8406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:36.762 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.749+0000 7f6ba259c700 1 --2- 172.21.15.67:0/2285026358 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f6b8406af00 0x7f6b8406d3c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6ba40c7b30 tx=0x7f6b8c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:36.762 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.753+0000 7f6b9affd700 1 -- 172.21.15.67:0/2285026358 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6ba41f3b40 con 0x7f6ba406be50 2023-10-14T10:36:36.958 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.956+0000 7f6ba8a1b700 1 -- 172.21.15.67:0/2285026358 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6ba41f3ad0 con 0x7f6ba406be50 2023-10-14T10:36:36.959 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.957+0000 7f6b9affd700 1 -- 172.21.15.67:0/2285026358 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16457 (secure 0 0 0) 0x7f6b9405e070 con 0x7f6ba406be50 2023-10-14T10:36:36.960 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-14T10:36:36.960 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":22,"fsid":"37f9ab46-3082-4b71-8b24-08d39897b008","created":"2023-10-14T10:35:46.067554+0000","modified":"2023-10-14T10:36:04.837296+0000","last_up_change":"2023-10-14T10:35:56.668142+0000","last_in_change":"2023-10-14T10:35:48.551490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-14T10:35:57.809566+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-14T10:36:01.667146+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-14T10:36:02.219700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2b59a07e-11ef-440c-b766-9f64ac885838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6801","nonce":1287255499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6803","nonce":1287255499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6807","nonce":1287255499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6805","nonce":1287255499}]},"public_addr":"172.21.15.67:6801/1287255499","cluster_addr":"172.21.15.67:6803/1287255499","heartbeat_back_addr":"172.21.15.67:6807/1287255499","heartbeat_front_addr":"172.21.15.67:6805/1287255499","state":["exists","up"]},{"osd":1,"uuid":"6f813bd6-fa0b-487d-a204-431eb1276a60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6818","nonce":2435538787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6822","nonce":2435538787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6830","nonce":2435538787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6826","nonce":2435538787}]},"public_addr":"172.21.15.67:6818/2435538787","cluster_addr":"172.21.15.67:6822/2435538787","heartbeat_back_addr":"172.21.15.67:6830/2435538787","heartbeat_front_addr":"172.21.15.67:6826/2435538787","state":["exists","up"]},{"osd":2,"uuid":"6af42c6a-1b59-4669-9c04-b6876788446d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6809","nonce":2827491791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6811","nonce":2827491791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6815","nonce":2827491791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6813","nonce":2827491791}]},"public_addr":"172.21.15.67:6809/2827491791","cluster_addr":"172.21.15.67:6811/2827491791","heartbeat_back_addr":"172.21.15.67:6815/2827491791","heartbeat_front_addr":"172.21.15.67:6813/2827491791","state":["exists","up"]},{"osd":3,"uuid":"1d889d76-a646-499d-bfff-ed5bae5434ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6819","nonce":1848001176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6821","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6823","nonce":1848001176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6831","nonce":1848001176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6825","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6827","nonce":1848001176}]},"public_addr":"172.21.15.67:6819/1848001176","cluster_addr":"172.21.15.67:6823/1848001176","heartbeat_back_addr":"172.21.15.67:6831/1848001176","heartbeat_front_addr":"172.21.15.67:6827/1848001176","state":["exists","up"]},{"osd":4,"uuid":"c3271dff-b437-4fdd-b13b-6e49a35473af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6804","nonce":1891230785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6808","nonce":1891230785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6815","nonce":1891230785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6811","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6813","nonce":1891230785}]},"public_addr":"172.21.15.72:6804/1891230785","cluster_addr":"172.21.15.72:6808/1891230785","heartbeat_back_addr":"172.21.15.72:6815/1891230785","heartbeat_front_addr":"172.21.15.72:6813/1891230785","state":["exists","up"]},{"osd":5,"uuid":"0337bb18-4b68-4eff-b528-a667275e6926","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6801","nonce":569686991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6803","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6805","nonce":569686991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6812","nonce":569686991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6807","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6809","nonce":569686991}]},"public_addr":"172.21.15.72:6801/569686991","cluster_addr":"172.21.15.72:6805/569686991","heartbeat_back_addr":"172.21.15.72:6812/569686991","heartbeat_front_addr":"172.21.15.72:6809/569686991","state":["exists","up"]},{"osd":6,"uuid":"8e92d9cf-0c8c-46eb-89a7-db2023ef1509","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6817","nonce":3767685934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6819","nonce":3767685934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6823","nonce":3767685934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6821","nonce":3767685934}]},"public_addr":"172.21.15.72:6817/3767685934","cluster_addr":"172.21.15.72:6819/3767685934","heartbeat_back_addr":"172.21.15.72:6823/3767685934","heartbeat_front_addr":"172.21.15.72:6821/3767685934","state":["exists","up"]},{"osd":7,"uuid":"3fd6aa7d-50a9-45c9-9561-5902a044a119","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6825","nonce":3567465926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6827","nonce":3567465926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6831","nonce":3567465926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6829","nonce":3567465926}]},"public_addr":"172.21.15.72:6825/3567465926","cluster_addr":"172.21.15.72:6827/3567465926","heartbeat_back_addr":"172.21.15.72:6831/3567465926","heartbeat_front_addr":"172.21.15.72:6829/3567465926","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.511466+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.795107+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.858674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.728895+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.794842+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.750064+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.732122+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.765185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-14T10:36:36.962 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.959+0000 7f6ba8a1b700 1 -- 172.21.15.67:0/2285026358 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f6b8406af00 msgr2=0x7f6b8406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:36.962 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.959+0000 7f6ba8a1b700 1 --2- 172.21.15.67:0/2285026358 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f6b8406af00 0x7f6b8406d3c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6ba40c7b30 tx=0x7f6b8c008030 comp rx=0 tx=0).stop 2023-10-14T10:36:36.963 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.959+0000 7f6ba8a1b700 1 -- 172.21.15.67:0/2285026358 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6ba406be50 msgr2=0x7f6ba41f6be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:36.963 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.959+0000 7f6ba8a1b700 1 --2- 172.21.15.67:0/2285026358 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6ba406be50 0x7f6ba41f6be0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6b9400c7f0 tx=0x7f6b9400cbb0 comp rx=0 tx=0).stop 2023-10-14T10:36:36.963 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.959+0000 7f6ba8a1b700 1 -- 172.21.15.67:0/2285026358 shutdown_connections 2023-10-14T10:36:36.963 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.959+0000 7f6ba8a1b700 1 --2- 172.21.15.67:0/2285026358 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f6b8406af00 0x7f6b8406d3c0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:36.964 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.959+0000 7f6ba8a1b700 1 --2- 172.21.15.67:0/2285026358 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6ba415e030 0x7f6ba40c9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:36.964 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.959+0000 7f6ba8a1b700 1 --2- 172.21.15.67:0/2285026358 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f6ba406a840 0x7f6ba415f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:36.964 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.959+0000 7f6ba8a1b700 1 --2- 172.21.15.67:0/2285026358 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6ba406be50 0x7f6ba41f6be0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:36.964 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.959+0000 7f6ba8a1b700 1 -- 172.21.15.67:0/2285026358 >> 172.21.15.67:0/2285026358 conn(0x7f6ba40cc6f0 msgr2=0x7f6ba415b9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:36.965 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.959+0000 7f6ba8a1b700 1 -- 172.21.15.67:0/2285026358 shutdown_connections 2023-10-14T10:36:36.965 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:36.959+0000 7f6ba8a1b700 1 -- 172.21.15.67:0/2285026358 wait complete. 2023-10-14T10:36:36.974 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-14T10:35:57.809566+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-14T10:36:01.667146+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-14T10:36:02.219700+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-14T10:36:36.975 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-14T10:36:37.103 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.101+0000 7f42502a0700 1 Processor -- start 2023-10-14T10:36:37.103 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.101+0000 7f42502a0700 1 -- start start 2023-10-14T10:36:37.107 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.102+0000 7f42502a0700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f42480cd950 0x7f424815e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:37.107 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.102+0000 7f42502a0700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f42480cede0 0x7f424815f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:37.107 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.102+0000 7f42502a0700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42480cf260 con 0x7f424806a840 2023-10-14T10:36:37.107 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.102+0000 7f42502a0700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42481619a0 con 0x7f42480cd950 2023-10-14T10:36:37.108 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.102+0000 7f42502a0700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42480b2d60 con 0x7f42480cede0 2023-10-14T10:36:37.108 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.102+0000 7f424e83d700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f424806a840 0x7f4248161df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:52750/0 (socket says 172.21.15.67:52750) 2023-10-14T10:36:37.108 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.102+0000 7f424e83d700 1 -- 172.21.15.67:0/1918916187 learned_addr learned my addr 172.21.15.67:0/1918916187 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:36:37.108 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.102+0000 7f424e03c700 1 --2- 172.21.15.67:0/1918916187 >> v2:172.21.15.72:3300/0 conn(0x7f42480cd950 0x7f424815e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:37.108 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.102+0000 7f424d83b700 1 --2- 172.21.15.67:0/1918916187 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f42480cede0 0x7f424815f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:37.109 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.102+0000 7f424d03a700 1 -- 172.21.15.67:0/1918916187 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2506260439 0 0) 0x7f42480cf260 con 0x7f424806a840 2023-10-14T10:36:37.109 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.102+0000 7f424d03a700 1 -- 172.21.15.67:0/1918916187 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42340035b0 con 0x7f424806a840 2023-10-14T10:36:37.109 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.102+0000 7f424d03a700 1 -- 172.21.15.67:0/1918916187 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 591813706 0 0) 0x7f42340035b0 con 0x7f424806a840 2023-10-14T10:36:37.109 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.102+0000 7f424d03a700 1 -- 172.21.15.67:0/1918916187 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f42480cede0 msgr2=0x7f424815f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:37.109 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.102+0000 7f424d03a700 1 --2- 172.21.15.67:0/1918916187 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f42480cede0 0x7f424815f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:37.110 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.102+0000 7f424d03a700 1 -- 172.21.15.67:0/1918916187 >> v2:172.21.15.72:3300/0 conn(0x7f42480cd950 msgr2=0x7f424815e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:37.110 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.102+0000 7f424d03a700 1 --2- 172.21.15.67:0/1918916187 >> v2:172.21.15.72:3300/0 conn(0x7f42480cd950 0x7f424815e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:37.110 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.102+0000 7f424d03a700 1 -- 172.21.15.67:0/1918916187 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42480cdd30 con 0x7f424806a840 2023-10-14T10:36:37.110 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.102+0000 7f424d03a700 1 -- 172.21.15.67:0/1918916187 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3180043755 0 0) 0x7f42400019f0 con 0x7f424806a840 2023-10-14T10:36:37.110 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.102+0000 7f424d03a700 1 -- 172.21.15.67:0/1918916187 >> v1:172.21.15.67:6789/0 conn(0x7f424806a840 legacy=0x7f4248161df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:37.111 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.102+0000 7f424d03a700 1 --2- 172.21.15.67:0/1918916187 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f42480cede0 0x7f4234003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:37.111 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.103+0000 7f424d03a700 1 --2- 172.21.15.67:0/1918916187 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4234003fd0 0x7f4234004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:37.111 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.103+0000 7f424d03a700 1 --2- 172.21.15.67:0/1918916187 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4234004d90 0x7f4234005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:37.111 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.103+0000 7f424d03a700 1 -- 172.21.15.67:0/1918916187 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42481619a0 con 0x7f4234004d90 2023-10-14T10:36:37.111 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.103+0000 7f424d03a700 1 -- 172.21.15.67:0/1918916187 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42480b2d60 con 0x7f4234003fd0 2023-10-14T10:36:37.111 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.103+0000 7f424d03a700 1 -- 172.21.15.67:0/1918916187 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f423400ba10 con 0x7f42480cede0 2023-10-14T10:36:37.112 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.103+0000 7f424e83d700 1 --2- 172.21.15.67:0/1918916187 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4234004d90 0x7f4234005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:37.112 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.103+0000 7f424d83b700 1 --2- 172.21.15.67:0/1918916187 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4234003fd0 0x7f4234004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:37.112 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.103+0000 7f424e03c700 1 --2- 172.21.15.67:0/1918916187 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f42480cede0 0x7f4234003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:37.112 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.103+0000 7f424e83d700 1 -- 172.21.15.67:0/1918916187 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f42480cede0 msgr2=0x7f4234003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:37.112 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.103+0000 7f424e83d700 1 --2- 172.21.15.67:0/1918916187 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f42480cede0 0x7f4234003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:37.113 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.103+0000 7f424e83d700 1 -- 172.21.15.67:0/1918916187 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4234003fd0 msgr2=0x7f4234004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:37.113 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.103+0000 7f424e83d700 1 --2- 172.21.15.67:0/1918916187 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4234003fd0 0x7f4234004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:37.113 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.103+0000 7f424e83d700 1 -- 172.21.15.67:0/1918916187 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42340035b0 con 0x7f4234004d90 2023-10-14T10:36:37.113 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.103+0000 7f424e83d700 1 --2- 172.21.15.67:0/1918916187 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4234004d90 0x7f4234005210 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f424000a7b0 tx=0x7f4234005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abaa7473d8dfc2cf server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:37.113 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.103+0000 7f424d03a700 1 -- 172.21.15.67:0/1918916187 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42400072d0 con 0x7f4234004d90 2023-10-14T10:36:37.113 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.103+0000 7f424d03a700 1 -- 172.21.15.67:0/1918916187 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4240007430 con 0x7f4234004d90 2023-10-14T10:36:37.114 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.103+0000 7f424d03a700 1 -- 172.21.15.67:0/1918916187 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42400077e0 con 0x7f4234004d90 2023-10-14T10:36:37.114 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.104+0000 7f42502a0700 1 -- 172.21.15.67:0/1918916187 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4234004d90 msgr2=0x7f4234005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:37.114 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.104+0000 7f42502a0700 1 --2- 172.21.15.67:0/1918916187 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4234004d90 0x7f4234005210 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f424000a7b0 tx=0x7f4234005880 comp rx=0 tx=0).stop 2023-10-14T10:36:37.114 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.104+0000 7f424e83d700 1 -- 172.21.15.67:0/1918916187 reap_dead start 2023-10-14T10:36:37.114 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.104+0000 7f42502a0700 1 -- 172.21.15.67:0/1918916187 shutdown_connections 2023-10-14T10:36:37.115 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.104+0000 7f42502a0700 1 -- 172.21.15.67:0/1918916187 >> 172.21.15.67:0/1918916187 conn(0x7f424806dae0 msgr2=0x7f424806def0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:37.115 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.104+0000 7f42502a0700 1 -- 172.21.15.67:0/1918916187 shutdown_connections 2023-10-14T10:36:37.115 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.104+0000 7f42502a0700 1 -- 172.21.15.67:0/1918916187 wait complete. 2023-10-14T10:36:37.118 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.105+0000 7f42502a0700 1 Processor -- start 2023-10-14T10:36:37.118 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.106+0000 7f42502a0700 1 -- start start 2023-10-14T10:36:37.118 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.106+0000 7f42502a0700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f4234004d90 0x7f42480ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:37.118 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.106+0000 7f42502a0700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f42480acdf0 0x7f42480ae8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:37.118 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.106+0000 7f42502a0700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f42480aede0 0x7f42480af260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:37.119 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.107+0000 7f42502a0700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42480ad270 con 0x7f42480acdf0 2023-10-14T10:36:37.119 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.107+0000 7f42502a0700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42480b4450 con 0x7f42480aede0 2023-10-14T10:36:37.119 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.107+0000 7f42502a0700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42480af7a0 con 0x7f4234004d90 2023-10-14T10:36:37.119 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.107+0000 7f424d83b700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f42480acdf0 0x7f42480ae8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:37.119 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.107+0000 7f424d83b700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f42480acdf0 0x7f42480ae8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44992/0 (socket says 172.21.15.67:44992) 2023-10-14T10:36:37.120 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.107+0000 7f424d83b700 1 -- 172.21.15.67:0/2646105701 learned_addr learned my addr 172.21.15.67:0/2646105701 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:37.120 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.107+0000 7f424e83d700 1 --2- 172.21.15.67:0/2646105701 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f42480aede0 0x7f42480af260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:37.120 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.107+0000 7f424e03c700 1 --2- 172.21.15.67:0/2646105701 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f4234004d90 0x7f42480ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:37.120 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.107+0000 7f424d83b700 1 -- 172.21.15.67:0/2646105701 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f4234004d90 msgr2=0x7f42480ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:37.120 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.107+0000 7f424d83b700 1 --2- 172.21.15.67:0/2646105701 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f4234004d90 0x7f42480ac8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:37.121 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.107+0000 7f424d83b700 1 -- 172.21.15.67:0/2646105701 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f42480aede0 msgr2=0x7f42480af260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:37.121 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.107+0000 7f424d83b700 1 --2- 172.21.15.67:0/2646105701 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f42480aede0 0x7f42480af260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:37.121 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.107+0000 7f424d83b700 1 -- 172.21.15.67:0/2646105701 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42480b3840 con 0x7f42480acdf0 2023-10-14T10:36:37.121 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.107+0000 7f424e03c700 1 --2- 172.21.15.67:0/2646105701 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f4234004d90 0x7f42480ac8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-14T10:36:37.121 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.107+0000 7f424d83b700 1 --2- 172.21.15.67:0/2646105701 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f42480acdf0 0x7f42480ae8a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4238009fd0 tx=0x7f4238011c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:37.122 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.107+0000 7f423effd700 1 -- 172.21.15.67:0/2646105701 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4238011ec0 con 0x7f42480acdf0 2023-10-14T10:36:37.122 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.107+0000 7f423effd700 1 -- 172.21.15.67:0/2646105701 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4238005c30 con 0x7f42480acdf0 2023-10-14T10:36:37.122 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.107+0000 7f42502a0700 1 -- 172.21.15.67:0/2646105701 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42480b3b20 con 0x7f42480acdf0 2023-10-14T10:36:37.122 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.107+0000 7f423effd700 1 -- 172.21.15.67:0/2646105701 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f423800f6d0 con 0x7f42480acdf0 2023-10-14T10:36:37.122 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.107+0000 7f42502a0700 1 -- 172.21.15.67:0/2646105701 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42480b40e0 con 0x7f42480acdf0 2023-10-14T10:36:37.123 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.108+0000 7f42502a0700 1 -- 172.21.15.67:0/2646105701 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42480b42a0 con 0x7f42480acdf0 2023-10-14T10:36:37.123 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.109+0000 7f423effd700 1 -- 172.21.15.67:0/2646105701 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f423800f830 con 0x7f42480acdf0 2023-10-14T10:36:37.123 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.109+0000 7f423effd700 1 --2- 172.21.15.67:0/2646105701 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f422c06af00 0x7f422c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:37.123 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.109+0000 7f423effd700 1 -- 172.21.15.67:0/2646105701 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4238089bb0 con 0x7f42480acdf0 2023-10-14T10:36:37.123 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.109+0000 7f424e03c700 1 --2- 172.21.15.67:0/2646105701 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f422c06af00 0x7f422c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:37.123 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.110+0000 7f424e03c700 1 --2- 172.21.15.67:0/2646105701 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f422c06af00 0x7f422c06d3c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4244009ce0 tx=0x7f4244008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:37.124 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.114+0000 7f423effd700 1 -- 172.21.15.67:0/2646105701 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4238054c60 con 0x7f42480acdf0 2023-10-14T10:36:37.313 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.311+0000 7f42502a0700 1 -- 172.21.15.67:0/2646105701 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4248067d20 con 0x7f42480acdf0 2023-10-14T10:36:37.314 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.312+0000 7f423effd700 1 -- 172.21.15.67:0/2646105701 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4238021020 con 0x7f42480acdf0 2023-10-14T10:36:37.314 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2023-10-14T10:36:37.316 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.314+0000 7f42502a0700 1 -- 172.21.15.67:0/2646105701 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f422c06af00 msgr2=0x7f422c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:37.316 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.314+0000 7f42502a0700 1 --2- 172.21.15.67:0/2646105701 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f422c06af00 0x7f422c06d3c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4244009ce0 tx=0x7f4244008040 comp rx=0 tx=0).stop 2023-10-14T10:36:37.317 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.314+0000 7f42502a0700 1 -- 172.21.15.67:0/2646105701 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f42480acdf0 msgr2=0x7f42480ae8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:37.317 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.314+0000 7f42502a0700 1 --2- 172.21.15.67:0/2646105701 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f42480acdf0 0x7f42480ae8a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4238009fd0 tx=0x7f4238011c80 comp rx=0 tx=0).stop 2023-10-14T10:36:37.317 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.314+0000 7f42502a0700 1 -- 172.21.15.67:0/2646105701 shutdown_connections 2023-10-14T10:36:37.317 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.314+0000 7f42502a0700 1 --2- 172.21.15.67:0/2646105701 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f422c06af00 0x7f422c06d3c0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:37.317 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.314+0000 7f42502a0700 1 --2- 172.21.15.67:0/2646105701 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f42480aede0 0x7f42480af260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:37.318 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.314+0000 7f42502a0700 1 --2- 172.21.15.67:0/2646105701 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f4234004d90 0x7f42480ac8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:37.318 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.314+0000 7f42502a0700 1 --2- 172.21.15.67:0/2646105701 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f42480acdf0 0x7f42480ae8a0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:37.318 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.314+0000 7f42502a0700 1 -- 172.21.15.67:0/2646105701 >> 172.21.15.67:0/2646105701 conn(0x7f424806dae0 msgr2=0x7f424815f550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:37.318 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.314+0000 7f42502a0700 1 -- 172.21.15.67:0/2646105701 shutdown_connections 2023-10-14T10:36:37.318 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.314+0000 7f42502a0700 1 -- 172.21.15.67:0/2646105701 wait complete. 2023-10-14T10:36:37.328 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-14T10:36:37.456 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.454+0000 7fadc0e14700 1 Processor -- start 2023-10-14T10:36:37.457 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.454+0000 7fadc0e14700 1 -- start start 2023-10-14T10:36:37.459 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.455+0000 7fadc0e14700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fadbc15b740 0x7fadbc0d1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:37.459 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.455+0000 7fadc0e14700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fadbc0ce6e0 0x7fadbc0cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:37.460 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.455+0000 7fadc0e14700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadbc0cf190 con 0x7fadbc0d0c70 2023-10-14T10:36:37.460 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.455+0000 7fadc0e14700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadbc15bb20 con 0x7fadbc15b740 2023-10-14T10:36:37.460 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.455+0000 7fadc0e14700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadbc0cbd40 con 0x7fadbc0ce6e0 2023-10-14T10:36:37.460 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.455+0000 7fadba59c700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fadbc15b740 0x7fadbc0d1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:37.460 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.455+0000 7fadba59c700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fadbc15b740 0x7fadbc0d1cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44998/0 (socket says 172.21.15.67:44998) 2023-10-14T10:36:37.461 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.455+0000 7fadba59c700 1 -- 172.21.15.67:0/16941634 learned_addr learned my addr 172.21.15.67:0/16941634 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:37.461 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.456+0000 7fadbad9d700 1 --2- 172.21.15.67:0/16941634 >> v2:172.21.15.72:3300/0 conn(0x7fadbc0ce6e0 0x7fadbc0cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:37.461 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.456+0000 7fadba59c700 1 -- 172.21.15.67:0/16941634 >> v2:172.21.15.72:3300/0 conn(0x7fadbc0ce6e0 msgr2=0x7fadbc0cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:37.461 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.456+0000 7fadba59c700 1 --2- 172.21.15.67:0/16941634 >> v2:172.21.15.72:3300/0 conn(0x7fadbc0ce6e0 0x7fadbc0cf690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:37.461 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.456+0000 7fadba59c700 1 -- 172.21.15.67:0/16941634 >> v1:172.21.15.72:6789/0 conn(0x7fadbc0d0c70 legacy=0x7fadbc0cdfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:37.462 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.456+0000 7fadba59c700 1 -- 172.21.15.67:0/16941634 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadbc0cc020 con 0x7fadbc15b740 2023-10-14T10:36:37.462 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.456+0000 7fadba59c700 1 --2- 172.21.15.67:0/16941634 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fadbc15b740 0x7fadbc0d1cd0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fadbc0c8620 tx=0x7fadb0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf0f9d60580f4447 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:37.462 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.456+0000 7fadb959a700 1 -- 172.21.15.67:0/16941634 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb0004230 con 0x7fadbc15b740 2023-10-14T10:36:37.462 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.456+0000 7fadb959a700 1 -- 172.21.15.67:0/16941634 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadb0004390 con 0x7fadbc15b740 2023-10-14T10:36:37.463 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.456+0000 7fadb959a700 1 -- 172.21.15.67:0/16941634 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb00036a0 con 0x7fadbc15b740 2023-10-14T10:36:37.463 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.457+0000 7fadc0e14700 1 -- 172.21.15.67:0/16941634 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fadbc15b740 msgr2=0x7fadbc0d1cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:37.463 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.457+0000 7fadc0e14700 1 --2- 172.21.15.67:0/16941634 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fadbc15b740 0x7fadbc0d1cd0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fadbc0c8620 tx=0x7fadb0009bd0 comp rx=0 tx=0).stop 2023-10-14T10:36:37.463 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.457+0000 7fadc0e14700 1 -- 172.21.15.67:0/16941634 shutdown_connections 2023-10-14T10:36:37.463 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.457+0000 7fadc0e14700 1 --2- 172.21.15.67:0/16941634 >> v2:172.21.15.72:3300/0 conn(0x7fadbc0ce6e0 0x7fadbc0cf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:37.464 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.457+0000 7fadc0e14700 1 --2- 172.21.15.67:0/16941634 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fadbc15b740 0x7fadbc0d1cd0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:37.464 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.457+0000 7fadc0e14700 1 -- 172.21.15.67:0/16941634 >> 172.21.15.67:0/16941634 conn(0x7fadbc0665a0 msgr2=0x7fadbc066980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:37.464 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.457+0000 7fadc0e14700 1 -- 172.21.15.67:0/16941634 shutdown_connections 2023-10-14T10:36:37.464 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.457+0000 7fadc0e14700 1 -- 172.21.15.67:0/16941634 wait complete. 2023-10-14T10:36:37.467 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.458+0000 7fadc0e14700 1 Processor -- start 2023-10-14T10:36:37.467 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.458+0000 7fadc0e14700 1 -- start start 2023-10-14T10:36:37.467 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.459+0000 7fadc0e14700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fadbc0ce6e0 0x7fadbc0b6d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:37.468 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.459+0000 7fadc0e14700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fadbc0d0c70 0x7fadbc0cadd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:37.468 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.459+0000 7fadc0e14700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fadbc15b740 0x7fadbc1f4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:37.468 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.459+0000 7fadc0e14700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadbc1f50e0 con 0x7fadbc15b740 2023-10-14T10:36:37.468 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.459+0000 7fadc0e14700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadbc0b7250 con 0x7fadbc0ce6e0 2023-10-14T10:36:37.469 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.459+0000 7fadc0e14700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadbc0ca010 con 0x7fadbc0d0c70 2023-10-14T10:36:37.469 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.459+0000 7fadbad9d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fadbc15b740 0x7fadbc1f4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:37.469 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.459+0000 7fadbad9d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fadbc15b740 0x7fadbc1f4470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45008/0 (socket says 172.21.15.67:45008) 2023-10-14T10:36:37.469 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.459+0000 7fadbad9d700 1 -- 172.21.15.67:0/2659885748 learned_addr learned my addr 172.21.15.67:0/2659885748 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:37.469 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.459+0000 7fadba59c700 1 --2- 172.21.15.67:0/2659885748 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fadbc0ce6e0 0x7fadbc0b6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:37.470 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.459+0000 7fadb9d9b700 1 --2- 172.21.15.67:0/2659885748 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fadbc0d0c70 0x7fadbc0cadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:37.470 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.459+0000 7fadbad9d700 1 -- 172.21.15.67:0/2659885748 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fadbc0d0c70 msgr2=0x7fadbc0cadd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:37.470 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.459+0000 7fadbad9d700 1 --2- 172.21.15.67:0/2659885748 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fadbc0d0c70 0x7fadbc0cadd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:37.470 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.459+0000 7fadbad9d700 1 -- 172.21.15.67:0/2659885748 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fadbc0ce6e0 msgr2=0x7fadbc0b6d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:37.470 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.459+0000 7fadbad9d700 1 --2- 172.21.15.67:0/2659885748 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fadbc0ce6e0 0x7fadbc0b6d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:37.471 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.459+0000 7fadbad9d700 1 -- 172.21.15.67:0/2659885748 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadb0009550 con 0x7fadbc15b740 2023-10-14T10:36:37.471 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.459+0000 7fadbad9d700 1 --2- 172.21.15.67:0/2659885748 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fadbc15b740 0x7fadbc1f4470 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fadb400cc00 tx=0x7fadb400a1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:37.471 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.460+0000 7fadaaffd700 1 -- 172.21.15.67:0/2659885748 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb400aad0 con 0x7fadbc15b740 2023-10-14T10:36:37.471 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.460+0000 7fadaaffd700 1 -- 172.21.15.67:0/2659885748 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadb400ac30 con 0x7fadbc15b740 2023-10-14T10:36:37.471 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.460+0000 7fadc0e14700 1 -- 172.21.15.67:0/2659885748 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadbc0ca2f0 con 0x7fadbc15b740 2023-10-14T10:36:37.472 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.460+0000 7fadaaffd700 1 -- 172.21.15.67:0/2659885748 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb4003a20 con 0x7fadbc15b740 2023-10-14T10:36:37.472 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.460+0000 7fadc0e14700 1 -- 172.21.15.67:0/2659885748 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadbc0ca680 con 0x7fadbc15b740 2023-10-14T10:36:37.472 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.461+0000 7fadc0e14700 1 -- 172.21.15.67:0/2659885748 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadbc0caac0 con 0x7fadbc15b740 2023-10-14T10:36:37.472 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.461+0000 7fadaaffd700 1 -- 172.21.15.67:0/2659885748 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fadb400f3f0 con 0x7fadbc15b740 2023-10-14T10:36:37.472 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.461+0000 7fadaaffd700 1 --2- 172.21.15.67:0/2659885748 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fad9c06af00 0x7fad9c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:37.473 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.461+0000 7fadaaffd700 1 -- 172.21.15.67:0/2659885748 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fadb4089660 con 0x7fadbc15b740 2023-10-14T10:36:37.473 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.462+0000 7fadba59c700 1 --2- 172.21.15.67:0/2659885748 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fad9c06af00 0x7fad9c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:37.473 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.462+0000 7fadba59c700 1 --2- 172.21.15.67:0/2659885748 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fad9c06af00 0x7fad9c06d3c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fadb0009130 tx=0x7fadb0008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:37.474 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.466+0000 7fadaaffd700 1 -- 172.21.15.67:0/2659885748 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fadb40547a0 con 0x7fadbc15b740 2023-10-14T10:36:37.665 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.663+0000 7fadc0e14700 1 -- 172.21.15.67:0/2659885748 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fadbc0c9b60 con 0x7fadbc15b740 2023-10-14T10:36:37.666 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.664+0000 7fadaaffd700 1 -- 172.21.15.67:0/2659885748 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fadb401d020 con 0x7fadbc15b740 2023-10-14T10:36:37.666 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2023-10-14T10:36:37.668 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.666+0000 7fadc0e14700 1 -- 172.21.15.67:0/2659885748 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fad9c06af00 msgr2=0x7fad9c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:37.668 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.666+0000 7fadc0e14700 1 --2- 172.21.15.67:0/2659885748 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fad9c06af00 0x7fad9c06d3c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fadb0009130 tx=0x7fadb0008030 comp rx=0 tx=0).stop 2023-10-14T10:36:37.669 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.666+0000 7fadc0e14700 1 -- 172.21.15.67:0/2659885748 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fadbc15b740 msgr2=0x7fadbc1f4470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:37.669 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.666+0000 7fadc0e14700 1 --2- 172.21.15.67:0/2659885748 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fadbc15b740 0x7fadbc1f4470 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fadb400cc00 tx=0x7fadb400a1f0 comp rx=0 tx=0).stop 2023-10-14T10:36:37.669 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.666+0000 7fadc0e14700 1 -- 172.21.15.67:0/2659885748 shutdown_connections 2023-10-14T10:36:37.669 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.666+0000 7fadc0e14700 1 --2- 172.21.15.67:0/2659885748 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fad9c06af00 0x7fad9c06d3c0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:37.669 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.666+0000 7fadc0e14700 1 --2- 172.21.15.67:0/2659885748 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fadbc15b740 0x7fadbc1f4470 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:37.670 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.666+0000 7fadc0e14700 1 --2- 172.21.15.67:0/2659885748 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fadbc0ce6e0 0x7fadbc0b6d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:37.670 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.666+0000 7fadc0e14700 1 --2- 172.21.15.67:0/2659885748 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fadbc0d0c70 0x7fadbc0cadd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:37.670 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.666+0000 7fadc0e14700 1 -- 172.21.15.67:0/2659885748 >> 172.21.15.67:0/2659885748 conn(0x7fadbc0665a0 msgr2=0x7fadbc0cdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:37.670 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.666+0000 7fadc0e14700 1 -- 172.21.15.67:0/2659885748 shutdown_connections 2023-10-14T10:36:37.670 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.666+0000 7fadc0e14700 1 -- 172.21.15.67:0/2659885748 wait complete. 2023-10-14T10:36:37.680 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-14T10:36:37.815 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.806+0000 7fc7dec56700 1 Processor -- start 2023-10-14T10:36:37.816 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.806+0000 7fc7dec56700 1 -- start start 2023-10-14T10:36:37.816 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.807+0000 7fc7dec56700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fc7d80d08a0 0x7fc7d80d4200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:37.816 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.807+0000 7fc7dec56700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fc7d80b7ab0 0x7fc7d80ce3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:37.816 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.807+0000 7fc7dec56700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7d80b6610 con 0x7fc7d8160e30 2023-10-14T10:36:37.816 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.807+0000 7fc7dec56700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7d80d1da0 con 0x7fc7d80d08a0 2023-10-14T10:36:37.817 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.807+0000 7fc7dec56700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d80d2080 con 0x7fc7d80b7ab0 2023-10-14T10:36:37.817 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.807+0000 7fc7dc9f2700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fc7d8160e30 0x7fc7d80d0190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:52780/0 (socket says 172.21.15.67:52780) 2023-10-14T10:36:37.817 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.807+0000 7fc7dc9f2700 1 -- 172.21.15.67:0/3923684044 learned_addr learned my addr 172.21.15.67:0/3923684044 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:36:37.817 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.807+0000 7fc7dd1f3700 1 --2- 172.21.15.67:0/3923684044 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fc7d80b7ab0 0x7fc7d80ce3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:37.817 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.807+0000 7fc7d7fff700 1 --2- 172.21.15.67:0/3923684044 >> v2:172.21.15.72:3300/0 conn(0x7fc7d80d08a0 0x7fc7d80d4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:37.818 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.807+0000 7fc7d77fe700 1 -- 172.21.15.67:0/3923684044 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4289753853 0 0) 0x7fc7d80b6610 con 0x7fc7d8160e30 2023-10-14T10:36:37.818 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.807+0000 7fc7d77fe700 1 -- 172.21.15.67:0/3923684044 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7c80035b0 con 0x7fc7d8160e30 2023-10-14T10:36:37.818 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.808+0000 7fc7d77fe700 1 -- 172.21.15.67:0/3923684044 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1705440475 0 0) 0x7fc7c80035b0 con 0x7fc7d8160e30 2023-10-14T10:36:37.818 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.808+0000 7fc7d77fe700 1 -- 172.21.15.67:0/3923684044 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fc7d80b7ab0 msgr2=0x7fc7d80ce3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:37.818 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.808+0000 7fc7d77fe700 1 --2- 172.21.15.67:0/3923684044 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fc7d80b7ab0 0x7fc7d80ce3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:37.819 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.808+0000 7fc7d77fe700 1 -- 172.21.15.67:0/3923684044 >> v2:172.21.15.72:3300/0 conn(0x7fc7d80d08a0 msgr2=0x7fc7d80d4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:37.819 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.808+0000 7fc7d77fe700 1 --2- 172.21.15.67:0/3923684044 >> v2:172.21.15.72:3300/0 conn(0x7fc7d80d08a0 0x7fc7d80d4200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:37.819 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.808+0000 7fc7d77fe700 1 -- 172.21.15.67:0/3923684044 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7d80d2360 con 0x7fc7d8160e30 2023-10-14T10:36:37.819 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.808+0000 7fc7d77fe700 1 -- 172.21.15.67:0/3923684044 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3180043755 0 0) 0x7fc7cc0017f0 con 0x7fc7d8160e30 2023-10-14T10:36:37.819 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.808+0000 7fc7d77fe700 1 -- 172.21.15.67:0/3923684044 >> v1:172.21.15.67:6789/0 conn(0x7fc7d8160e30 legacy=0x7fc7d80d0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:37.819 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.808+0000 7fc7d77fe700 1 --2- 172.21.15.67:0/3923684044 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fc7c8003a00 0x7fc7c8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:37.820 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.808+0000 7fc7d77fe700 1 --2- 172.21.15.67:0/3923684044 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc7c80043f0 0x7fc7c80048d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:37.820 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.808+0000 7fc7d77fe700 1 --2- 172.21.15.67:0/3923684044 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc7c8004fe0 0x7fc7c800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:37.820 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.808+0000 7fc7d77fe700 1 -- 172.21.15.67:0/3923684044 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d80d1da0 con 0x7fc7c8004fe0 2023-10-14T10:36:37.820 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.808+0000 7fc7d77fe700 1 -- 172.21.15.67:0/3923684044 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d80d2080 con 0x7fc7c80043f0 2023-10-14T10:36:37.820 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.808+0000 7fc7d77fe700 1 -- 172.21.15.67:0/3923684044 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c800bf80 con 0x7fc7c8003a00 2023-10-14T10:36:37.821 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.808+0000 7fc7dd1f3700 1 --2- 172.21.15.67:0/3923684044 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fc7d80b7ab0 0x7fc7d80ce3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-14T10:36:37.821 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.808+0000 7fc7dd1f3700 1 --2- 172.21.15.67:0/3923684044 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc7c8004fe0 0x7fc7c800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:37.821 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.808+0000 7fc7d7fff700 1 --2- 172.21.15.67:0/3923684044 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc7c80043f0 0x7fc7c80048d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:37.821 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.808+0000 7fc7dc9f2700 1 --2- 172.21.15.67:0/3923684044 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fc7c8003a00 0x7fc7c8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:37.821 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.808+0000 7fc7dd1f3700 1 -- 172.21.15.67:0/3923684044 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fc7c8003a00 msgr2=0x7fc7c8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:37.822 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.808+0000 7fc7dd1f3700 1 --2- 172.21.15.67:0/3923684044 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fc7c8003a00 0x7fc7c8003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:37.822 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.808+0000 7fc7dd1f3700 1 -- 172.21.15.67:0/3923684044 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc7c80043f0 msgr2=0x7fc7c80048d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:37.822 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.808+0000 7fc7dd1f3700 1 --2- 172.21.15.67:0/3923684044 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc7c80043f0 0x7fc7c80048d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:37.822 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.808+0000 7fc7dd1f3700 1 -- 172.21.15.67:0/3923684044 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7c80035b0 con 0x7fc7c8004fe0 2023-10-14T10:36:37.822 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.808+0000 7fc7dd1f3700 1 --2- 172.21.15.67:0/3923684044 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc7c8004fe0 0x7fc7c800ba10 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc7d80b7e90 tx=0x7fc7c8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5855a45e5bdf4006 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:37.823 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.809+0000 7fc7d77fe700 1 -- 172.21.15.67:0/3923684044 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7d0003c20 con 0x7fc7c8004fe0 2023-10-14T10:36:37.823 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.809+0000 7fc7d77fe700 1 -- 172.21.15.67:0/3923684044 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7d000f040 con 0x7fc7c8004fe0 2023-10-14T10:36:37.823 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.809+0000 7fc7d77fe700 1 -- 172.21.15.67:0/3923684044 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7d000e9d0 con 0x7fc7c8004fe0 2023-10-14T10:36:37.823 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.809+0000 7fc7dec56700 1 -- 172.21.15.67:0/3923684044 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc7c8004fe0 msgr2=0x7fc7c800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:37.823 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.809+0000 7fc7dec56700 1 --2- 172.21.15.67:0/3923684044 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc7c8004fe0 0x7fc7c800ba10 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc7d80b7e90 tx=0x7fc7c8005880 comp rx=0 tx=0).stop 2023-10-14T10:36:37.823 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.809+0000 7fc7dd1f3700 1 -- 172.21.15.67:0/3923684044 reap_dead start 2023-10-14T10:36:37.824 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.809+0000 7fc7dec56700 1 -- 172.21.15.67:0/3923684044 shutdown_connections 2023-10-14T10:36:37.824 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.809+0000 7fc7dec56700 1 -- 172.21.15.67:0/3923684044 >> 172.21.15.67:0/3923684044 conn(0x7fc7d809cb40 msgr2=0x7fc7d80d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:37.824 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.809+0000 7fc7dec56700 1 -- 172.21.15.67:0/3923684044 shutdown_connections 2023-10-14T10:36:37.824 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.809+0000 7fc7dec56700 1 -- 172.21.15.67:0/3923684044 wait complete. 2023-10-14T10:36:37.824 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.810+0000 7fc7dec56700 1 Processor -- start 2023-10-14T10:36:37.824 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.811+0000 7fc7dec56700 1 -- start start 2023-10-14T10:36:37.825 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.811+0000 7fc7dec56700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fc7d8155710 0x7fc7d815a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:37.825 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.811+0000 7fc7dec56700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc7d8155af0 0x7fc7d80d0f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:37.825 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.811+0000 7fc7dec56700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc7d815c680 0x7fc7d80d14c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:37.825 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.811+0000 7fc7dec56700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d80cb0a0 con 0x7fc7d815c680 2023-10-14T10:36:37.825 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.811+0000 7fc7dec56700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d815cbf0 con 0x7fc7d8155af0 2023-10-14T10:36:37.826 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.811+0000 7fc7dec56700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d815ced0 con 0x7fc7d8155710 2023-10-14T10:36:37.826 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.812+0000 7fc7dd1f3700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc7d815c680 0x7fc7d80d14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:37.826 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.812+0000 7fc7dd1f3700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc7d815c680 0x7fc7d80d14c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45028/0 (socket says 172.21.15.67:45028) 2023-10-14T10:36:37.826 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.812+0000 7fc7dd1f3700 1 -- 172.21.15.67:0/2605516905 learned_addr learned my addr 172.21.15.67:0/2605516905 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:37.827 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.812+0000 7fc7dc9f2700 1 --2- 172.21.15.67:0/2605516905 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fc7d8155710 0x7fc7d815a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:37.827 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.812+0000 7fc7d7fff700 1 --2- 172.21.15.67:0/2605516905 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc7d8155af0 0x7fc7d80d0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:37.827 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.812+0000 7fc7dd1f3700 1 -- 172.21.15.67:0/2605516905 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fc7d8155710 msgr2=0x7fc7d815a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:37.827 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.812+0000 7fc7dd1f3700 1 --2- 172.21.15.67:0/2605516905 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fc7d8155710 0x7fc7d815a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:37.827 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.812+0000 7fc7dd1f3700 1 -- 172.21.15.67:0/2605516905 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc7d8155af0 msgr2=0x7fc7d80d0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:37.828 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.812+0000 7fc7dd1f3700 1 --2- 172.21.15.67:0/2605516905 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc7d8155af0 0x7fc7d80d0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:37.828 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.812+0000 7fc7dd1f3700 1 -- 172.21.15.67:0/2605516905 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7d815d1b0 con 0x7fc7d815c680 2023-10-14T10:36:37.828 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.812+0000 7fc7dd1f3700 1 --2- 172.21.15.67:0/2605516905 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc7d815c680 0x7fc7d80d14c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc7d0015000 tx=0x7fc7d0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:37.828 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.812+0000 7fc7d57fa700 1 -- 172.21.15.67:0/2605516905 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7d0003e30 con 0x7fc7d815c680 2023-10-14T10:36:37.828 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.812+0000 7fc7d57fa700 1 -- 172.21.15.67:0/2605516905 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7d0009410 con 0x7fc7d815c680 2023-10-14T10:36:37.829 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.812+0000 7fc7dec56700 1 -- 172.21.15.67:0/2605516905 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7d80cad30 con 0x7fc7d815c680 2023-10-14T10:36:37.829 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.812+0000 7fc7d57fa700 1 -- 172.21.15.67:0/2605516905 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7d0014a00 con 0x7fc7d815c680 2023-10-14T10:36:37.829 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.812+0000 7fc7dec56700 1 -- 172.21.15.67:0/2605516905 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7d80c9280 con 0x7fc7d815c680 2023-10-14T10:36:37.829 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.813+0000 7fc7dec56700 1 -- 172.21.15.67:0/2605516905 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7d80cb210 con 0x7fc7d815c680 2023-10-14T10:36:37.830 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.814+0000 7fc7d57fa700 1 -- 172.21.15.67:0/2605516905 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fc7d0024070 con 0x7fc7d815c680 2023-10-14T10:36:37.831 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.814+0000 7fc7d57fa700 1 --2- 172.21.15.67:0/2605516905 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fc7bc06af00 0x7fc7bc06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:37.831 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.814+0000 7fc7d57fa700 1 -- 172.21.15.67:0/2605516905 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc7d008adb0 con 0x7fc7d815c680 2023-10-14T10:36:37.831 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.814+0000 7fc7dc9f2700 1 --2- 172.21.15.67:0/2605516905 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fc7bc06af00 0x7fc7bc06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:37.831 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.815+0000 7fc7dc9f2700 1 --2- 172.21.15.67:0/2605516905 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fc7bc06af00 0x7fc7bc06d3c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc000c00 tx=0x7fc7cc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:37.831 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:37.819+0000 7fc7d57fa700 1 -- 172.21.15.67:0/2605516905 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc7d0059a40 con 0x7fc7d815c680 2023-10-14T10:36:38.020 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.017+0000 7fc7dec56700 1 -- 172.21.15.67:0/2605516905 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc7d8067d20 con 0x7fc7d815c680 2023-10-14T10:36:38.021 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.018+0000 7fc7d57fa700 1 -- 172.21.15.67:0/2605516905 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc7d00595d0 con 0x7fc7d815c680 2023-10-14T10:36:38.021 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2023-10-14T10:36:38.022 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.020+0000 7fc7dec56700 1 -- 172.21.15.67:0/2605516905 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fc7bc06af00 msgr2=0x7fc7bc06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:38.022 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.020+0000 7fc7dec56700 1 --2- 172.21.15.67:0/2605516905 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fc7bc06af00 0x7fc7bc06d3c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc000c00 tx=0x7fc7cc009040 comp rx=0 tx=0).stop 2023-10-14T10:36:38.022 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.020+0000 7fc7dec56700 1 -- 172.21.15.67:0/2605516905 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc7d815c680 msgr2=0x7fc7d80d14c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:38.023 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.020+0000 7fc7dec56700 1 --2- 172.21.15.67:0/2605516905 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc7d815c680 0x7fc7d80d14c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc7d0015000 tx=0x7fc7d0009750 comp rx=0 tx=0).stop 2023-10-14T10:36:38.023 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.020+0000 7fc7dec56700 1 -- 172.21.15.67:0/2605516905 shutdown_connections 2023-10-14T10:36:38.023 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.020+0000 7fc7dec56700 1 --2- 172.21.15.67:0/2605516905 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fc7bc06af00 0x7fc7bc06d3c0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:38.023 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.020+0000 7fc7dec56700 1 --2- 172.21.15.67:0/2605516905 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc7d815c680 0x7fc7d80d14c0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:38.023 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.020+0000 7fc7dec56700 1 --2- 172.21.15.67:0/2605516905 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fc7d8155710 0x7fc7d815a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:38.024 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.020+0000 7fc7dec56700 1 --2- 172.21.15.67:0/2605516905 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc7d8155af0 0x7fc7d80d0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:38.024 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.020+0000 7fc7dec56700 1 -- 172.21.15.67:0/2605516905 >> 172.21.15.67:0/2605516905 conn(0x7fc7d809cb40 msgr2=0x7fc7d80b8180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:38.024 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.020+0000 7fc7dec56700 1 -- 172.21.15.67:0/2605516905 shutdown_connections 2023-10-14T10:36:38.024 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.020+0000 7fc7dec56700 1 -- 172.21.15.67:0/2605516905 wait complete. 2023-10-14T10:36:38.034 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-14T10:36:38.161 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.159+0000 7fca8d40a700 1 Processor -- start 2023-10-14T10:36:38.161 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.159+0000 7fca8d40a700 1 -- start start 2023-10-14T10:36:38.164 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.160+0000 7fca8d40a700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fca880cc3f0 0x7fca880cc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:38.164 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.160+0000 7fca8d40a700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fca8806af50 0x7fca88071f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:38.164 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.160+0000 7fca8d40a700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca88071ab0 con 0x7fca88161670 2023-10-14T10:36:38.164 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.160+0000 7fca8d40a700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca88161bc0 con 0x7fca880cc3f0 2023-10-14T10:36:38.165 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.160+0000 7fca8d40a700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca88161ea0 con 0x7fca8806af50 2023-10-14T10:36:38.165 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.160+0000 7fca86ffd700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fca880cc3f0 0x7fca880cc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:38.165 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.160+0000 7fca867fc700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fca88161670 0x7fca8806a840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:52802/0 (socket says 172.21.15.67:52802) 2023-10-14T10:36:38.165 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.160+0000 7fca867fc700 1 -- 172.21.15.67:0/2059875201 learned_addr learned my addr 172.21.15.67:0/2059875201 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:36:38.165 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.160+0000 7fca877fe700 1 --2- 172.21.15.67:0/2059875201 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fca8806af50 0x7fca88071f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:38.165 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.160+0000 7fca86ffd700 1 -- 172.21.15.67:0/2059875201 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fca8806af50 msgr2=0x7fca88071f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:38.166 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.160+0000 7fca86ffd700 1 --2- 172.21.15.67:0/2059875201 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fca8806af50 0x7fca88071f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:38.166 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.160+0000 7fca86ffd700 1 -- 172.21.15.67:0/2059875201 >> v1:172.21.15.67:6789/0 conn(0x7fca88161670 legacy=0x7fca8806a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:38.166 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.160+0000 7fca86ffd700 1 -- 172.21.15.67:0/2059875201 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca88162180 con 0x7fca880cc3f0 2023-10-14T10:36:38.166 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.160+0000 7fca85ffb700 1 -- 172.21.15.67:0/2059875201 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2788428260 0 0) 0x7fca88071ab0 con 0x7fca88161670 2023-10-14T10:36:38.167 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.160+0000 7fca86ffd700 1 --2- 172.21.15.67:0/2059875201 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fca880cc3f0 0x7fca880cc7d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fca78009840 tx=0x7fca78009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0b97b45ffb63d4b server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:38.167 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.160+0000 7fca85ffb700 1 -- 172.21.15.67:0/2059875201 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca78011830 con 0x7fca880cc3f0 2023-10-14T10:36:38.167 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.161+0000 7fca85ffb700 1 -- 172.21.15.67:0/2059875201 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca78011990 con 0x7fca880cc3f0 2023-10-14T10:36:38.167 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.161+0000 7fca85ffb700 1 -- 172.21.15.67:0/2059875201 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca78011d20 con 0x7fca880cc3f0 2023-10-14T10:36:38.167 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.161+0000 7fca8d40a700 1 -- 172.21.15.67:0/2059875201 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fca880cc3f0 msgr2=0x7fca880cc7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:38.167 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.161+0000 7fca8d40a700 1 --2- 172.21.15.67:0/2059875201 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fca880cc3f0 0x7fca880cc7d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fca78009840 tx=0x7fca78009b50 comp rx=0 tx=0).stop 2023-10-14T10:36:38.168 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.161+0000 7fca8d40a700 1 -- 172.21.15.67:0/2059875201 shutdown_connections 2023-10-14T10:36:38.168 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.161+0000 7fca8d40a700 1 --2- 172.21.15.67:0/2059875201 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fca8806af50 0x7fca88071f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:38.168 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.161+0000 7fca8d40a700 1 --2- 172.21.15.67:0/2059875201 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fca880cc3f0 0x7fca880cc7d0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:38.168 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.161+0000 7fca8d40a700 1 -- 172.21.15.67:0/2059875201 >> 172.21.15.67:0/2059875201 conn(0x7fca8806dac0 msgr2=0x7fca8806dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:38.168 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.161+0000 7fca8d40a700 1 -- 172.21.15.67:0/2059875201 shutdown_connections 2023-10-14T10:36:38.169 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.161+0000 7fca8d40a700 1 -- 172.21.15.67:0/2059875201 wait complete. 2023-10-14T10:36:38.171 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.162+0000 7fca8d40a700 1 Processor -- start 2023-10-14T10:36:38.172 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.163+0000 7fca8d40a700 1 -- start start 2023-10-14T10:36:38.172 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.163+0000 7fca8d40a700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fca8806af50 0x7fca8814aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:38.172 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.163+0000 7fca8d40a700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fca880cc3f0 0x7fca880c7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:38.172 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.163+0000 7fca8d40a700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fca88161670 0x7fca880b0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:38.172 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.163+0000 7fca8d40a700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca881450b0 con 0x7fca880cc3f0 2023-10-14T10:36:38.172 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.163+0000 7fca8d40a700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca8814afa0 con 0x7fca88161670 2023-10-14T10:36:38.173 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.163+0000 7fca8d40a700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca880c9d70 con 0x7fca8806af50 2023-10-14T10:36:38.173 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.163+0000 7fca867fc700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fca880cc3f0 0x7fca880c7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:38.173 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.163+0000 7fca867fc700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fca880cc3f0 0x7fca880c7580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45040/0 (socket says 172.21.15.67:45040) 2023-10-14T10:36:38.174 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.163+0000 7fca867fc700 1 -- 172.21.15.67:0/1334672453 learned_addr learned my addr 172.21.15.67:0/1334672453 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:38.174 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.164+0000 7fca86ffd700 1 --2- 172.21.15.67:0/1334672453 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fca8806af50 0x7fca8814aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:38.174 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.164+0000 7fca877fe700 1 --2- 172.21.15.67:0/1334672453 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fca88161670 0x7fca880b0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:38.174 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.164+0000 7fca867fc700 1 -- 172.21.15.67:0/1334672453 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fca8806af50 msgr2=0x7fca8814aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:38.175 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.164+0000 7fca867fc700 1 --2- 172.21.15.67:0/1334672453 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fca8806af50 0x7fca8814aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:38.175 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.164+0000 7fca867fc700 1 -- 172.21.15.67:0/1334672453 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fca88161670 msgr2=0x7fca880b0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:38.175 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.164+0000 7fca867fc700 1 --2- 172.21.15.67:0/1334672453 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fca88161670 0x7fca880b0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:38.175 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.164+0000 7fca867fc700 1 -- 172.21.15.67:0/1334672453 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca780094a0 con 0x7fca880cc3f0 2023-10-14T10:36:38.175 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.164+0000 7fca86ffd700 1 --2- 172.21.15.67:0/1334672453 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fca8806af50 0x7fca8814aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-14T10:36:38.176 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.164+0000 7fca867fc700 1 --2- 172.21.15.67:0/1334672453 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fca880cc3f0 0x7fca880c7580 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fca7c00c550 tx=0x7fca7c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:38.176 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.164+0000 7fca6f7fe700 1 -- 172.21.15.67:0/1334672453 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca7c004c80 con 0x7fca880cc3f0 2023-10-14T10:36:38.176 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.164+0000 7fca6f7fe700 1 -- 172.21.15.67:0/1334672453 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca7c004e00 con 0x7fca880cc3f0 2023-10-14T10:36:38.176 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.164+0000 7fca8d40a700 1 -- 172.21.15.67:0/1334672453 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca880c9fc0 con 0x7fca880cc3f0 2023-10-14T10:36:38.176 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.164+0000 7fca6f7fe700 1 -- 172.21.15.67:0/1334672453 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca7c007310 con 0x7fca880cc3f0 2023-10-14T10:36:38.177 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.164+0000 7fca8d40a700 1 -- 172.21.15.67:0/1334672453 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca880ca580 con 0x7fca880cc3f0 2023-10-14T10:36:38.177 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.165+0000 7fca8d40a700 1 -- 172.21.15.67:0/1334672453 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca88067d20 con 0x7fca880cc3f0 2023-10-14T10:36:38.177 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.165+0000 7fca6f7fe700 1 -- 172.21.15.67:0/1334672453 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fca7c007470 con 0x7fca880cc3f0 2023-10-14T10:36:38.177 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.166+0000 7fca6f7fe700 1 --2- 172.21.15.67:0/1334672453 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fca7406af00 0x7fca7406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:38.177 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.166+0000 7fca6f7fe700 1 -- 172.21.15.67:0/1334672453 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fca7c089260 con 0x7fca880cc3f0 2023-10-14T10:36:38.177 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.166+0000 7fca86ffd700 1 --2- 172.21.15.67:0/1334672453 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fca7406af00 0x7fca7406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:38.178 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.167+0000 7fca86ffd700 1 --2- 172.21.15.67:0/1334672453 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fca7406af00 0x7fca7406d3c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fca780090b0 tx=0x7fca78008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:38.178 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.171+0000 7fca6f7fe700 1 -- 172.21.15.67:0/1334672453 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fca7c0543a0 con 0x7fca880cc3f0 2023-10-14T10:36:38.390 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.388+0000 7fca8d40a700 1 -- 172.21.15.67:0/1334672453 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fca880c6410 con 0x7fca880cc3f0 2023-10-14T10:36:38.391 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.388+0000 7fca6f7fe700 1 -- 172.21.15.67:0/1334672453 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7fca7c0579c0 con 0x7fca880cc3f0 2023-10-14T10:36:38.393 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.391+0000 7fca8d40a700 1 -- 172.21.15.67:0/1334672453 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fca7406af00 msgr2=0x7fca7406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:38.393 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.391+0000 7fca8d40a700 1 --2- 172.21.15.67:0/1334672453 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fca7406af00 0x7fca7406d3c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fca780090b0 tx=0x7fca78008030 comp rx=0 tx=0).stop 2023-10-14T10:36:38.394 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.391+0000 7fca8d40a700 1 -- 172.21.15.67:0/1334672453 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fca880cc3f0 msgr2=0x7fca880c7580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:38.394 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.391+0000 7fca8d40a700 1 --2- 172.21.15.67:0/1334672453 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fca880cc3f0 0x7fca880c7580 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fca7c00c550 tx=0x7fca7c00c910 comp rx=0 tx=0).stop 2023-10-14T10:36:38.394 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.391+0000 7fca8d40a700 1 -- 172.21.15.67:0/1334672453 shutdown_connections 2023-10-14T10:36:38.394 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.391+0000 7fca8d40a700 1 --2- 172.21.15.67:0/1334672453 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fca7406af00 0x7fca7406d3c0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:38.394 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.391+0000 7fca8d40a700 1 --2- 172.21.15.67:0/1334672453 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fca88161670 0x7fca880b0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:38.395 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.391+0000 7fca8d40a700 1 --2- 172.21.15.67:0/1334672453 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fca8806af50 0x7fca8814aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:38.395 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.391+0000 7fca8d40a700 1 --2- 172.21.15.67:0/1334672453 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fca880cc3f0 0x7fca880c7580 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:38.395 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.391+0000 7fca8d40a700 1 -- 172.21.15.67:0/1334672453 >> 172.21.15.67:0/1334672453 conn(0x7fca8806dac0 msgr2=0x7fca8815f1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:38.395 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.391+0000 7fca8d40a700 1 -- 172.21.15.67:0/1334672453 shutdown_connections 2023-10-14T10:36:38.396 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.391+0000 7fca8d40a700 1 -- 172.21.15.67:0/1334672453 wait complete. 2023-10-14T10:36:38.396 INFO:teuthology.orchestra.run.smithi067.stderr:listed 0 entries 2023-10-14T10:36:38.405 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-14T10:36:38.406 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:36:38.406 DEBUG:teuthology.orchestra.run.smithi067:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2023-10-14T10:36:38.465 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi067.front.sepia.ceph.com 2023-10-14T10:36:38.465 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:36:38.465 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-14T10:36:38.486 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-10-14T10:36:38.487 DEBUG:teuthology.orchestra.run.smithi072:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-14T10:36:38.509 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-10-14T10:36:38.509 DEBUG:teuthology.orchestra.run.smithi072:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-14T10:36:38.569 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-10-14T10:36:38.569 DEBUG:teuthology.orchestra.run.smithi072:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-14T10:36:38.631 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-14T10:36:38.758 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.756+0000 7f921140b700 1 Processor -- start 2023-10-14T10:36:38.758 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.756+0000 7f921140b700 1 -- start start 2023-10-14T10:36:38.762 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.756+0000 7f921140b700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f920c15b740 0x7f920c0d1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:38.762 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.756+0000 7f921140b700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f920c0cf190 con 0x7f920c0d0c70 2023-10-14T10:36:38.763 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.756+0000 7f921140b700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f920c0cbd40 con 0x7f920c0ce6e0 2023-10-14T10:36:38.763 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.756+0000 7f921140b700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f920c15bb20 con 0x7f920c15b740 2023-10-14T10:36:38.763 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.757+0000 7f920a7fc700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f920c0d0c70 0x7f920c0cdfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:52814/0 (socket says 172.21.15.67:52814) 2023-10-14T10:36:38.763 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.757+0000 7f920a7fc700 1 -- 172.21.15.67:0/268888760 learned_addr learned my addr 172.21.15.67:0/268888760 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:36:38.763 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.757+0000 7f920affd700 1 --2- 172.21.15.67:0/268888760 >> v2:172.21.15.72:3300/0 conn(0x7f920c15b740 0x7f920c0d1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:38.764 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.757+0000 7f9209ffb700 1 -- 172.21.15.67:0/268888760 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2331041410 0 0) 0x7f920c0cf190 con 0x7f920c0d0c70 2023-10-14T10:36:38.764 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.757+0000 7f9209ffb700 1 -- 172.21.15.67:0/268888760 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91f80035b0 con 0x7f920c0d0c70 2023-10-14T10:36:38.764 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.757+0000 7f9209ffb700 1 -- 172.21.15.67:0/268888760 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1918876688 0 0) 0x7f91f80035b0 con 0x7f920c0d0c70 2023-10-14T10:36:38.764 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.757+0000 7f9209ffb700 1 -- 172.21.15.67:0/268888760 >> v2:172.21.15.72:3300/0 conn(0x7f920c15b740 msgr2=0x7f920c0d1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:38.764 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.757+0000 7f9209ffb700 1 --2- 172.21.15.67:0/268888760 >> v2:172.21.15.72:3300/0 conn(0x7f920c15b740 0x7f920c0d1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:38.765 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.757+0000 7f9209ffb700 1 -- 172.21.15.67:0/268888760 >> v1:172.21.15.72:6789/0 conn(0x7f920c0ce6e0 legacy=0x7f920c0cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:38.765 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.757+0000 7f9209ffb700 1 -- 172.21.15.67:0/268888760 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f920c0cc020 con 0x7f920c0d0c70 2023-10-14T10:36:38.765 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.757+0000 7f9209ffb700 1 -- 172.21.15.67:0/268888760 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3180043755 0 0) 0x7f91f4001670 con 0x7f920c0d0c70 2023-10-14T10:36:38.765 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.757+0000 7f920affd700 1 --2- 172.21.15.67:0/268888760 >> v2:172.21.15.72:3300/0 conn(0x7f920c15b740 0x7f920c0d1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-14T10:36:38.765 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.757+0000 7f9209ffb700 1 -- 172.21.15.67:0/268888760 >> v1:172.21.15.67:6789/0 conn(0x7f920c0d0c70 legacy=0x7f920c0cdfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:38.766 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.757+0000 7f9209ffb700 1 --2- 172.21.15.67:0/268888760 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f91f8003a00 0x7f91f8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:38.766 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.757+0000 7f9209ffb700 1 --2- 172.21.15.67:0/268888760 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f91f8004850 0x7f91f8004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:38.766 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.757+0000 7f9209ffb700 1 --2- 172.21.15.67:0/268888760 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f91f8005210 0x7f91f800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:38.766 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.757+0000 7f9209ffb700 1 -- 172.21.15.67:0/268888760 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f920c15bb20 con 0x7f91f8004850 2023-10-14T10:36:38.766 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.757+0000 7f9209ffb700 1 -- 172.21.15.67:0/268888760 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91f800c1b0 con 0x7f91f8005210 2023-10-14T10:36:38.767 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.757+0000 7f9209ffb700 1 -- 172.21.15.67:0/268888760 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91f800c370 con 0x7f91f8003a00 2023-10-14T10:36:38.767 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.757+0000 7f920a7fc700 1 --2- 172.21.15.67:0/268888760 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f91f8004850 0x7f91f8004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:38.767 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.757+0000 7f920affd700 1 --2- 172.21.15.67:0/268888760 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f91f8003a00 0x7f91f8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:38.767 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.757+0000 7f920b7fe700 1 --2- 172.21.15.67:0/268888760 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f91f8005210 0x7f91f800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:38.767 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.758+0000 7f920a7fc700 1 -- 172.21.15.67:0/268888760 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f91f8003a00 msgr2=0x7f91f8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:38.767 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.758+0000 7f920a7fc700 1 --2- 172.21.15.67:0/268888760 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f91f8003a00 0x7f91f8003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:38.768 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.758+0000 7f920a7fc700 1 -- 172.21.15.67:0/268888760 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f91f8005210 msgr2=0x7f91f800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:38.768 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.758+0000 7f920a7fc700 1 --2- 172.21.15.67:0/268888760 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f91f8005210 0x7f91f800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:38.768 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.758+0000 7f920a7fc700 1 -- 172.21.15.67:0/268888760 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91f80035b0 con 0x7f91f8004850 2023-10-14T10:36:38.768 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.758+0000 7f920b7fe700 1 -- 172.21.15.67:0/268888760 reap_dead start 2023-10-14T10:36:38.769 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.758+0000 7f920a7fc700 1 --2- 172.21.15.67:0/268888760 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f91f8004850 0x7f91f8004cd0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f91f400c240 tx=0x7f91f8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b61fa21ed520128 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:38.769 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.758+0000 7f9209ffb700 1 -- 172.21.15.67:0/268888760 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f4007c10 con 0x7f91f8004850 2023-10-14T10:36:38.769 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.758+0000 7f9209ffb700 1 -- 172.21.15.67:0/268888760 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91f400ce00 con 0x7f91f8004850 2023-10-14T10:36:38.769 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.758+0000 7f9209ffb700 1 -- 172.21.15.67:0/268888760 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f4005ba0 con 0x7f91f8004850 2023-10-14T10:36:38.769 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.758+0000 7f921140b700 1 -- 172.21.15.67:0/268888760 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f91f8004850 msgr2=0x7f91f8004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:38.769 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.758+0000 7f921140b700 1 --2- 172.21.15.67:0/268888760 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f91f8004850 0x7f91f8004cd0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f91f400c240 tx=0x7f91f8005880 comp rx=0 tx=0).stop 2023-10-14T10:36:38.770 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.759+0000 7f921140b700 1 -- 172.21.15.67:0/268888760 shutdown_connections 2023-10-14T10:36:38.770 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.759+0000 7f921140b700 1 --2- 172.21.15.67:0/268888760 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f91f8004850 0x7f91f8004cd0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:38.770 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.759+0000 7f921140b700 1 -- 172.21.15.67:0/268888760 >> 172.21.15.67:0/268888760 conn(0x7f920c0665a0 msgr2=0x7f920c066980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:38.770 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.759+0000 7f921140b700 1 -- 172.21.15.67:0/268888760 shutdown_connections 2023-10-14T10:36:38.771 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.759+0000 7f921140b700 1 -- 172.21.15.67:0/268888760 wait complete. 2023-10-14T10:36:38.771 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.760+0000 7f921140b700 1 Processor -- start 2023-10-14T10:36:38.771 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.760+0000 7f921140b700 1 -- start start 2023-10-14T10:36:38.774 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.761+0000 7f921140b700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f91f8004850 0x7f920c0b0080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:38.774 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.761+0000 7f921140b700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f920c1f0ef0 0x7f920c15bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:38.774 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.761+0000 7f921140b700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f920c15e200 0x7f920c15e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:38.774 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.761+0000 7f921140b700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f920c15ebd0 con 0x7f920c1f0ef0 2023-10-14T10:36:38.775 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.761+0000 7f921140b700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f920c15d6a0 con 0x7f91f8004850 2023-10-14T10:36:38.775 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.761+0000 7f921140b700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f920c15d210 con 0x7f920c15e200 2023-10-14T10:36:38.775 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.761+0000 7f920a7fc700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f920c1f0ef0 0x7f920c15bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:38.775 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.761+0000 7f920a7fc700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f920c1f0ef0 0x7f920c15bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45060/0 (socket says 172.21.15.67:45060) 2023-10-14T10:36:38.775 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.761+0000 7f920a7fc700 1 -- 172.21.15.67:0/3437672566 learned_addr learned my addr 172.21.15.67:0/3437672566 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:38.776 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.761+0000 7f920b7fe700 1 --2- 172.21.15.67:0/3437672566 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f920c15e200 0x7f920c15e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:38.776 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.761+0000 7f920affd700 1 --2- 172.21.15.67:0/3437672566 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f91f8004850 0x7f920c0b0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:38.776 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.761+0000 7f920a7fc700 1 -- 172.21.15.67:0/3437672566 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f920c15e200 msgr2=0x7f920c15e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:38.776 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.761+0000 7f920a7fc700 1 --2- 172.21.15.67:0/3437672566 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f920c15e200 0x7f920c15e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:38.776 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.761+0000 7f920a7fc700 1 -- 172.21.15.67:0/3437672566 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f91f8004850 msgr2=0x7f920c0b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:38.777 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.761+0000 7f920a7fc700 1 --2- 172.21.15.67:0/3437672566 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f91f8004850 0x7f920c0b0080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:38.777 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.761+0000 7f920a7fc700 1 -- 172.21.15.67:0/3437672566 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f920c15c600 con 0x7f920c1f0ef0 2023-10-14T10:36:38.777 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.761+0000 7f920a7fc700 1 --2- 172.21.15.67:0/3437672566 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f920c1f0ef0 0x7f920c15bc90 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f91f800c180 tx=0x7f91f4001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:38.777 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.761+0000 7f92037fe700 1 -- 172.21.15.67:0/3437672566 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f4001670 con 0x7f920c1f0ef0 2023-10-14T10:36:38.777 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.762+0000 7f92037fe700 1 -- 172.21.15.67:0/3437672566 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91f40070d0 con 0x7f920c1f0ef0 2023-10-14T10:36:38.778 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.762+0000 7f921140b700 1 -- 172.21.15.67:0/3437672566 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f920c15c8e0 con 0x7f920c1f0ef0 2023-10-14T10:36:38.778 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.762+0000 7f92037fe700 1 -- 172.21.15.67:0/3437672566 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f4007460 con 0x7f920c1f0ef0 2023-10-14T10:36:38.778 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.762+0000 7f921140b700 1 -- 172.21.15.67:0/3437672566 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f920c15cea0 con 0x7f920c1f0ef0 2023-10-14T10:36:38.778 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.762+0000 7f921140b700 1 -- 172.21.15.67:0/3437672566 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f920c15d030 con 0x7f920c1f0ef0 2023-10-14T10:36:38.779 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.763+0000 7f92037fe700 1 -- 172.21.15.67:0/3437672566 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f91f40075c0 con 0x7f920c1f0ef0 2023-10-14T10:36:38.779 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.763+0000 7f92037fe700 1 --2- 172.21.15.67:0/3437672566 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f91ec06af00 0x7f91ec06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:38.779 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.764+0000 7f92037fe700 1 -- 172.21.15.67:0/3437672566 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f91f408a4c0 con 0x7f920c1f0ef0 2023-10-14T10:36:38.779 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.764+0000 7f920affd700 1 --2- 172.21.15.67:0/3437672566 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f91ec06af00 0x7f91ec06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:38.779 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.764+0000 7f920affd700 1 --2- 172.21.15.67:0/3437672566 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f91ec06af00 0x7f91ec06d3c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f91fc009e60 tx=0x7f91fc009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:38.779 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.768+0000 7f92037fe700 1 -- 172.21.15.67:0/3437672566 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f91f4055600 con 0x7f920c1f0ef0 2023-10-14T10:36:38.969 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.967+0000 7f921140b700 1 -- 172.21.15.67:0/3437672566 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f920c15d350 con 0x7f920c1f0ef0 2023-10-14T10:36:38.972 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.970+0000 7f92037fe700 1 -- 172.21.15.67:0/3437672566 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16457 (secure 0 0 0) 0x7f91f4058c20 con 0x7f920c1f0ef0 2023-10-14T10:36:38.972 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-14T10:36:38.972 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":22,"fsid":"37f9ab46-3082-4b71-8b24-08d39897b008","created":"2023-10-14T10:35:46.067554+0000","modified":"2023-10-14T10:36:04.837296+0000","last_up_change":"2023-10-14T10:35:56.668142+0000","last_in_change":"2023-10-14T10:35:48.551490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-14T10:35:57.809566+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-14T10:36:01.667146+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-14T10:36:02.219700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2b59a07e-11ef-440c-b766-9f64ac885838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6801","nonce":1287255499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6803","nonce":1287255499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6807","nonce":1287255499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6805","nonce":1287255499}]},"public_addr":"172.21.15.67:6801/1287255499","cluster_addr":"172.21.15.67:6803/1287255499","heartbeat_back_addr":"172.21.15.67:6807/1287255499","heartbeat_front_addr":"172.21.15.67:6805/1287255499","state":["exists","up"]},{"osd":1,"uuid":"6f813bd6-fa0b-487d-a204-431eb1276a60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6818","nonce":2435538787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6822","nonce":2435538787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6830","nonce":2435538787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6826","nonce":2435538787}]},"public_addr":"172.21.15.67:6818/2435538787","cluster_addr":"172.21.15.67:6822/2435538787","heartbeat_back_addr":"172.21.15.67:6830/2435538787","heartbeat_front_addr":"172.21.15.67:6826/2435538787","state":["exists","up"]},{"osd":2,"uuid":"6af42c6a-1b59-4669-9c04-b6876788446d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6809","nonce":2827491791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6811","nonce":2827491791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6815","nonce":2827491791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6813","nonce":2827491791}]},"public_addr":"172.21.15.67:6809/2827491791","cluster_addr":"172.21.15.67:6811/2827491791","heartbeat_back_addr":"172.21.15.67:6815/2827491791","heartbeat_front_addr":"172.21.15.67:6813/2827491791","state":["exists","up"]},{"osd":3,"uuid":"1d889d76-a646-499d-bfff-ed5bae5434ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6819","nonce":1848001176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6821","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6823","nonce":1848001176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6831","nonce":1848001176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6825","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6827","nonce":1848001176}]},"public_addr":"172.21.15.67:6819/1848001176","cluster_addr":"172.21.15.67:6823/1848001176","heartbeat_back_addr":"172.21.15.67:6831/1848001176","heartbeat_front_addr":"172.21.15.67:6827/1848001176","state":["exists","up"]},{"osd":4,"uuid":"c3271dff-b437-4fdd-b13b-6e49a35473af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6804","nonce":1891230785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6808","nonce":1891230785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6815","nonce":1891230785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6811","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6813","nonce":1891230785}]},"public_addr":"172.21.15.72:6804/1891230785","cluster_addr":"172.21.15.72:6808/1891230785","heartbeat_back_addr":"172.21.15.72:6815/1891230785","heartbeat_front_addr":"172.21.15.72:6813/1891230785","state":["exists","up"]},{"osd":5,"uuid":"0337bb18-4b68-4eff-b528-a667275e6926","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6801","nonce":569686991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6803","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6805","nonce":569686991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6812","nonce":569686991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6807","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6809","nonce":569686991}]},"public_addr":"172.21.15.72:6801/569686991","cluster_addr":"172.21.15.72:6805/569686991","heartbeat_back_addr":"172.21.15.72:6812/569686991","heartbeat_front_addr":"172.21.15.72:6809/569686991","state":["exists","up"]},{"osd":6,"uuid":"8e92d9cf-0c8c-46eb-89a7-db2023ef1509","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6817","nonce":3767685934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6819","nonce":3767685934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6823","nonce":3767685934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6821","nonce":3767685934}]},"public_addr":"172.21.15.72:6817/3767685934","cluster_addr":"172.21.15.72:6819/3767685934","heartbeat_back_addr":"172.21.15.72:6823/3767685934","heartbeat_front_addr":"172.21.15.72:6821/3767685934","state":["exists","up"]},{"osd":7,"uuid":"3fd6aa7d-50a9-45c9-9561-5902a044a119","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6825","nonce":3567465926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6827","nonce":3567465926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6831","nonce":3567465926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6829","nonce":3567465926}]},"public_addr":"172.21.15.72:6825/3567465926","cluster_addr":"172.21.15.72:6827/3567465926","heartbeat_back_addr":"172.21.15.72:6831/3567465926","heartbeat_front_addr":"172.21.15.72:6829/3567465926","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.511466+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.795107+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.858674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.728895+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.794842+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.750064+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.732122+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.765185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-14T10:36:38.974 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.972+0000 7f921140b700 1 -- 172.21.15.67:0/3437672566 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f91ec06af00 msgr2=0x7f91ec06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:38.975 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.972+0000 7f921140b700 1 --2- 172.21.15.67:0/3437672566 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f91ec06af00 0x7f91ec06d3c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f91fc009e60 tx=0x7f91fc009210 comp rx=0 tx=0).stop 2023-10-14T10:36:38.975 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.972+0000 7f921140b700 1 -- 172.21.15.67:0/3437672566 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f920c1f0ef0 msgr2=0x7f920c15bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:38.975 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.972+0000 7f921140b700 1 --2- 172.21.15.67:0/3437672566 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f920c1f0ef0 0x7f920c15bc90 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f91f800c180 tx=0x7f91f4001a00 comp rx=0 tx=0).stop 2023-10-14T10:36:38.975 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.972+0000 7f921140b700 1 -- 172.21.15.67:0/3437672566 shutdown_connections 2023-10-14T10:36:38.976 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.972+0000 7f921140b700 1 --2- 172.21.15.67:0/3437672566 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f91ec06af00 0x7f91ec06d3c0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:38.976 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.972+0000 7f921140b700 1 --2- 172.21.15.67:0/3437672566 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f920c15e200 0x7f920c15e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:38.976 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.972+0000 7f921140b700 1 --2- 172.21.15.67:0/3437672566 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f91f8004850 0x7f920c0b0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:38.976 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.972+0000 7f921140b700 1 --2- 172.21.15.67:0/3437672566 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f920c1f0ef0 0x7f920c15bc90 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:38.977 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.972+0000 7f921140b700 1 -- 172.21.15.67:0/3437672566 >> 172.21.15.67:0/3437672566 conn(0x7f920c0665a0 msgr2=0x7f920c0c6be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:38.977 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.972+0000 7f921140b700 1 -- 172.21.15.67:0/3437672566 shutdown_connections 2023-10-14T10:36:38.977 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:38.972+0000 7f921140b700 1 -- 172.21.15.67:0/3437672566 wait complete. 2023-10-14T10:36:38.987 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-14T10:35:57.809566+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-14T10:36:01.667146+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-14T10:36:02.219700+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-14T10:36:38.987 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-14T10:36:39.115 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.113+0000 7f88b8a92700 1 Processor -- start 2023-10-14T10:36:39.116 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.113+0000 7f88b8a92700 1 -- start start 2023-10-14T10:36:39.120 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.114+0000 7f88b8a92700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f88b415f1f0 0x7f88b40cf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:39.120 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.114+0000 7f88b8a92700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f88b40cfc60 0x7f88b40cbf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:39.121 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.114+0000 7f88b8a92700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88b415e7a0 con 0x7f88b415ab10 2023-10-14T10:36:39.121 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.114+0000 7f88b8a92700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88b40d00d0 con 0x7f88b415f1f0 2023-10-14T10:36:39.121 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.114+0000 7f88b8a92700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88b40b29a0 con 0x7f88b40cfc60 2023-10-14T10:36:39.121 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.115+0000 7f88b259c700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f88b415ab10 0x7f88b40b1f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:52834/0 (socket says 172.21.15.67:52834) 2023-10-14T10:36:39.121 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.115+0000 7f88b259c700 1 -- 172.21.15.67:0/319289574 learned_addr learned my addr 172.21.15.67:0/319289574 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:36:39.122 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.115+0000 7f88b2d9d700 1 --2- 172.21.15.67:0/319289574 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f88b40cfc60 0x7f88b40cbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:39.122 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.115+0000 7f88b1d9b700 1 --2- 172.21.15.67:0/319289574 >> v2:172.21.15.72:3300/0 conn(0x7f88b415f1f0 0x7f88b40cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:39.122 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.115+0000 7f88b159a700 1 -- 172.21.15.67:0/319289574 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1071865259 0 0) 0x7f88b415e7a0 con 0x7f88b415ab10 2023-10-14T10:36:39.122 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.115+0000 7f88b159a700 1 -- 172.21.15.67:0/319289574 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88a00035b0 con 0x7f88b415ab10 2023-10-14T10:36:39.122 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.115+0000 7f88b159a700 1 -- 172.21.15.67:0/319289574 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 223083801 0 0) 0x7f88a00035b0 con 0x7f88b415ab10 2023-10-14T10:36:39.123 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.115+0000 7f88b159a700 1 -- 172.21.15.67:0/319289574 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f88b40cfc60 msgr2=0x7f88b40cbf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:39.123 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.115+0000 7f88b159a700 1 --2- 172.21.15.67:0/319289574 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f88b40cfc60 0x7f88b40cbf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:39.123 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.115+0000 7f88b159a700 1 -- 172.21.15.67:0/319289574 >> v2:172.21.15.72:3300/0 conn(0x7f88b415f1f0 msgr2=0x7f88b40cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:39.123 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.115+0000 7f88b159a700 1 --2- 172.21.15.67:0/319289574 >> v2:172.21.15.72:3300/0 conn(0x7f88b415f1f0 0x7f88b40cf720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:39.123 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.115+0000 7f88b159a700 1 -- 172.21.15.67:0/319289574 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88b40b2c80 con 0x7f88b415ab10 2023-10-14T10:36:39.124 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.115+0000 7f88b159a700 1 -- 172.21.15.67:0/319289574 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3180043755 0 0) 0x7f88a4001670 con 0x7f88b415ab10 2023-10-14T10:36:39.124 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.115+0000 7f88b1d9b700 1 --2- 172.21.15.67:0/319289574 >> v2:172.21.15.72:3300/0 conn(0x7f88b415f1f0 0x7f88b40cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-14T10:36:39.124 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.115+0000 7f88b2d9d700 1 --2- 172.21.15.67:0/319289574 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f88b40cfc60 0x7f88b40cbf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-14T10:36:39.124 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.115+0000 7f88b159a700 1 -- 172.21.15.67:0/319289574 >> v1:172.21.15.67:6789/0 conn(0x7f88b415ab10 legacy=0x7f88b40b1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:39.124 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.115+0000 7f88b159a700 1 --2- 172.21.15.67:0/319289574 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f88a0003a00 0x7f88a0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:39.124 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.115+0000 7f88b159a700 1 --2- 172.21.15.67:0/319289574 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f88a0004850 0x7f88a0004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:39.125 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.115+0000 7f88b159a700 1 --2- 172.21.15.67:0/319289574 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f88b40cfc60 0x7f88a0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:39.125 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.115+0000 7f88b159a700 1 -- 172.21.15.67:0/319289574 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b40d00d0 con 0x7f88a0003a00 2023-10-14T10:36:39.125 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.115+0000 7f88b159a700 1 -- 172.21.15.67:0/319289574 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b40b29a0 con 0x7f88a0004850 2023-10-14T10:36:39.125 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.115+0000 7f88b159a700 1 -- 172.21.15.67:0/319289574 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88a000ba10 con 0x7f88b40cfc60 2023-10-14T10:36:39.125 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.115+0000 7f88b259c700 1 --2- 172.21.15.67:0/319289574 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f88a0003a00 0x7f88a0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:39.126 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.115+0000 7f88b1d9b700 1 --2- 172.21.15.67:0/319289574 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f88a0004850 0x7f88a0004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:39.126 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.116+0000 7f88b2d9d700 1 --2- 172.21.15.67:0/319289574 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f88b40cfc60 0x7f88a0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:39.126 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.116+0000 7f88b259c700 1 -- 172.21.15.67:0/319289574 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f88b40cfc60 msgr2=0x7f88a0005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:39.126 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.116+0000 7f88b259c700 1 --2- 172.21.15.67:0/319289574 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f88b40cfc60 0x7f88a0005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:39.126 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.116+0000 7f88b259c700 1 -- 172.21.15.67:0/319289574 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f88a0004850 msgr2=0x7f88a0004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:39.127 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.116+0000 7f88b259c700 1 --2- 172.21.15.67:0/319289574 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f88a0004850 0x7f88a0004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:39.127 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.116+0000 7f88b259c700 1 -- 172.21.15.67:0/319289574 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88ac009a70 con 0x7f88a0003a00 2023-10-14T10:36:39.127 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.116+0000 7f88b2d9d700 1 --2- 172.21.15.67:0/319289574 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f88b40cfc60 0x7f88a0005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-14T10:36:39.127 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.116+0000 7f88b259c700 1 --2- 172.21.15.67:0/319289574 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f88a0003a00 0x7f88a0003e20 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f88a400c320 tx=0x7f88a0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be98f3b05cec9337 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:39.127 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.116+0000 7f88b159a700 1 -- 172.21.15.67:0/319289574 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a400ce60 con 0x7f88a0003a00 2023-10-14T10:36:39.128 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.116+0000 7f88b159a700 1 -- 172.21.15.67:0/319289574 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88a4005960 con 0x7f88a0003a00 2023-10-14T10:36:39.128 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.116+0000 7f88b159a700 1 -- 172.21.15.67:0/319289574 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a4005cf0 con 0x7f88a0003a00 2023-10-14T10:36:39.128 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.116+0000 7f88b8a92700 1 -- 172.21.15.67:0/319289574 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f88a0003a00 msgr2=0x7f88a0003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:39.128 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.116+0000 7f88b8a92700 1 --2- 172.21.15.67:0/319289574 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f88a0003a00 0x7f88a0003e20 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f88a400c320 tx=0x7f88a0005880 comp rx=0 tx=0).stop 2023-10-14T10:36:39.128 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.116+0000 7f88b2d9d700 1 -- 172.21.15.67:0/319289574 reap_dead start 2023-10-14T10:36:39.129 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.117+0000 7f88b8a92700 1 -- 172.21.15.67:0/319289574 shutdown_connections 2023-10-14T10:36:39.129 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.117+0000 7f88b8a92700 1 -- 172.21.15.67:0/319289574 >> 172.21.15.67:0/319289574 conn(0x7f88b406dae0 msgr2=0x7f88b406def0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:39.129 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.117+0000 7f88b8a92700 1 -- 172.21.15.67:0/319289574 shutdown_connections 2023-10-14T10:36:39.129 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.117+0000 7f88b8a92700 1 -- 172.21.15.67:0/319289574 wait complete. 2023-10-14T10:36:39.129 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.118+0000 7f88b8a92700 1 Processor -- start 2023-10-14T10:36:39.130 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.118+0000 7f88b8a92700 1 -- start start 2023-10-14T10:36:39.132 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.119+0000 7f88b8a92700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f88a0003a00 0x7f88b41f4a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:39.132 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.119+0000 7f88b8a92700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f88b41f4480 0x7f88b41f6820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:39.132 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.119+0000 7f88b8a92700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f88b4154e60 0x7f88b40b3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:39.133 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.119+0000 7f88b8a92700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b4155ef0 con 0x7f88a0003a00 2023-10-14T10:36:39.133 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.119+0000 7f88b8a92700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b40c8ac0 con 0x7f88b4154e60 2023-10-14T10:36:39.133 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.119+0000 7f88b8a92700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88b4155650 con 0x7f88b41f4480 2023-10-14T10:36:39.133 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.119+0000 7f88b259c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f88a0003a00 0x7f88b41f4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:39.133 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.119+0000 7f88b259c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f88a0003a00 0x7f88b41f4a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45078/0 (socket says 172.21.15.67:45078) 2023-10-14T10:36:39.134 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.119+0000 7f88b259c700 1 -- 172.21.15.67:0/4084842411 learned_addr learned my addr 172.21.15.67:0/4084842411 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:39.134 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.119+0000 7f88b2d9d700 1 --2- 172.21.15.67:0/4084842411 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f88b4154e60 0x7f88b40b3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:39.134 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.119+0000 7f88b1d9b700 1 --2- 172.21.15.67:0/4084842411 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f88b41f4480 0x7f88b41f6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:39.134 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.119+0000 7f88b259c700 1 -- 172.21.15.67:0/4084842411 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f88b41f4480 msgr2=0x7f88b41f6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:39.134 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.119+0000 7f88b259c700 1 --2- 172.21.15.67:0/4084842411 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f88b41f4480 0x7f88b41f6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:39.135 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.119+0000 7f88b259c700 1 -- 172.21.15.67:0/4084842411 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f88b4154e60 msgr2=0x7f88b40b3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:39.135 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.119+0000 7f88b259c700 1 --2- 172.21.15.67:0/4084842411 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f88b4154e60 0x7f88b40b3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:39.135 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.119+0000 7f88b259c700 1 -- 172.21.15.67:0/4084842411 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88b4155930 con 0x7f88a0003a00 2023-10-14T10:36:39.135 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.119+0000 7f88b259c700 1 --2- 172.21.15.67:0/4084842411 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f88a0003a00 0x7f88b41f4a30 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f88a4008fd0 tx=0x7f88a4001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:39.135 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.119+0000 7f88aaffd700 1 -- 172.21.15.67:0/4084842411 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a4001680 con 0x7f88a0003a00 2023-10-14T10:36:39.136 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.119+0000 7f88aaffd700 1 -- 172.21.15.67:0/4084842411 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88a4004a80 con 0x7f88a0003a00 2023-10-14T10:36:39.136 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.119+0000 7f88aaffd700 1 -- 172.21.15.67:0/4084842411 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a4004e30 con 0x7f88a0003a00 2023-10-14T10:36:39.136 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.119+0000 7f88b8a92700 1 -- 172.21.15.67:0/4084842411 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88b4155c10 con 0x7f88a0003a00 2023-10-14T10:36:39.136 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.120+0000 7f88b8a92700 1 -- 172.21.15.67:0/4084842411 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88b41552e0 con 0x7f88a0003a00 2023-10-14T10:36:39.136 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.120+0000 7f88b8a92700 1 -- 172.21.15.67:0/4084842411 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88b40c90d0 con 0x7f88a0003a00 2023-10-14T10:36:39.136 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.121+0000 7f88aaffd700 1 -- 172.21.15.67:0/4084842411 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f88a40070d0 con 0x7f88a0003a00 2023-10-14T10:36:39.137 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.121+0000 7f88aaffd700 1 --2- 172.21.15.67:0/4084842411 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f889406af00 0x7f889406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:39.137 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.121+0000 7f88aaffd700 1 -- 172.21.15.67:0/4084842411 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f88a408a930 con 0x7f88a0003a00 2023-10-14T10:36:39.137 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.122+0000 7f88b1d9b700 1 --2- 172.21.15.67:0/4084842411 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f889406af00 0x7f889406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:39.137 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.122+0000 7f88b1d9b700 1 --2- 172.21.15.67:0/4084842411 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f889406af00 0x7f889406d3c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f88b40af4e0 tx=0x7f889c0091c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:39.137 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.126+0000 7f88aaffd700 1 -- 172.21.15.67:0/4084842411 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f88a4055c50 con 0x7f88a0003a00 2023-10-14T10:36:39.326 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.324+0000 7f88b8a92700 1 -- 172.21.15.67:0/4084842411 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f88b41f4860 con 0x7f88a0003a00 2023-10-14T10:36:39.327 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.325+0000 7f88aaffd700 1 -- 172.21.15.67:0/4084842411 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f88a4007da0 con 0x7f88a0003a00 2023-10-14T10:36:39.328 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2023-10-14T10:36:39.329 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.327+0000 7f88b8a92700 1 -- 172.21.15.67:0/4084842411 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f889406af00 msgr2=0x7f889406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:39.329 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.327+0000 7f88b8a92700 1 --2- 172.21.15.67:0/4084842411 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f889406af00 0x7f889406d3c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f88b40af4e0 tx=0x7f889c0091c0 comp rx=0 tx=0).stop 2023-10-14T10:36:39.330 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.327+0000 7f88b8a92700 1 -- 172.21.15.67:0/4084842411 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f88a0003a00 msgr2=0x7f88b41f4a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:39.330 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.327+0000 7f88b8a92700 1 --2- 172.21.15.67:0/4084842411 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f88a0003a00 0x7f88b41f4a30 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f88a4008fd0 tx=0x7f88a4001a00 comp rx=0 tx=0).stop 2023-10-14T10:36:39.330 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.327+0000 7f88b8a92700 1 -- 172.21.15.67:0/4084842411 shutdown_connections 2023-10-14T10:36:39.330 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.327+0000 7f88b8a92700 1 --2- 172.21.15.67:0/4084842411 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f889406af00 0x7f889406d3c0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:39.331 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.327+0000 7f88b8a92700 1 --2- 172.21.15.67:0/4084842411 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f88b4154e60 0x7f88b40b3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:39.331 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.327+0000 7f88b8a92700 1 --2- 172.21.15.67:0/4084842411 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f88a0003a00 0x7f88b41f4a30 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:39.331 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.327+0000 7f88b8a92700 1 --2- 172.21.15.67:0/4084842411 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f88b41f4480 0x7f88b41f6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:39.331 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.327+0000 7f88b8a92700 1 -- 172.21.15.67:0/4084842411 >> 172.21.15.67:0/4084842411 conn(0x7f88b406dae0 msgr2=0x7f88b40b5200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:39.331 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.327+0000 7f88b8a92700 1 -- 172.21.15.67:0/4084842411 shutdown_connections 2023-10-14T10:36:39.332 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.327+0000 7f88b8a92700 1 -- 172.21.15.67:0/4084842411 wait complete. 2023-10-14T10:36:39.341 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-14T10:36:39.359 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-14T10:36:39.386 DEBUG:teuthology.orchestra.run.smithi072:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-14T10:36:39.473 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.471+0000 7fe2b9234700 1 Processor -- start 2023-10-14T10:36:39.473 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.471+0000 7fe2b9234700 1 -- start start 2023-10-14T10:36:39.475 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.472+0000 7fe2b9234700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fe2b415e2f0 0x7fe2b40cede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:39.476 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.472+0000 7fe2b9234700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2b40b2d60 con 0x7fe2b406a840 2023-10-14T10:36:39.476 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.472+0000 7fe2b9234700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2b40cdd30 con 0x7fe2b40cd950 2023-10-14T10:36:39.476 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.472+0000 7fe2b9234700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2b41619a0 con 0x7fe2b415e2f0 2023-10-14T10:36:39.476 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.472+0000 7fe2b359e700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fe2b406a840 0x7fe2b406ac60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:52858/0 (socket says 172.21.15.67:52858) 2023-10-14T10:36:39.476 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.472+0000 7fe2b359e700 1 -- 172.21.15.67:0/2067757835 learned_addr learned my addr 172.21.15.67:0/2067757835 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:36:39.477 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.472+0000 7fe2b259c700 1 --2- 172.21.15.67:0/2067757835 >> v2:172.21.15.72:3300/0 conn(0x7fe2b415e2f0 0x7fe2b40cede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:39.477 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.472+0000 7fe2b1d9b700 1 -- 172.21.15.67:0/2067757835 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1232059475 0 0) 0x7fe2b40b2d60 con 0x7fe2b406a840 2023-10-14T10:36:39.477 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.472+0000 7fe2b1d9b700 1 -- 172.21.15.67:0/2067757835 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe29c0035b0 con 0x7fe2b406a840 2023-10-14T10:36:39.477 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.472+0000 7fe2b1d9b700 1 -- 172.21.15.67:0/2067757835 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3805435588 0 0) 0x7fe29c0035b0 con 0x7fe2b406a840 2023-10-14T10:36:39.477 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.472+0000 7fe2b259c700 1 -- 172.21.15.67:0/2067757835 >> v1:172.21.15.72:6789/0 conn(0x7fe2b40cd950 legacy=0x7fe2b415f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:39.478 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.472+0000 7fe2b259c700 1 -- 172.21.15.67:0/2067757835 >> v1:172.21.15.67:6789/0 conn(0x7fe2b406a840 legacy=0x7fe2b406ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:39.478 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.472+0000 7fe2b259c700 1 -- 172.21.15.67:0/2067757835 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2b415e920 con 0x7fe2b415e2f0 2023-10-14T10:36:39.478 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.472+0000 7fe2b259c700 1 --2- 172.21.15.67:0/2067757835 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe2b415e2f0 0x7fe2b40cede0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8009820 tx=0x7fe2a8009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d99cec1d3432c8bc server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:39.478 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.473+0000 7fe2b1d9b700 1 -- 172.21.15.67:0/2067757835 <== mon.1 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2ac0019f0 con 0x7fe2b415e2f0 2023-10-14T10:36:39.479 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.473+0000 7fe2b1d9b700 1 -- 172.21.15.67:0/2067757835 <== mon.1 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2a8005c30 con 0x7fe2b415e2f0 2023-10-14T10:36:39.479 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.473+0000 7fe2b1d9b700 1 -- 172.21.15.67:0/2067757835 <== mon.1 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a8004030 con 0x7fe2b415e2f0 2023-10-14T10:36:39.479 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.473+0000 7fe2b9234700 1 -- 172.21.15.67:0/2067757835 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe2b415e2f0 msgr2=0x7fe2b40cede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:39.479 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.473+0000 7fe2b9234700 1 --2- 172.21.15.67:0/2067757835 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe2b415e2f0 0x7fe2b40cede0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8009820 tx=0x7fe2a8009b30 comp rx=0 tx=0).stop 2023-10-14T10:36:39.479 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.473+0000 7fe2b9234700 1 -- 172.21.15.67:0/2067757835 shutdown_connections 2023-10-14T10:36:39.480 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.473+0000 7fe2b9234700 1 --2- 172.21.15.67:0/2067757835 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe2b415e2f0 0x7fe2b40cede0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:39.480 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.473+0000 7fe2b9234700 1 -- 172.21.15.67:0/2067757835 >> 172.21.15.67:0/2067757835 conn(0x7fe2b406dae0 msgr2=0x7fe2b406def0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:39.480 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.473+0000 7fe2b9234700 1 -- 172.21.15.67:0/2067757835 shutdown_connections 2023-10-14T10:36:39.480 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.473+0000 7fe2b9234700 1 -- 172.21.15.67:0/2067757835 wait complete. 2023-10-14T10:36:39.483 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.475+0000 7fe2b9234700 1 Processor -- start 2023-10-14T10:36:39.483 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.475+0000 7fe2b9234700 1 -- start start 2023-10-14T10:36:39.484 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.475+0000 7fe2b9234700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fe2b406a840 0x7fe2b40c5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:39.484 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.475+0000 7fe2b9234700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe2b40cd950 0x7fe2b40c6c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:39.484 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.475+0000 7fe2b9234700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2b415e2f0 0x7fe2b415ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:39.484 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.475+0000 7fe2b9234700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b40b4aa0 con 0x7fe2b415e2f0 2023-10-14T10:36:39.484 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.475+0000 7fe2b9234700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b40c57b0 con 0x7fe2b40cd950 2023-10-14T10:36:39.485 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.475+0000 7fe2b9234700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b40c7150 con 0x7fe2b406a840 2023-10-14T10:36:39.485 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.476+0000 7fe2b359e700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2b415e2f0 0x7fe2b415ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:39.485 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.476+0000 7fe2b359e700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2b415e2f0 0x7fe2b415ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45096/0 (socket says 172.21.15.67:45096) 2023-10-14T10:36:39.485 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.476+0000 7fe2b359e700 1 -- 172.21.15.67:0/3966922201 learned_addr learned my addr 172.21.15.67:0/3966922201 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:39.485 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.476+0000 7fe2b259c700 1 --2- 172.21.15.67:0/3966922201 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe2b40cd950 0x7fe2b40c6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:39.486 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.476+0000 7fe2b2d9d700 1 --2- 172.21.15.67:0/3966922201 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fe2b406a840 0x7fe2b40c5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:39.486 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.476+0000 7fe2b359e700 1 -- 172.21.15.67:0/3966922201 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fe2b406a840 msgr2=0x7fe2b40c5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:39.486 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.476+0000 7fe2b359e700 1 --2- 172.21.15.67:0/3966922201 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fe2b406a840 0x7fe2b40c5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:39.486 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.476+0000 7fe2b359e700 1 -- 172.21.15.67:0/3966922201 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe2b40cd950 msgr2=0x7fe2b40c6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:39.486 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.476+0000 7fe2b359e700 1 --2- 172.21.15.67:0/3966922201 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe2b40cd950 0x7fe2b40c6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:39.487 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.476+0000 7fe2b359e700 1 -- 172.21.15.67:0/3966922201 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2a8009430 con 0x7fe2b415e2f0 2023-10-14T10:36:39.487 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.476+0000 7fe2b2d9d700 1 --2- 172.21.15.67:0/3966922201 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fe2b406a840 0x7fe2b40c5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-14T10:36:39.487 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.476+0000 7fe2b359e700 1 --2- 172.21.15.67:0/3966922201 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2b415e2f0 0x7fe2b415ba90 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac00cc70 tx=0x7fe2ac00cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:39.487 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.476+0000 7fe29b7fe700 1 -- 172.21.15.67:0/3966922201 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2ac00ab70 con 0x7fe2b415e2f0 2023-10-14T10:36:39.487 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.476+0000 7fe29b7fe700 1 -- 172.21.15.67:0/3966922201 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2ac00acd0 con 0x7fe2b415e2f0 2023-10-14T10:36:39.488 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.476+0000 7fe2b9234700 1 -- 172.21.15.67:0/3966922201 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2b40b4c80 con 0x7fe2b415e2f0 2023-10-14T10:36:39.488 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.476+0000 7fe29b7fe700 1 -- 172.21.15.67:0/3966922201 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2ac007310 con 0x7fe2b415e2f0 2023-10-14T10:36:39.488 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.476+0000 7fe2b9234700 1 -- 172.21.15.67:0/3966922201 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2b40b5240 con 0x7fe2b415e2f0 2023-10-14T10:36:39.488 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.477+0000 7fe2b9234700 1 -- 172.21.15.67:0/3966922201 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2b4067d20 con 0x7fe2b415e2f0 2023-10-14T10:36:39.489 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.478+0000 7fe29b7fe700 1 -- 172.21.15.67:0/3966922201 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe2ac007470 con 0x7fe2b415e2f0 2023-10-14T10:36:39.489 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.478+0000 7fe29b7fe700 1 --2- 172.21.15.67:0/3966922201 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fe2a006af00 0x7fe2a006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:39.489 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.478+0000 7fe29b7fe700 1 -- 172.21.15.67:0/3966922201 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe2ac089460 con 0x7fe2b415e2f0 2023-10-14T10:36:39.489 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.478+0000 7fe2b2d9d700 1 --2- 172.21.15.67:0/3966922201 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fe2a006af00 0x7fe2a006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:39.489 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.479+0000 7fe2b2d9d700 1 --2- 172.21.15.67:0/3966922201 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fe2a006af00 0x7fe2a006d3c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe2a4000f60 tx=0x7fe2a4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:39.490 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.483+0000 7fe29b7fe700 1 -- 172.21.15.67:0/3966922201 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe2ac054510 con 0x7fe2b415e2f0 2023-10-14T10:36:39.684 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.682+0000 7fe2b9234700 1 -- 172.21.15.67:0/3966922201 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe2b40b2630 con 0x7fe2b415e2f0 2023-10-14T10:36:39.684 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.682+0000 7fe29b7fe700 1 -- 172.21.15.67:0/3966922201 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe2ac057b30 con 0x7fe2b415e2f0 2023-10-14T10:36:39.685 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2023-10-14T10:36:39.686 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.684+0000 7fe2b9234700 1 -- 172.21.15.67:0/3966922201 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fe2a006af00 msgr2=0x7fe2a006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:39.687 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.684+0000 7fe2b9234700 1 --2- 172.21.15.67:0/3966922201 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fe2a006af00 0x7fe2a006d3c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe2a4000f60 tx=0x7fe2a4009040 comp rx=0 tx=0).stop 2023-10-14T10:36:39.687 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.684+0000 7fe2b9234700 1 -- 172.21.15.67:0/3966922201 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2b415e2f0 msgr2=0x7fe2b415ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:39.687 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.684+0000 7fe2b9234700 1 --2- 172.21.15.67:0/3966922201 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2b415e2f0 0x7fe2b415ba90 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac00cc70 tx=0x7fe2ac00cf80 comp rx=0 tx=0).stop 2023-10-14T10:36:39.687 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.684+0000 7fe2b9234700 1 -- 172.21.15.67:0/3966922201 shutdown_connections 2023-10-14T10:36:39.688 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.684+0000 7fe2b9234700 1 --2- 172.21.15.67:0/3966922201 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fe2a006af00 0x7fe2a006d3c0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:39.688 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.684+0000 7fe2b9234700 1 --2- 172.21.15.67:0/3966922201 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2b415e2f0 0x7fe2b415ba90 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:39.688 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.684+0000 7fe2b9234700 1 --2- 172.21.15.67:0/3966922201 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fe2b406a840 0x7fe2b40c5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:39.688 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.684+0000 7fe2b9234700 1 --2- 172.21.15.67:0/3966922201 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe2b40cd950 0x7fe2b40c6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:39.688 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.684+0000 7fe2b9234700 1 -- 172.21.15.67:0/3966922201 >> 172.21.15.67:0/3966922201 conn(0x7fe2b406dae0 msgr2=0x7fe2b40b1880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:39.689 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.684+0000 7fe2b9234700 1 -- 172.21.15.67:0/3966922201 shutdown_connections 2023-10-14T10:36:39.689 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.684+0000 7fe2b9234700 1 -- 172.21.15.67:0/3966922201 wait complete. 2023-10-14T10:36:39.699 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-14T10:36:39.866 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.864+0000 7fdc4903e700 1 Processor -- start 2023-10-14T10:36:39.867 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.864+0000 7fdc4903e700 1 -- start start 2023-10-14T10:36:39.869 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.865+0000 7fdc4903e700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fdc4415ab10 0x7fdc4415f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:39.869 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.865+0000 7fdc4903e700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fdc440b1f50 0x7fdc440cf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:39.869 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.865+0000 7fdc4903e700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fdc440b23d0 0x7fdc440cfc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:39.870 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.865+0000 7fdc4903e700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc4415e7a0 con 0x7fdc440b23d0 2023-10-14T10:36:39.870 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.865+0000 7fdc4903e700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc4415f730 con 0x7fdc440b1f50 2023-10-14T10:36:39.870 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.865+0000 7fdc4903e700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc440b29a0 con 0x7fdc4415ab10 2023-10-14T10:36:39.870 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.865+0000 7fdc4359e700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fdc440b23d0 0x7fdc440cfc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:39.871 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.865+0000 7fdc4259c700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fdc440b1f50 0x7fdc440cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:39.871 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.865+0000 7fdc42d9d700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fdc4415ab10 0x7fdc4415f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:39.871 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.865+0000 7fdc4259c700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fdc440b1f50 0x7fdc440cf720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.67:39892/0 (socket says 172.21.15.67:39892) 2023-10-14T10:36:39.871 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.865+0000 7fdc4259c700 1 -- 172.21.15.67:0/686111299 learned_addr learned my addr 172.21.15.67:0/686111299 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:39.871 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.865+0000 7fdc4359e700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fdc440b23d0 0x7fdc440cfc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45104/0 (socket says 172.21.15.67:45104) 2023-10-14T10:36:39.872 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.865+0000 7fdc42d9d700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fdc4415ab10 0x7fdc4415f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3301/0 says I am v2:172.21.15.67:33150/0 (socket says 172.21.15.67:33150) 2023-10-14T10:36:39.872 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.866+0000 7fdc4359e700 1 -- 172.21.15.67:0/686111299 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fdc4415ab10 msgr2=0x7fdc4415f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:39.872 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.866+0000 7fdc4359e700 1 --2- 172.21.15.67:0/686111299 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fdc4415ab10 0x7fdc4415f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:39.872 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.866+0000 7fdc4359e700 1 -- 172.21.15.67:0/686111299 >> v2:172.21.15.72:3300/0 conn(0x7fdc440b1f50 msgr2=0x7fdc440cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:39.872 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.866+0000 7fdc4359e700 1 --2- 172.21.15.67:0/686111299 >> v2:172.21.15.72:3300/0 conn(0x7fdc440b1f50 0x7fdc440cf720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:39.873 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.866+0000 7fdc4359e700 1 -- 172.21.15.67:0/686111299 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc440b2c80 con 0x7fdc440b23d0 2023-10-14T10:36:39.873 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.866+0000 7fdc4259c700 1 --2- 172.21.15.67:0/686111299 >> v2:172.21.15.72:3300/0 conn(0x7fdc440b1f50 0x7fdc440cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-14T10:36:39.873 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.866+0000 7fdc42d9d700 1 --2- 172.21.15.67:0/686111299 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fdc4415ab10 0x7fdc4415f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-14T10:36:39.873 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.866+0000 7fdc4359e700 1 --2- 172.21.15.67:0/686111299 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc440b23d0 0x7fdc440cfc60 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fdc440d1560 tx=0x7fdc3c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f619feef70fc0e4e server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:39.873 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.866+0000 7fdc41d9b700 1 -- 172.21.15.67:0/686111299 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc3c00e070 con 0x7fdc440b23d0 2023-10-14T10:36:39.874 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.866+0000 7fdc41d9b700 1 -- 172.21.15.67:0/686111299 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc3c00bb20 con 0x7fdc440b23d0 2023-10-14T10:36:39.874 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.866+0000 7fdc41d9b700 1 -- 172.21.15.67:0/686111299 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc3c012430 con 0x7fdc440b23d0 2023-10-14T10:36:39.874 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.866+0000 7fdc4903e700 1 -- 172.21.15.67:0/686111299 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc440b23d0 msgr2=0x7fdc440cfc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:39.874 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.866+0000 7fdc4903e700 1 --2- 172.21.15.67:0/686111299 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc440b23d0 0x7fdc440cfc60 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fdc440d1560 tx=0x7fdc3c00b330 comp rx=0 tx=0).stop 2023-10-14T10:36:39.874 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.867+0000 7fdc4903e700 1 -- 172.21.15.67:0/686111299 shutdown_connections 2023-10-14T10:36:39.875 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.867+0000 7fdc4903e700 1 --2- 172.21.15.67:0/686111299 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc440b23d0 0x7fdc440cfc60 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:39.875 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.867+0000 7fdc4903e700 1 --2- 172.21.15.67:0/686111299 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fdc4415ab10 0x7fdc4415f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:39.875 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.867+0000 7fdc4903e700 1 --2- 172.21.15.67:0/686111299 >> v2:172.21.15.72:3300/0 conn(0x7fdc440b1f50 0x7fdc440cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:39.875 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.867+0000 7fdc4903e700 1 -- 172.21.15.67:0/686111299 >> 172.21.15.67:0/686111299 conn(0x7fdc4406dae0 msgr2=0x7fdc4406def0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:39.876 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.867+0000 7fdc4903e700 1 -- 172.21.15.67:0/686111299 shutdown_connections 2023-10-14T10:36:39.876 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.867+0000 7fdc4903e700 1 -- 172.21.15.67:0/686111299 wait complete. 2023-10-14T10:36:39.879 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.868+0000 7fdc4903e700 1 Processor -- start 2023-10-14T10:36:39.880 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.869+0000 7fdc4903e700 1 -- start start 2023-10-14T10:36:39.880 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.869+0000 7fdc4903e700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdc440b1f50 0x7fdc4415c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:39.880 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.869+0000 7fdc4903e700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fdc440b23d0 0x7fdc44149340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:39.880 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.870+0000 7fdc4903e700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc4415ab10 0x7fdc4415aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:39.881 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.870+0000 7fdc4903e700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc4415cee0 con 0x7fdc4415ab10 2023-10-14T10:36:39.881 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.870+0000 7fdc4903e700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc4414a730 con 0x7fdc440b1f50 2023-10-14T10:36:39.881 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.870+0000 7fdc4903e700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc44149880 con 0x7fdc440b23d0 2023-10-14T10:36:39.881 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.870+0000 7fdc4359e700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc4415ab10 0x7fdc4415aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:39.881 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.870+0000 7fdc4359e700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc4415ab10 0x7fdc4415aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45110/0 (socket says 172.21.15.67:45110) 2023-10-14T10:36:39.882 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.870+0000 7fdc4359e700 1 -- 172.21.15.67:0/1832470401 learned_addr learned my addr 172.21.15.67:0/1832470401 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:39.882 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.870+0000 7fdc42d9d700 1 --2- 172.21.15.67:0/1832470401 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdc440b1f50 0x7fdc4415c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:39.882 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.870+0000 7fdc4259c700 1 --2- 172.21.15.67:0/1832470401 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fdc440b23d0 0x7fdc44149340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:39.883 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.870+0000 7fdc4359e700 1 -- 172.21.15.67:0/1832470401 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fdc440b23d0 msgr2=0x7fdc44149340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:39.883 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.870+0000 7fdc4359e700 1 --2- 172.21.15.67:0/1832470401 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fdc440b23d0 0x7fdc44149340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:39.883 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.870+0000 7fdc4359e700 1 -- 172.21.15.67:0/1832470401 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdc440b1f50 msgr2=0x7fdc4415c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:39.883 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.870+0000 7fdc4359e700 1 --2- 172.21.15.67:0/1832470401 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdc440b1f50 0x7fdc4415c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:39.884 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.870+0000 7fdc4359e700 1 -- 172.21.15.67:0/1832470401 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc34009500 con 0x7fdc4415ab10 2023-10-14T10:36:39.884 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.870+0000 7fdc4359e700 1 --2- 172.21.15.67:0/1832470401 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc4415ab10 0x7fdc4415aef0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c014040 tx=0x7fdc3c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:39.884 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.870+0000 7fdc2b7fe700 1 -- 172.21.15.67:0/1832470401 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc3c0048b0 con 0x7fdc4415ab10 2023-10-14T10:36:39.884 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.870+0000 7fdc2b7fe700 1 -- 172.21.15.67:0/1832470401 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc3c020070 con 0x7fdc4415ab10 2023-10-14T10:36:39.884 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.870+0000 7fdc4903e700 1 -- 172.21.15.67:0/1832470401 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc3800b740 con 0x7fdc4415ab10 2023-10-14T10:36:39.884 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.870+0000 7fdc2b7fe700 1 -- 172.21.15.67:0/1832470401 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc3c0276c0 con 0x7fdc4415ab10 2023-10-14T10:36:39.885 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.870+0000 7fdc4903e700 1 -- 172.21.15.67:0/1832470401 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc3c009a70 con 0x7fdc4415ab10 2023-10-14T10:36:39.885 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.871+0000 7fdc4903e700 1 -- 172.21.15.67:0/1832470401 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc44067d20 con 0x7fdc4415ab10 2023-10-14T10:36:39.885 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.872+0000 7fdc2b7fe700 1 -- 172.21.15.67:0/1832470401 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fdc3c012430 con 0x7fdc4415ab10 2023-10-14T10:36:39.885 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.872+0000 7fdc2b7fe700 1 --2- 172.21.15.67:0/1832470401 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fdc3006af00 0x7fdc3006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:39.885 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.872+0000 7fdc2b7fe700 1 -- 172.21.15.67:0/1832470401 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdc3c022070 con 0x7fdc4415ab10 2023-10-14T10:36:39.886 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.873+0000 7fdc42d9d700 1 --2- 172.21.15.67:0/1832470401 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fdc3006af00 0x7fdc3006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:39.886 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.873+0000 7fdc42d9d700 1 --2- 172.21.15.67:0/1832470401 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fdc3006af00 0x7fdc3006d3c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fdc380114a0 tx=0x7fdc3800b420 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:39.886 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:39.877+0000 7fdc2b7fe700 1 -- 172.21.15.67:0/1832470401 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdc3c0561f0 con 0x7fdc4415ab10 2023-10-14T10:36:40.080 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.078+0000 7fdc4903e700 1 -- 172.21.15.67:0/1832470401 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdc4414a870 con 0x7fdc4415ab10 2023-10-14T10:36:40.080 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.078+0000 7fdc2b7fe700 1 -- 172.21.15.67:0/1832470401 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdc3c02c020 con 0x7fdc4415ab10 2023-10-14T10:36:40.081 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2023-10-14T10:36:40.083 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.080+0000 7fdc4903e700 1 -- 172.21.15.67:0/1832470401 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fdc3006af00 msgr2=0x7fdc3006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:40.083 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.080+0000 7fdc4903e700 1 --2- 172.21.15.67:0/1832470401 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fdc3006af00 0x7fdc3006d3c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fdc380114a0 tx=0x7fdc3800b420 comp rx=0 tx=0).stop 2023-10-14T10:36:40.083 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.080+0000 7fdc4903e700 1 -- 172.21.15.67:0/1832470401 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc4415ab10 msgr2=0x7fdc4415aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:40.083 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.080+0000 7fdc4903e700 1 --2- 172.21.15.67:0/1832470401 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc4415ab10 0x7fdc4415aef0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c014040 tx=0x7fdc3c004ad0 comp rx=0 tx=0).stop 2023-10-14T10:36:40.084 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.080+0000 7fdc4903e700 1 -- 172.21.15.67:0/1832470401 shutdown_connections 2023-10-14T10:36:40.084 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.080+0000 7fdc4903e700 1 --2- 172.21.15.67:0/1832470401 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fdc3006af00 0x7fdc3006d3c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:40.084 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.080+0000 7fdc4903e700 1 --2- 172.21.15.67:0/1832470401 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc4415ab10 0x7fdc4415aef0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:40.084 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.080+0000 7fdc4903e700 1 --2- 172.21.15.67:0/1832470401 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdc440b1f50 0x7fdc4415c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:40.084 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.080+0000 7fdc4903e700 1 --2- 172.21.15.67:0/1832470401 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fdc440b23d0 0x7fdc44149340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:40.085 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.080+0000 7fdc4903e700 1 -- 172.21.15.67:0/1832470401 >> 172.21.15.67:0/1832470401 conn(0x7fdc4406dae0 msgr2=0x7fdc440b5040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:40.085 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.080+0000 7fdc4903e700 1 -- 172.21.15.67:0/1832470401 shutdown_connections 2023-10-14T10:36:40.085 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.080+0000 7fdc4903e700 1 -- 172.21.15.67:0/1832470401 wait complete. 2023-10-14T10:36:40.095 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-14T10:36:40.220 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.218+0000 7f3cfd195700 1 Processor -- start 2023-10-14T10:36:40.220 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.218+0000 7f3cfd195700 1 -- start start 2023-10-14T10:36:40.223 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.219+0000 7f3cfd195700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f3cf815a850 0x7f3cf80cbc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:40.223 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.219+0000 7f3cfd195700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f3cf80b2940 0x7f3cf80ccda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:40.223 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.219+0000 7f3cfd195700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cf80ce3b0 con 0x7f3cf80cd2e0 2023-10-14T10:36:40.224 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.219+0000 7f3cfd195700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cf80b2dc0 con 0x7f3cf80b2940 2023-10-14T10:36:40.224 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.219+0000 7f3cfd195700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf80b6290 con 0x7f3cf815a850 2023-10-14T10:36:40.224 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.219+0000 7f3cf659c700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f3cf80b2940 0x7f3cf80ccda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:40.225 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.219+0000 7f3cf759e700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f3cf80cd2e0 0x7f3cf8071ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:52886/0 (socket says 172.21.15.67:52886) 2023-10-14T10:36:40.225 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.219+0000 7f3cf759e700 1 -- 172.21.15.67:0/1629680279 learned_addr learned my addr 172.21.15.67:0/1629680279 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:36:40.225 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.219+0000 7f3cf6d9d700 1 --2- 172.21.15.67:0/1629680279 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f3cf815a850 0x7f3cf80cbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:40.225 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.219+0000 7f3cf659c700 1 -- 172.21.15.67:0/1629680279 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f3cf815a850 msgr2=0x7f3cf80cbc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:40.225 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.219+0000 7f3cf659c700 1 --2- 172.21.15.67:0/1629680279 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f3cf815a850 0x7f3cf80cbc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:40.226 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.219+0000 7f3cf659c700 1 -- 172.21.15.67:0/1629680279 >> v1:172.21.15.67:6789/0 conn(0x7f3cf80cd2e0 legacy=0x7f3cf8071ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:40.226 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.219+0000 7f3cf5d9b700 1 -- 172.21.15.67:0/1629680279 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2904577587 0 0) 0x7f3cf80ce3b0 con 0x7f3cf80cd2e0 2023-10-14T10:36:40.226 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.219+0000 7f3cf659c700 1 -- 172.21.15.67:0/1629680279 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cf80cc220 con 0x7f3cf80b2940 2023-10-14T10:36:40.226 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.219+0000 7f3cf6d9d700 1 --2- 172.21.15.67:0/1629680279 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f3cf815a850 0x7f3cf80cbc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-14T10:36:40.226 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.219+0000 7f3cf659c700 1 --2- 172.21.15.67:0/1629680279 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3cf80b2940 0x7f3cf80ccda0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3ce8009840 tx=0x7f3ce8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a88331d9d88e69b server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:40.227 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.220+0000 7f3cf5d9b700 1 -- 172.21.15.67:0/1629680279 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ce8011830 con 0x7f3cf80b2940 2023-10-14T10:36:40.227 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.220+0000 7f3cf5d9b700 1 -- 172.21.15.67:0/1629680279 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ce8011990 con 0x7f3cf80b2940 2023-10-14T10:36:40.227 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.220+0000 7f3cf5d9b700 1 -- 172.21.15.67:0/1629680279 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ce8011d20 con 0x7f3cf80b2940 2023-10-14T10:36:40.227 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.220+0000 7f3cfd195700 1 -- 172.21.15.67:0/1629680279 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3cf80b2940 msgr2=0x7f3cf80ccda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:40.227 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.220+0000 7f3cfd195700 1 --2- 172.21.15.67:0/1629680279 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3cf80b2940 0x7f3cf80ccda0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3ce8009840 tx=0x7f3ce8009b50 comp rx=0 tx=0).stop 2023-10-14T10:36:40.228 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.220+0000 7f3cfd195700 1 -- 172.21.15.67:0/1629680279 shutdown_connections 2023-10-14T10:36:40.228 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.220+0000 7f3cfd195700 1 --2- 172.21.15.67:0/1629680279 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f3cf815a850 0x7f3cf80cbc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:40.228 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.220+0000 7f3cfd195700 1 --2- 172.21.15.67:0/1629680279 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3cf80b2940 0x7f3cf80ccda0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:40.228 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.220+0000 7f3cfd195700 1 -- 172.21.15.67:0/1629680279 >> 172.21.15.67:0/1629680279 conn(0x7f3cf806dac0 msgr2=0x7f3cf806dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:40.228 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.220+0000 7f3cfd195700 1 -- 172.21.15.67:0/1629680279 shutdown_connections 2023-10-14T10:36:40.229 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.220+0000 7f3cfd195700 1 -- 172.21.15.67:0/1629680279 wait complete. 2023-10-14T10:36:40.231 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.222+0000 7f3cfd195700 1 Processor -- start 2023-10-14T10:36:40.232 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.222+0000 7f3cfd195700 1 -- start start 2023-10-14T10:36:40.232 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.222+0000 7f3cfd195700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f3cf80b2940 0x7f3cf8157ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:40.232 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.222+0000 7f3cfd195700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3cf80cd2e0 0x7f3cf80aec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:40.232 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.222+0000 7f3cfd195700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3cf815a850 0x7f3cf80cec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:40.232 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.222+0000 7f3cfd195700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf81583e0 con 0x7f3cf815a850 2023-10-14T10:36:40.233 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.222+0000 7f3cfd195700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf80c64d0 con 0x7f3cf80cd2e0 2023-10-14T10:36:40.233 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.222+0000 7f3cfd195700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf80c67b0 con 0x7f3cf80b2940 2023-10-14T10:36:40.233 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.223+0000 7f3cf759e700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3cf815a850 0x7f3cf80cec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:40.233 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.223+0000 7f3cf759e700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3cf815a850 0x7f3cf80cec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45124/0 (socket says 172.21.15.67:45124) 2023-10-14T10:36:40.233 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.223+0000 7f3cf759e700 1 -- 172.21.15.67:0/3541591060 learned_addr learned my addr 172.21.15.67:0/3541591060 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:40.234 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.223+0000 7f3cf6d9d700 1 --2- 172.21.15.67:0/3541591060 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f3cf80b2940 0x7f3cf8157ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:40.234 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.223+0000 7f3cf659c700 1 --2- 172.21.15.67:0/3541591060 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3cf80cd2e0 0x7f3cf80aec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:40.234 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.223+0000 7f3cf759e700 1 -- 172.21.15.67:0/3541591060 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f3cf80b2940 msgr2=0x7f3cf8157ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:40.234 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.223+0000 7f3cf759e700 1 --2- 172.21.15.67:0/3541591060 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f3cf80b2940 0x7f3cf8157ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:40.234 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.223+0000 7f3cf759e700 1 -- 172.21.15.67:0/3541591060 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3cf80cd2e0 msgr2=0x7f3cf80aec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:40.235 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.223+0000 7f3cf759e700 1 --2- 172.21.15.67:0/3541591060 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3cf80cd2e0 0x7f3cf80aec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:40.235 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.223+0000 7f3cf759e700 1 -- 172.21.15.67:0/3541591060 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ce80094a0 con 0x7f3cf815a850 2023-10-14T10:36:40.235 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.223+0000 7f3cf6d9d700 1 --2- 172.21.15.67:0/3541591060 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f3cf80b2940 0x7f3cf8157ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-14T10:36:40.235 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.223+0000 7f3cf759e700 1 --2- 172.21.15.67:0/3541591060 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3cf815a850 0x7f3cf80cec90 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3cf000cba0 tx=0x7f3cf000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:40.235 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.223+0000 7f3cdf7fe700 1 -- 172.21.15.67:0/3541591060 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf000ab80 con 0x7f3cf815a850 2023-10-14T10:36:40.235 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.223+0000 7f3cdf7fe700 1 -- 172.21.15.67:0/3541591060 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cf000ad00 con 0x7f3cf815a850 2023-10-14T10:36:40.236 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.223+0000 7f3cfd195700 1 -- 172.21.15.67:0/3541591060 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cf80c6a90 con 0x7f3cf815a850 2023-10-14T10:36:40.236 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.223+0000 7f3cdf7fe700 1 -- 172.21.15.67:0/3541591060 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf0007310 con 0x7f3cf815a850 2023-10-14T10:36:40.236 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.223+0000 7f3cfd195700 1 -- 172.21.15.67:0/3541591060 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cf80c6d70 con 0x7f3cf815a850 2023-10-14T10:36:40.236 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.224+0000 7f3cfd195700 1 -- 172.21.15.67:0/3541591060 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cf8067d20 con 0x7f3cf815a850 2023-10-14T10:36:40.236 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.225+0000 7f3cdf7fe700 1 -- 172.21.15.67:0/3541591060 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f3cf0007470 con 0x7f3cf815a850 2023-10-14T10:36:40.237 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.225+0000 7f3cdf7fe700 1 --2- 172.21.15.67:0/3541591060 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f3ce406af00 0x7f3ce406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:40.237 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.225+0000 7f3cdf7fe700 1 -- 172.21.15.67:0/3541591060 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3cf0089430 con 0x7f3cf815a850 2023-10-14T10:36:40.237 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.225+0000 7f3cf6d9d700 1 --2- 172.21.15.67:0/3541591060 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f3ce406af00 0x7f3ce406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:40.237 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.226+0000 7f3cf6d9d700 1 --2- 172.21.15.67:0/3541591060 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f3ce406af00 0x7f3ce406d3c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3cec006fd0 tx=0x7f3cec008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:40.237 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.230+0000 7f3cdf7fe700 1 -- 172.21.15.67:0/3541591060 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3cf00544e0 con 0x7f3cf815a850 2023-10-14T10:36:40.447 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.445+0000 7f3cfd195700 1 -- 172.21.15.67:0/3541591060 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f3cf80b2150 con 0x7f3cf815a850 2023-10-14T10:36:40.448 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.446+0000 7f3cdf7fe700 1 -- 172.21.15.67:0/3541591060 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7f3cf0057b00 con 0x7f3cf815a850 2023-10-14T10:36:40.450 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.448+0000 7f3cfd195700 1 -- 172.21.15.67:0/3541591060 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f3ce406af00 msgr2=0x7f3ce406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:40.450 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.448+0000 7f3cfd195700 1 --2- 172.21.15.67:0/3541591060 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f3ce406af00 0x7f3ce406d3c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3cec006fd0 tx=0x7f3cec008040 comp rx=0 tx=0).stop 2023-10-14T10:36:40.450 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.448+0000 7f3cfd195700 1 -- 172.21.15.67:0/3541591060 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3cf815a850 msgr2=0x7f3cf80cec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:40.451 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.448+0000 7f3cfd195700 1 --2- 172.21.15.67:0/3541591060 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3cf815a850 0x7f3cf80cec90 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3cf000cba0 tx=0x7f3cf000cf60 comp rx=0 tx=0).stop 2023-10-14T10:36:40.451 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.448+0000 7f3cfd195700 1 -- 172.21.15.67:0/3541591060 shutdown_connections 2023-10-14T10:36:40.451 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.448+0000 7f3cfd195700 1 --2- 172.21.15.67:0/3541591060 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f3ce406af00 0x7f3ce406d3c0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:40.451 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.448+0000 7f3cfd195700 1 --2- 172.21.15.67:0/3541591060 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3cf815a850 0x7f3cf80cec90 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:40.451 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.448+0000 7f3cfd195700 1 --2- 172.21.15.67:0/3541591060 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f3cf80b2940 0x7f3cf8157ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:40.452 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.448+0000 7f3cfd195700 1 --2- 172.21.15.67:0/3541591060 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3cf80cd2e0 0x7f3cf80aec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:40.452 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.448+0000 7f3cfd195700 1 -- 172.21.15.67:0/3541591060 >> 172.21.15.67:0/3541591060 conn(0x7f3cf806dac0 msgr2=0x7f3cf80b3070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:40.452 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.448+0000 7f3cfd195700 1 -- 172.21.15.67:0/3541591060 shutdown_connections 2023-10-14T10:36:40.452 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.448+0000 7f3cfd195700 1 -- 172.21.15.67:0/3541591060 wait complete. 2023-10-14T10:36:40.453 INFO:teuthology.orchestra.run.smithi067.stderr:listed 0 entries 2023-10-14T10:36:40.462 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-14T10:36:40.462 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:36:40.463 DEBUG:teuthology.orchestra.run.smithi072:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2023-10-14T10:36:40.514 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi072.front.sepia.ceph.com 2023-10-14T10:36:40.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:36:40.515 DEBUG:teuthology.orchestra.run.smithi072:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-14T10:36:40.546 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2023-10-14T10:36:40.546 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-14T10:36:40.565 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2023-10-14T10:36:40.565 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-14T10:36:40.622 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2023-10-14T10:36:40.623 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-14T10:36:40.680 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-14T10:36:40.846 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.843+0000 7f38c43a3700 1 Processor -- start 2023-10-14T10:36:40.846 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.843+0000 7f38c43a3700 1 -- start start 2023-10-14T10:36:40.848 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.844+0000 7f38c43a3700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f38bc15e8d0 0x7f38bc0727b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:40.848 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.844+0000 7f38c43a3700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38bc071aa0 con 0x7f38bc15a580 2023-10-14T10:36:40.849 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.844+0000 7f38c43a3700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38bc15f140 con 0x7f38bc06d0c0 2023-10-14T10:36:40.849 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.844+0000 7f38c43a3700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38bc15ee60 con 0x7f38bc15e8d0 2023-10-14T10:36:40.849 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.844+0000 7f38c2940700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f38bc15e8d0 0x7f38bc0727b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:40.849 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.844+0000 7f38c2940700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f38bc15e8d0 0x7f38bc0727b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45132/0 (socket says 172.21.15.67:45132) 2023-10-14T10:36:40.849 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.845+0000 7f38c2940700 1 -- 172.21.15.67:0/2069754713 learned_addr learned my addr 172.21.15.67:0/2069754713 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:40.850 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.845+0000 7f38c213f700 1 --1- 172.21.15.67:0/2069754713 >> v1:172.21.15.67:6789/0 conn(0x7f38bc15a580 0x7f38bc06c9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:52896/0 (socket says 172.21.15.67:52896) 2023-10-14T10:36:40.850 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.845+0000 7f38c2940700 1 -- 172.21.15.67:0/2069754713 >> v1:172.21.15.72:6789/0 conn(0x7f38bc06d0c0 legacy=0x7f38bc15e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:40.850 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.845+0000 7f38c2940700 1 -- 172.21.15.67:0/2069754713 >> v1:172.21.15.67:6789/0 conn(0x7f38bc15a580 legacy=0x7f38bc06c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:40.850 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.845+0000 7f38c113d700 1 -- 172.21.15.67:0/2069754713 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4169992598 0 0) 0x7f38bc071aa0 con 0x7f38bc15a580 2023-10-14T10:36:40.850 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.845+0000 7f38c2940700 1 -- 172.21.15.67:0/2069754713 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38bc071cd0 con 0x7f38bc15e8d0 2023-10-14T10:36:40.851 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.845+0000 7f38c2940700 1 --2- 172.21.15.67:0/2069754713 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38bc15e8d0 0x7f38bc0727b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f38b4009e50 tx=0x7f38b400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26d81d43d17476c server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:40.851 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.845+0000 7f38c113d700 1 -- 172.21.15.67:0/2069754713 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38b400e070 con 0x7f38bc15e8d0 2023-10-14T10:36:40.851 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.845+0000 7f38c113d700 1 -- 172.21.15.67:0/2069754713 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38b400b9c0 con 0x7f38bc15e8d0 2023-10-14T10:36:40.851 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.845+0000 7f38c113d700 1 -- 172.21.15.67:0/2069754713 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38b400bd70 con 0x7f38bc15e8d0 2023-10-14T10:36:40.851 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.846+0000 7f38c43a3700 1 -- 172.21.15.67:0/2069754713 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38bc15e8d0 msgr2=0x7f38bc0727b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:40.852 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.846+0000 7f38c43a3700 1 --2- 172.21.15.67:0/2069754713 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38bc15e8d0 0x7f38bc0727b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f38b4009e50 tx=0x7f38b400b330 comp rx=0 tx=0).stop 2023-10-14T10:36:40.852 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.846+0000 7f38c43a3700 1 -- 172.21.15.67:0/2069754713 shutdown_connections 2023-10-14T10:36:40.852 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.846+0000 7f38c43a3700 1 --2- 172.21.15.67:0/2069754713 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38bc15e8d0 0x7f38bc0727b0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:40.852 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.846+0000 7f38c43a3700 1 -- 172.21.15.67:0/2069754713 >> 172.21.15.67:0/2069754713 conn(0x7f38bc06afb0 msgr2=0x7f38bc0cbe70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:40.852 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.846+0000 7f38c43a3700 1 -- 172.21.15.67:0/2069754713 shutdown_connections 2023-10-14T10:36:40.852 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.846+0000 7f38c43a3700 1 -- 172.21.15.67:0/2069754713 wait complete. 2023-10-14T10:36:40.855 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.847+0000 7f38c43a3700 1 Processor -- start 2023-10-14T10:36:40.855 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.847+0000 7f38c43a3700 1 -- start start 2023-10-14T10:36:40.856 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.848+0000 7f38c43a3700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f38bc06d0c0 0x7f38bc14d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:40.856 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.848+0000 7f38c43a3700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f38bc15a580 0x7f38bc0c5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:40.856 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.848+0000 7f38c43a3700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38bc15e8d0 0x7f38bc0b2a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:40.856 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.848+0000 7f38c43a3700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38bc0c4c00 con 0x7f38bc15e8d0 2023-10-14T10:36:40.857 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.848+0000 7f38c43a3700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38bc0c3400 con 0x7f38bc06d0c0 2023-10-14T10:36:40.857 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.848+0000 7f38c43a3700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38bc0c4e30 con 0x7f38bc15a580 2023-10-14T10:36:40.857 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.848+0000 7f38c2940700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38bc15e8d0 0x7f38bc0b2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:40.857 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.848+0000 7f38c2940700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38bc15e8d0 0x7f38bc0b2a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45138/0 (socket says 172.21.15.67:45138) 2023-10-14T10:36:40.857 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.848+0000 7f38c2940700 1 -- 172.21.15.67:0/3107352353 learned_addr learned my addr 172.21.15.67:0/3107352353 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:40.857 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.848+0000 7f38c213f700 1 --2- 172.21.15.67:0/3107352353 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f38bc06d0c0 0x7f38bc14d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:40.858 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.848+0000 7f38c193e700 1 --2- 172.21.15.67:0/3107352353 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f38bc15a580 0x7f38bc0c5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:40.858 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.848+0000 7f38c2940700 1 -- 172.21.15.67:0/3107352353 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f38bc15a580 msgr2=0x7f38bc0c5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:40.858 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.848+0000 7f38c2940700 1 --2- 172.21.15.67:0/3107352353 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f38bc15a580 0x7f38bc0c5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:40.858 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.848+0000 7f38c2940700 1 -- 172.21.15.67:0/3107352353 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f38bc06d0c0 msgr2=0x7f38bc14d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:40.859 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.848+0000 7f38c2940700 1 --2- 172.21.15.67:0/3107352353 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f38bc06d0c0 0x7f38bc14d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:40.859 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.848+0000 7f38c2940700 1 -- 172.21.15.67:0/3107352353 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38b4009b30 con 0x7f38bc15e8d0 2023-10-14T10:36:40.859 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.848+0000 7f38c2940700 1 --2- 172.21.15.67:0/3107352353 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38bc15e8d0 0x7f38bc0b2a00 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f38b400b730 tx=0x7f38b4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:40.859 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.849+0000 7f38b2ffd700 1 -- 172.21.15.67:0/3107352353 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38b401b630 con 0x7f38bc15e8d0 2023-10-14T10:36:40.859 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.849+0000 7f38b2ffd700 1 -- 172.21.15.67:0/3107352353 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38b401b7b0 con 0x7f38bc15e8d0 2023-10-14T10:36:40.860 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.849+0000 7f38b2ffd700 1 -- 172.21.15.67:0/3107352353 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38b401bb60 con 0x7f38bc15e8d0 2023-10-14T10:36:40.860 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.849+0000 7f38c43a3700 1 -- 172.21.15.67:0/3107352353 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38bc0c5110 con 0x7f38bc15e8d0 2023-10-14T10:36:40.860 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.849+0000 7f38c43a3700 1 -- 172.21.15.67:0/3107352353 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38bc14ddb0 con 0x7f38bc15e8d0 2023-10-14T10:36:40.860 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.850+0000 7f38c43a3700 1 -- 172.21.15.67:0/3107352353 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38bc0c5630 con 0x7f38bc15e8d0 2023-10-14T10:36:40.860 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.850+0000 7f38b2ffd700 1 -- 172.21.15.67:0/3107352353 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f38b401bcc0 con 0x7f38bc15e8d0 2023-10-14T10:36:40.861 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.851+0000 7f38b2ffd700 1 --2- 172.21.15.67:0/3107352353 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f38a006af00 0x7f38a006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:40.861 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.851+0000 7f38b2ffd700 1 -- 172.21.15.67:0/3107352353 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f38b408ab10 con 0x7f38bc15e8d0 2023-10-14T10:36:40.861 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.851+0000 7f38c213f700 1 --2- 172.21.15.67:0/3107352353 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f38a006af00 0x7f38a006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:40.861 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.851+0000 7f38c213f700 1 --2- 172.21.15.67:0/3107352353 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f38a006af00 0x7f38a006d3c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f38ac007b10 tx=0x7f38ac007a60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:40.862 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:40.855+0000 7f38b2ffd700 1 -- 172.21.15.67:0/3107352353 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f38bc0c5630 con 0x7f38bc15e8d0 2023-10-14T10:36:41.056 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.054+0000 7f38c43a3700 1 -- 172.21.15.67:0/3107352353 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f38bc0c5630 con 0x7f38bc15e8d0 2023-10-14T10:36:41.057 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.055+0000 7f38b2ffd700 1 -- 172.21.15.67:0/3107352353 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16457 (secure 0 0 0) 0x7f38bc0c5630 con 0x7f38bc15e8d0 2023-10-14T10:36:41.057 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-14T10:36:41.058 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":22,"fsid":"37f9ab46-3082-4b71-8b24-08d39897b008","created":"2023-10-14T10:35:46.067554+0000","modified":"2023-10-14T10:36:04.837296+0000","last_up_change":"2023-10-14T10:35:56.668142+0000","last_in_change":"2023-10-14T10:35:48.551490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-14T10:35:57.809566+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-14T10:36:01.667146+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-14T10:36:02.219700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2b59a07e-11ef-440c-b766-9f64ac885838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6801","nonce":1287255499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6803","nonce":1287255499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6807","nonce":1287255499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6805","nonce":1287255499}]},"public_addr":"172.21.15.67:6801/1287255499","cluster_addr":"172.21.15.67:6803/1287255499","heartbeat_back_addr":"172.21.15.67:6807/1287255499","heartbeat_front_addr":"172.21.15.67:6805/1287255499","state":["exists","up"]},{"osd":1,"uuid":"6f813bd6-fa0b-487d-a204-431eb1276a60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6818","nonce":2435538787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6822","nonce":2435538787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6830","nonce":2435538787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6826","nonce":2435538787}]},"public_addr":"172.21.15.67:6818/2435538787","cluster_addr":"172.21.15.67:6822/2435538787","heartbeat_back_addr":"172.21.15.67:6830/2435538787","heartbeat_front_addr":"172.21.15.67:6826/2435538787","state":["exists","up"]},{"osd":2,"uuid":"6af42c6a-1b59-4669-9c04-b6876788446d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6809","nonce":2827491791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6811","nonce":2827491791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6815","nonce":2827491791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6813","nonce":2827491791}]},"public_addr":"172.21.15.67:6809/2827491791","cluster_addr":"172.21.15.67:6811/2827491791","heartbeat_back_addr":"172.21.15.67:6815/2827491791","heartbeat_front_addr":"172.21.15.67:6813/2827491791","state":["exists","up"]},{"osd":3,"uuid":"1d889d76-a646-499d-bfff-ed5bae5434ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6819","nonce":1848001176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6821","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6823","nonce":1848001176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6831","nonce":1848001176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6825","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6827","nonce":1848001176}]},"public_addr":"172.21.15.67:6819/1848001176","cluster_addr":"172.21.15.67:6823/1848001176","heartbeat_back_addr":"172.21.15.67:6831/1848001176","heartbeat_front_addr":"172.21.15.67:6827/1848001176","state":["exists","up"]},{"osd":4,"uuid":"c3271dff-b437-4fdd-b13b-6e49a35473af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6804","nonce":1891230785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6808","nonce":1891230785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6815","nonce":1891230785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6811","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6813","nonce":1891230785}]},"public_addr":"172.21.15.72:6804/1891230785","cluster_addr":"172.21.15.72:6808/1891230785","heartbeat_back_addr":"172.21.15.72:6815/1891230785","heartbeat_front_addr":"172.21.15.72:6813/1891230785","state":["exists","up"]},{"osd":5,"uuid":"0337bb18-4b68-4eff-b528-a667275e6926","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6801","nonce":569686991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6803","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6805","nonce":569686991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6812","nonce":569686991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6807","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6809","nonce":569686991}]},"public_addr":"172.21.15.72:6801/569686991","cluster_addr":"172.21.15.72:6805/569686991","heartbeat_back_addr":"172.21.15.72:6812/569686991","heartbeat_front_addr":"172.21.15.72:6809/569686991","state":["exists","up"]},{"osd":6,"uuid":"8e92d9cf-0c8c-46eb-89a7-db2023ef1509","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6817","nonce":3767685934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6819","nonce":3767685934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6823","nonce":3767685934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6821","nonce":3767685934}]},"public_addr":"172.21.15.72:6817/3767685934","cluster_addr":"172.21.15.72:6819/3767685934","heartbeat_back_addr":"172.21.15.72:6823/3767685934","heartbeat_front_addr":"172.21.15.72:6821/3767685934","state":["exists","up"]},{"osd":7,"uuid":"3fd6aa7d-50a9-45c9-9561-5902a044a119","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6825","nonce":3567465926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6827","nonce":3567465926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6831","nonce":3567465926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6829","nonce":3567465926}]},"public_addr":"172.21.15.72:6825/3567465926","cluster_addr":"172.21.15.72:6827/3567465926","heartbeat_back_addr":"172.21.15.72:6831/3567465926","heartbeat_front_addr":"172.21.15.72:6829/3567465926","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.511466+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.795107+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.858674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.728895+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.794842+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.750064+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.732122+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.765185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-14T10:36:41.060 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.057+0000 7f38c43a3700 1 -- 172.21.15.67:0/3107352353 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f38a006af00 msgr2=0x7f38a006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:41.060 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.057+0000 7f38c43a3700 1 --2- 172.21.15.67:0/3107352353 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f38a006af00 0x7f38a006d3c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f38ac007b10 tx=0x7f38ac007a60 comp rx=0 tx=0).stop 2023-10-14T10:36:41.060 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.057+0000 7f38c43a3700 1 -- 172.21.15.67:0/3107352353 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38bc15e8d0 msgr2=0x7f38bc0b2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:41.060 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.057+0000 7f38c43a3700 1 --2- 172.21.15.67:0/3107352353 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38bc15e8d0 0x7f38bc0b2a00 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f38b400b730 tx=0x7f38b4004ad0 comp rx=0 tx=0).stop 2023-10-14T10:36:41.061 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.057+0000 7f38c43a3700 1 -- 172.21.15.67:0/3107352353 shutdown_connections 2023-10-14T10:36:41.061 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.057+0000 7f38c43a3700 1 --2- 172.21.15.67:0/3107352353 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f38a006af00 0x7f38a006d3c0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:41.061 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.057+0000 7f38c43a3700 1 --2- 172.21.15.67:0/3107352353 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38bc15e8d0 0x7f38bc0b2a00 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:41.061 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.057+0000 7f38c43a3700 1 --2- 172.21.15.67:0/3107352353 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f38bc06d0c0 0x7f38bc14d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:41.061 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.057+0000 7f38c43a3700 1 --2- 172.21.15.67:0/3107352353 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f38bc15a580 0x7f38bc0c5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:41.062 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.057+0000 7f38c43a3700 1 -- 172.21.15.67:0/3107352353 >> 172.21.15.67:0/3107352353 conn(0x7f38bc06afb0 msgr2=0x7f38bc0b14d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:41.062 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.057+0000 7f38c43a3700 1 -- 172.21.15.67:0/3107352353 shutdown_connections 2023-10-14T10:36:41.062 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.057+0000 7f38c43a3700 1 -- 172.21.15.67:0/3107352353 wait complete. 2023-10-14T10:36:41.072 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-14T10:35:57.809566+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-14T10:36:01.667146+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-14T10:36:02.219700+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-14T10:36:41.073 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-14T10:36:41.201 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.199+0000 7f35a68d4700 1 Processor -- start 2023-10-14T10:36:41.202 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.199+0000 7f35a68d4700 1 -- start start 2023-10-14T10:36:41.204 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.200+0000 7f35a68d4700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f35a00cd950 0x7f35a015e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:41.204 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.200+0000 7f35a68d4700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f35a006a840 0x7f35a006acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:41.204 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.200+0000 7f35a68d4700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35a00cf260 con 0x7f35a00cede0 2023-10-14T10:36:41.205 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.200+0000 7f35a68d4700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35a01619a0 con 0x7f35a006a840 2023-10-14T10:36:41.205 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.200+0000 7f35a68d4700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35a006b1f0 con 0x7f35a00cd950 2023-10-14T10:36:41.205 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.200+0000 7f35a4e71700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f35a006a840 0x7f35a006acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:41.205 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.200+0000 7f35a4e71700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f35a006a840 0x7f35a006acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45146/0 (socket says 172.21.15.67:45146) 2023-10-14T10:36:41.205 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.200+0000 7f35a4e71700 1 -- 172.21.15.67:0/2402930702 learned_addr learned my addr 172.21.15.67:0/2402930702 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:41.206 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.200+0000 7f359ffff700 1 --2- 172.21.15.67:0/2402930702 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f35a00cd950 0x7f35a015e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:41.206 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.200+0000 7f35a4e71700 1 -- 172.21.15.67:0/2402930702 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f35a00cd950 msgr2=0x7f35a015e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:41.206 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.200+0000 7f35a4e71700 1 --2- 172.21.15.67:0/2402930702 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f35a00cd950 0x7f35a015e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:41.206 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.200+0000 7f35a4e71700 1 -- 172.21.15.67:0/2402930702 >> v1:172.21.15.72:6789/0 conn(0x7f35a00cede0 legacy=0x7f35a015f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:41.206 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.200+0000 7f35a4e71700 1 -- 172.21.15.67:0/2402930702 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35a00b2d60 con 0x7f35a006a840 2023-10-14T10:36:41.207 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.200+0000 7f359ffff700 1 --2- 172.21.15.67:0/2402930702 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f35a00cd950 0x7f35a015e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-14T10:36:41.207 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.200+0000 7f35a4e71700 1 --2- 172.21.15.67:0/2402930702 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35a006a840 0x7f35a006acb0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f3598009e50 tx=0x7f359800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d52e66245de632 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:41.207 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.200+0000 7f359effd700 1 -- 172.21.15.67:0/2402930702 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f359800e070 con 0x7f35a006a840 2023-10-14T10:36:41.207 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.200+0000 7f359effd700 1 -- 172.21.15.67:0/2402930702 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f359800b9c0 con 0x7f35a006a840 2023-10-14T10:36:41.207 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.200+0000 7f359effd700 1 -- 172.21.15.67:0/2402930702 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f359800bd70 con 0x7f35a006a840 2023-10-14T10:36:41.207 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.201+0000 7f35a68d4700 1 -- 172.21.15.67:0/2402930702 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35a006a840 msgr2=0x7f35a006acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:41.208 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.201+0000 7f35a68d4700 1 --2- 172.21.15.67:0/2402930702 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35a006a840 0x7f35a006acb0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f3598009e50 tx=0x7f359800b330 comp rx=0 tx=0).stop 2023-10-14T10:36:41.208 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.201+0000 7f35a68d4700 1 -- 172.21.15.67:0/2402930702 shutdown_connections 2023-10-14T10:36:41.208 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.201+0000 7f35a68d4700 1 --2- 172.21.15.67:0/2402930702 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35a006a840 0x7f35a006acb0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:41.208 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.201+0000 7f35a68d4700 1 --2- 172.21.15.67:0/2402930702 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f35a00cd950 0x7f35a015e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:41.209 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.201+0000 7f35a68d4700 1 -- 172.21.15.67:0/2402930702 >> 172.21.15.67:0/2402930702 conn(0x7f35a006dae0 msgr2=0x7f35a006def0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:41.209 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.201+0000 7f35a68d4700 1 -- 172.21.15.67:0/2402930702 shutdown_connections 2023-10-14T10:36:41.209 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.201+0000 7f35a68d4700 1 -- 172.21.15.67:0/2402930702 wait complete. 2023-10-14T10:36:41.215 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.203+0000 7f35a68d4700 1 Processor -- start 2023-10-14T10:36:41.215 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.203+0000 7f35a68d4700 1 -- start start 2023-10-14T10:36:41.216 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.204+0000 7f35a68d4700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f35a006a840 0x7f35a00ca690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:41.216 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.204+0000 7f35a68d4700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35a00cd950 0x7f35a015b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:41.216 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.204+0000 7f35a68d4700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f35a00cede0 0x7f35a00cbec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:41.217 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.204+0000 7f35a68d4700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35a00c6b50 con 0x7f35a00cd950 2023-10-14T10:36:41.217 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.204+0000 7f35a68d4700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35a00c4490 con 0x7f35a00cede0 2023-10-14T10:36:41.217 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.204+0000 7f35a68d4700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35a00c6d80 con 0x7f35a006a840 2023-10-14T10:36:41.218 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.204+0000 7f359f7fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35a00cd950 0x7f35a015b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:41.218 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.204+0000 7f359f7fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35a00cd950 0x7f35a015b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45150/0 (socket says 172.21.15.67:45150) 2023-10-14T10:36:41.219 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.204+0000 7f359f7fe700 1 -- 172.21.15.67:0/3437736985 learned_addr learned my addr 172.21.15.67:0/3437736985 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:41.219 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.204+0000 7f359ffff700 1 --2- 172.21.15.67:0/3437736985 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f35a006a840 0x7f35a00ca690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:41.219 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.204+0000 7f35a4e71700 1 --2- 172.21.15.67:0/3437736985 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f35a00cede0 0x7f35a00cbec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:41.220 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.204+0000 7f359f7fe700 1 -- 172.21.15.67:0/3437736985 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f35a006a840 msgr2=0x7f35a00ca690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:41.220 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.204+0000 7f359f7fe700 1 --2- 172.21.15.67:0/3437736985 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f35a006a840 0x7f35a00ca690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:41.220 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.204+0000 7f359f7fe700 1 -- 172.21.15.67:0/3437736985 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f35a00cede0 msgr2=0x7f35a00cbec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:41.220 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.204+0000 7f359f7fe700 1 --2- 172.21.15.67:0/3437736985 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f35a00cede0 0x7f35a00cbec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:41.221 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.204+0000 7f359f7fe700 1 -- 172.21.15.67:0/3437736985 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3598009b30 con 0x7f35a00cd950 2023-10-14T10:36:41.221 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.204+0000 7f359f7fe700 1 --2- 172.21.15.67:0/3437736985 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35a00cd950 0x7f35a015b3a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f358800c4d0 tx=0x7f358800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:41.221 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.205+0000 7f359cff9700 1 -- 172.21.15.67:0/3437736985 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35880052d0 con 0x7f35a00cd950 2023-10-14T10:36:41.221 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.205+0000 7f359cff9700 1 -- 172.21.15.67:0/3437736985 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3588005430 con 0x7f35a00cd950 2023-10-14T10:36:41.221 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.205+0000 7f35a68d4700 1 -- 172.21.15.67:0/3437736985 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35a00c7060 con 0x7f35a00cd950 2023-10-14T10:36:41.221 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.205+0000 7f359cff9700 1 -- 172.21.15.67:0/3437736985 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35880038f0 con 0x7f35a00cd950 2023-10-14T10:36:41.222 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.205+0000 7f35a68d4700 1 -- 172.21.15.67:0/3437736985 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35a00b4520 con 0x7f35a00cd950 2023-10-14T10:36:41.222 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.206+0000 7f35a68d4700 1 -- 172.21.15.67:0/3437736985 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35a0067d20 con 0x7f35a00cd950 2023-10-14T10:36:41.222 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.206+0000 7f359cff9700 1 -- 172.21.15.67:0/3437736985 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f3588003a50 con 0x7f35a00cd950 2023-10-14T10:36:41.222 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.206+0000 7f359cff9700 1 --2- 172.21.15.67:0/3437736985 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f358006af00 0x7f358006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:41.223 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.207+0000 7f359cff9700 1 -- 172.21.15.67:0/3437736985 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3588088ad0 con 0x7f35a00cd950 2023-10-14T10:36:41.223 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.207+0000 7f359ffff700 1 --2- 172.21.15.67:0/3437736985 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f358006af00 0x7f358006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:41.223 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.207+0000 7f359ffff700 1 --2- 172.21.15.67:0/3437736985 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f358006af00 0x7f358006d3c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3590009910 tx=0x7f3590008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:41.223 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.211+0000 7f359cff9700 1 -- 172.21.15.67:0/3437736985 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3588053af0 con 0x7f35a00cd950 2023-10-14T10:36:41.417 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.415+0000 7f35a68d4700 1 -- 172.21.15.67:0/3437736985 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f35a00b2100 con 0x7f35a00cd950 2023-10-14T10:36:41.417 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.415+0000 7f359cff9700 1 -- 172.21.15.67:0/3437736985 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3588017020 con 0x7f35a00cd950 2023-10-14T10:36:41.418 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2023-10-14T10:36:41.419 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.417+0000 7f35a68d4700 1 -- 172.21.15.67:0/3437736985 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f358006af00 msgr2=0x7f358006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:41.420 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.417+0000 7f35a68d4700 1 --2- 172.21.15.67:0/3437736985 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f358006af00 0x7f358006d3c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3590009910 tx=0x7f3590008040 comp rx=0 tx=0).stop 2023-10-14T10:36:41.420 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.417+0000 7f35a68d4700 1 -- 172.21.15.67:0/3437736985 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35a00cd950 msgr2=0x7f35a015b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:41.420 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.417+0000 7f35a68d4700 1 --2- 172.21.15.67:0/3437736985 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35a00cd950 0x7f35a015b3a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f358800c4d0 tx=0x7f358800c890 comp rx=0 tx=0).stop 2023-10-14T10:36:41.420 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.417+0000 7f35a68d4700 1 -- 172.21.15.67:0/3437736985 shutdown_connections 2023-10-14T10:36:41.420 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.417+0000 7f35a68d4700 1 --2- 172.21.15.67:0/3437736985 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f358006af00 0x7f358006d3c0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:41.421 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.417+0000 7f35a68d4700 1 --2- 172.21.15.67:0/3437736985 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f35a00cede0 0x7f35a00cbec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:41.421 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.417+0000 7f35a68d4700 1 --2- 172.21.15.67:0/3437736985 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f35a006a840 0x7f35a00ca690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:41.421 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.417+0000 7f35a68d4700 1 --2- 172.21.15.67:0/3437736985 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35a00cd950 0x7f35a015b3a0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:41.421 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.417+0000 7f35a68d4700 1 -- 172.21.15.67:0/3437736985 >> 172.21.15.67:0/3437736985 conn(0x7f35a006dae0 msgr2=0x7f35a00b0f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:41.421 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.417+0000 7f35a68d4700 1 -- 172.21.15.67:0/3437736985 shutdown_connections 2023-10-14T10:36:41.422 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.417+0000 7f35a68d4700 1 -- 172.21.15.67:0/3437736985 wait complete. 2023-10-14T10:36:41.432 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-14T10:36:41.560 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.558+0000 7f64cf6b9700 1 Processor -- start 2023-10-14T10:36:41.560 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.558+0000 7f64cf6b9700 1 -- start start 2023-10-14T10:36:41.563 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.559+0000 7f64cf6b9700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f64c815b740 0x7f64c80d1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:41.563 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.559+0000 7f64cf6b9700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f64c80d0c70 0x7f64c80cdfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:41.563 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.559+0000 7f64cf6b9700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64c80cf190 con 0x7f64c80c5110 2023-10-14T10:36:41.563 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.559+0000 7f64cf6b9700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64c815bb20 con 0x7f64c80d0c70 2023-10-14T10:36:41.564 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.559+0000 7f64cf6b9700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64c80cbd40 con 0x7f64c815b740 2023-10-14T10:36:41.564 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.559+0000 7f64ccc54700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f64c80d0c70 0x7f64c80cdfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:41.564 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.559+0000 7f64ccc54700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f64c80d0c70 0x7f64c80cdfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45158/0 (socket says 172.21.15.67:45158) 2023-10-14T10:36:41.564 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.559+0000 7f64cdc56700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f64c80c5110 0x7f64c80cf690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:52928/0 (socket says 172.21.15.67:52928) 2023-10-14T10:36:41.564 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.559+0000 7f64ccc54700 1 -- 172.21.15.67:0/584172586 learned_addr learned my addr 172.21.15.67:0/584172586 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:41.565 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.559+0000 7f64cd455700 1 --2- 172.21.15.67:0/584172586 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f64c815b740 0x7f64c80d1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:41.565 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.559+0000 7f64ccc54700 1 -- 172.21.15.67:0/584172586 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f64c815b740 msgr2=0x7f64c80d1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:41.565 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.559+0000 7f64ccc54700 1 --2- 172.21.15.67:0/584172586 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f64c815b740 0x7f64c80d1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:41.565 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.559+0000 7f64ccc54700 1 -- 172.21.15.67:0/584172586 >> v1:172.21.15.67:6789/0 conn(0x7f64c80c5110 legacy=0x7f64c80cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:41.565 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.559+0000 7f64bffff700 1 -- 172.21.15.67:0/584172586 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3731826618 0 0) 0x7f64c80cf190 con 0x7f64c80c5110 2023-10-14T10:36:41.566 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.559+0000 7f64ccc54700 1 -- 172.21.15.67:0/584172586 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64c80cc020 con 0x7f64c80d0c70 2023-10-14T10:36:41.566 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.559+0000 7f64cd455700 1 --2- 172.21.15.67:0/584172586 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f64c815b740 0x7f64c80d1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-14T10:36:41.566 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.559+0000 7f64ccc54700 1 --2- 172.21.15.67:0/584172586 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f64c80d0c70 0x7f64c80cdfd0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f64b80098f0 tx=0x7f64b8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca653a264e07a114 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:41.566 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.560+0000 7f64bffff700 1 -- 172.21.15.67:0/584172586 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b800b8a0 con 0x7f64c80d0c70 2023-10-14T10:36:41.567 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.560+0000 7f64bffff700 1 -- 172.21.15.67:0/584172586 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64b800ba00 con 0x7f64c80d0c70 2023-10-14T10:36:41.567 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.560+0000 7f64bffff700 1 -- 172.21.15.67:0/584172586 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b800bd90 con 0x7f64c80d0c70 2023-10-14T10:36:41.567 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.560+0000 7f64cf6b9700 1 -- 172.21.15.67:0/584172586 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f64c80d0c70 msgr2=0x7f64c80cdfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:41.567 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.560+0000 7f64cf6b9700 1 --2- 172.21.15.67:0/584172586 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f64c80d0c70 0x7f64c80cdfd0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f64b80098f0 tx=0x7f64b8009c00 comp rx=0 tx=0).stop 2023-10-14T10:36:41.567 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.560+0000 7f64cf6b9700 1 -- 172.21.15.67:0/584172586 shutdown_connections 2023-10-14T10:36:41.568 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.560+0000 7f64cf6b9700 1 --2- 172.21.15.67:0/584172586 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f64c815b740 0x7f64c80d1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:41.568 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.560+0000 7f64cf6b9700 1 --2- 172.21.15.67:0/584172586 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f64c80d0c70 0x7f64c80cdfd0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:41.568 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.560+0000 7f64cf6b9700 1 -- 172.21.15.67:0/584172586 >> 172.21.15.67:0/584172586 conn(0x7f64c80665a0 msgr2=0x7f64c8066980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:41.568 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.560+0000 7f64cf6b9700 1 -- 172.21.15.67:0/584172586 shutdown_connections 2023-10-14T10:36:41.568 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.560+0000 7f64cf6b9700 1 -- 172.21.15.67:0/584172586 wait complete. 2023-10-14T10:36:41.572 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.561+0000 7f64cf6b9700 1 Processor -- start 2023-10-14T10:36:41.572 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.561+0000 7f64cf6b9700 1 -- start start 2023-10-14T10:36:41.572 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.562+0000 7f64cf6b9700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f64c80c5110 0x7f64c8158780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:41.572 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.562+0000 7f64cf6b9700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f64c80d0c70 0x7f64c81f61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:41.573 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.562+0000 7f64cf6b9700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f64c815b740 0x7f64c81f6710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:41.573 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.562+0000 7f64cf6b9700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64c815d8f0 con 0x7f64c815b740 2023-10-14T10:36:41.573 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.562+0000 7f64cf6b9700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64c8158cc0 con 0x7f64c80c5110 2023-10-14T10:36:41.573 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.562+0000 7f64cf6b9700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64c81e7800 con 0x7f64c80d0c70 2023-10-14T10:36:41.573 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.562+0000 7f64cdc56700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f64c815b740 0x7f64c81f6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:41.573 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.562+0000 7f64cdc56700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f64c815b740 0x7f64c81f6710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45166/0 (socket says 172.21.15.67:45166) 2023-10-14T10:36:41.574 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.562+0000 7f64cdc56700 1 -- 172.21.15.67:0/1659572146 learned_addr learned my addr 172.21.15.67:0/1659572146 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:41.574 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.562+0000 7f64cd455700 1 --2- 172.21.15.67:0/1659572146 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f64c80c5110 0x7f64c8158780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:41.574 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.563+0000 7f64ccc54700 1 --2- 172.21.15.67:0/1659572146 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f64c80d0c70 0x7f64c81f61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:41.574 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.563+0000 7f64cdc56700 1 -- 172.21.15.67:0/1659572146 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f64c80d0c70 msgr2=0x7f64c81f61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:41.574 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.563+0000 7f64cdc56700 1 --2- 172.21.15.67:0/1659572146 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f64c80d0c70 0x7f64c81f61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:41.575 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.563+0000 7f64cdc56700 1 -- 172.21.15.67:0/1659572146 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f64c80c5110 msgr2=0x7f64c8158780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:41.575 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.563+0000 7f64cdc56700 1 --2- 172.21.15.67:0/1659572146 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f64c80c5110 0x7f64c8158780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:41.575 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.563+0000 7f64cdc56700 1 -- 172.21.15.67:0/1659572146 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64b8009550 con 0x7f64c815b740 2023-10-14T10:36:41.575 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.563+0000 7f64cdc56700 1 --2- 172.21.15.67:0/1659572146 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f64c815b740 0x7f64c81f6710 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f64c000cc70 tx=0x7f64c000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:41.575 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.563+0000 7f64bdffb700 1 -- 172.21.15.67:0/1659572146 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c000ad80 con 0x7f64c815b740 2023-10-14T10:36:41.576 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.563+0000 7f64bdffb700 1 -- 172.21.15.67:0/1659572146 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64c000ce70 con 0x7f64c815b740 2023-10-14T10:36:41.576 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.563+0000 7f64cf6b9700 1 -- 172.21.15.67:0/1659572146 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64c815dba0 con 0x7f64c815b740 2023-10-14T10:36:41.576 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.563+0000 7f64bdffb700 1 -- 172.21.15.67:0/1659572146 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c00072f0 con 0x7f64c815b740 2023-10-14T10:36:41.576 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.563+0000 7f64cf6b9700 1 -- 172.21.15.67:0/1659572146 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64c815e160 con 0x7f64c815b740 2023-10-14T10:36:41.576 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.564+0000 7f64cf6b9700 1 -- 172.21.15.67:0/1659572146 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64c815e350 con 0x7f64c815b740 2023-10-14T10:36:41.577 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.564+0000 7f64bdffb700 1 -- 172.21.15.67:0/1659572146 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f64c0007450 con 0x7f64c815b740 2023-10-14T10:36:41.577 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.565+0000 7f64bdffb700 1 --2- 172.21.15.67:0/1659572146 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f64ac06af00 0x7f64ac06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:41.577 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.565+0000 7f64bdffb700 1 -- 172.21.15.67:0/1659572146 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f64c0089510 con 0x7f64c815b740 2023-10-14T10:36:41.577 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.565+0000 7f64cd455700 1 --2- 172.21.15.67:0/1659572146 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f64ac06af00 0x7f64ac06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:41.577 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.565+0000 7f64cd455700 1 --2- 172.21.15.67:0/1659572146 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f64ac06af00 0x7f64ac06d3c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f64c4006fd0 tx=0x7f64c4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:41.578 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.569+0000 7f64bdffb700 1 -- 172.21.15.67:0/1659572146 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f64c00545c0 con 0x7f64c815b740 2023-10-14T10:36:41.772 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.770+0000 7f64cf6b9700 1 -- 172.21.15.67:0/1659572146 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f64c8067d20 con 0x7f64c815b740 2023-10-14T10:36:41.773 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.770+0000 7f64bdffb700 1 -- 172.21.15.67:0/1659572146 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f64c0087e60 con 0x7f64c815b740 2023-10-14T10:36:41.773 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2023-10-14T10:36:41.775 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.772+0000 7f64cf6b9700 1 -- 172.21.15.67:0/1659572146 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f64ac06af00 msgr2=0x7f64ac06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:41.775 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.772+0000 7f64cf6b9700 1 --2- 172.21.15.67:0/1659572146 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f64ac06af00 0x7f64ac06d3c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f64c4006fd0 tx=0x7f64c4008040 comp rx=0 tx=0).stop 2023-10-14T10:36:41.775 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.772+0000 7f64cf6b9700 1 -- 172.21.15.67:0/1659572146 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f64c815b740 msgr2=0x7f64c81f6710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:41.776 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.772+0000 7f64cf6b9700 1 --2- 172.21.15.67:0/1659572146 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f64c815b740 0x7f64c81f6710 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f64c000cc70 tx=0x7f64c000a340 comp rx=0 tx=0).stop 2023-10-14T10:36:41.776 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.773+0000 7f64cf6b9700 1 -- 172.21.15.67:0/1659572146 shutdown_connections 2023-10-14T10:36:41.776 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.773+0000 7f64cf6b9700 1 --2- 172.21.15.67:0/1659572146 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f64ac06af00 0x7f64ac06d3c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:41.776 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.773+0000 7f64cf6b9700 1 --2- 172.21.15.67:0/1659572146 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f64c815b740 0x7f64c81f6710 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:41.776 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.773+0000 7f64cf6b9700 1 --2- 172.21.15.67:0/1659572146 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f64c80c5110 0x7f64c8158780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:41.777 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.773+0000 7f64cf6b9700 1 --2- 172.21.15.67:0/1659572146 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f64c80d0c70 0x7f64c81f61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:41.777 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.773+0000 7f64cf6b9700 1 -- 172.21.15.67:0/1659572146 >> 172.21.15.67:0/1659572146 conn(0x7f64c80665a0 msgr2=0x7f64c80cf690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:41.777 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.773+0000 7f64cf6b9700 1 -- 172.21.15.67:0/1659572146 shutdown_connections 2023-10-14T10:36:41.777 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.773+0000 7f64cf6b9700 1 -- 172.21.15.67:0/1659572146 wait complete. 2023-10-14T10:36:41.787 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-14T10:36:41.916 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.913+0000 7f0ba3b15700 1 Processor -- start 2023-10-14T10:36:41.916 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.913+0000 7f0ba3b15700 1 -- start start 2023-10-14T10:36:41.918 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.914+0000 7f0ba3b15700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f0b9c0cd950 0x7f0b9c15e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:41.918 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.914+0000 7f0ba3b15700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f0b9c0cede0 0x7f0b9c15f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:41.918 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.914+0000 7f0ba3b15700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f0b9c06a840 0x7f0b9c06acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:41.919 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.914+0000 7f0ba3b15700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b9c1619a0 con 0x7f0b9c0cede0 2023-10-14T10:36:41.919 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.914+0000 7f0ba3b15700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b9c06b1f0 con 0x7f0b9c06a840 2023-10-14T10:36:41.919 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.914+0000 7f0ba3b15700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9c0cf260 con 0x7f0b9c0cd950 2023-10-14T10:36:41.919 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.914+0000 7f0ba10b0700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f0b9c0cede0 0x7f0b9c15f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:41.919 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.914+0000 7f0ba10b0700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f0b9c0cede0 0x7f0b9c15f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45172/0 (socket says 172.21.15.67:45172) 2023-10-14T10:36:41.920 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.914+0000 7f0ba10b0700 1 -- 172.21.15.67:0/321536131 learned_addr learned my addr 172.21.15.67:0/321536131 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:41.920 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.915+0000 7f0ba18b1700 1 --2- 172.21.15.67:0/321536131 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f0b9c0cd950 0x7f0b9c15e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:41.920 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.915+0000 7f0ba20b2700 1 --2- 172.21.15.67:0/321536131 >> v2:172.21.15.72:3300/0 conn(0x7f0b9c06a840 0x7f0b9c06acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:41.920 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.915+0000 7f0ba10b0700 1 -- 172.21.15.67:0/321536131 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f0b9c0cd950 msgr2=0x7f0b9c15e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:41.920 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.915+0000 7f0ba10b0700 1 --2- 172.21.15.67:0/321536131 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f0b9c0cd950 0x7f0b9c15e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:41.921 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.915+0000 7f0ba10b0700 1 -- 172.21.15.67:0/321536131 >> v2:172.21.15.72:3300/0 conn(0x7f0b9c06a840 msgr2=0x7f0b9c06acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:41.921 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.915+0000 7f0ba10b0700 1 --2- 172.21.15.67:0/321536131 >> v2:172.21.15.72:3300/0 conn(0x7f0b9c06a840 0x7f0b9c06acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:41.921 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.915+0000 7f0ba10b0700 1 -- 172.21.15.67:0/321536131 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b9c0b2d60 con 0x7f0b9c0cede0 2023-10-14T10:36:41.921 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.915+0000 7f0ba10b0700 1 --2- 172.21.15.67:0/321536131 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b9c0cede0 0x7f0b9c15f570 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c06ebc0 tx=0x7f0b98009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eebb6682e7bac15 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:41.921 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.915+0000 7f0ba08af700 1 -- 172.21.15.67:0/321536131 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b98005e30 con 0x7f0b9c0cede0 2023-10-14T10:36:41.922 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.915+0000 7f0ba08af700 1 -- 172.21.15.67:0/321536131 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b98004030 con 0x7f0b9c0cede0 2023-10-14T10:36:41.922 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.915+0000 7f0ba08af700 1 -- 172.21.15.67:0/321536131 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b980043e0 con 0x7f0b9c0cede0 2023-10-14T10:36:41.922 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.916+0000 7f0ba3b15700 1 -- 172.21.15.67:0/321536131 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b9c0cede0 msgr2=0x7f0b9c15f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:41.922 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.916+0000 7f0ba3b15700 1 --2- 172.21.15.67:0/321536131 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b9c0cede0 0x7f0b9c15f570 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c06ebc0 tx=0x7f0b98009bd0 comp rx=0 tx=0).stop 2023-10-14T10:36:41.922 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.916+0000 7f0ba3b15700 1 -- 172.21.15.67:0/321536131 shutdown_connections 2023-10-14T10:36:41.923 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.916+0000 7f0ba3b15700 1 --2- 172.21.15.67:0/321536131 >> v2:172.21.15.72:3300/0 conn(0x7f0b9c06a840 0x7f0b9c06acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:41.923 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.916+0000 7f0ba3b15700 1 --2- 172.21.15.67:0/321536131 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f0b9c0cd950 0x7f0b9c15e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:41.923 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.916+0000 7f0ba3b15700 1 --2- 172.21.15.67:0/321536131 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b9c0cede0 0x7f0b9c15f570 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:41.923 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.916+0000 7f0ba3b15700 1 -- 172.21.15.67:0/321536131 >> 172.21.15.67:0/321536131 conn(0x7f0b9c06dae0 msgr2=0x7f0b9c06def0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:41.923 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.916+0000 7f0ba3b15700 1 -- 172.21.15.67:0/321536131 shutdown_connections 2023-10-14T10:36:41.924 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.916+0000 7f0ba3b15700 1 -- 172.21.15.67:0/321536131 wait complete. 2023-10-14T10:36:41.926 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.917+0000 7f0ba3b15700 1 Processor -- start 2023-10-14T10:36:41.926 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.917+0000 7f0ba3b15700 1 -- start start 2023-10-14T10:36:41.927 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.918+0000 7f0ba3b15700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0b9c06a840 0x7f0b9c0c9260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:41.927 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.918+0000 7f0ba3b15700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b9c0cd950 0x7f0b9c0c97a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:41.927 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.918+0000 7f0ba3b15700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f0b9c0cede0 0x7f0b9c1f36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:41.927 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.918+0000 7f0ba3b15700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9c1f4330 con 0x7f0b9c0cd950 2023-10-14T10:36:41.927 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.918+0000 7f0ba3b15700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9c1f4610 con 0x7f0b9c06a840 2023-10-14T10:36:41.928 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.918+0000 7f0ba3b15700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9c0c3fe0 con 0x7f0b9c0cede0 2023-10-14T10:36:41.928 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.918+0000 7f0ba10b0700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b9c0cd950 0x7f0b9c0c97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:41.928 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.918+0000 7f0ba10b0700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b9c0cd950 0x7f0b9c0c97a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45178/0 (socket says 172.21.15.67:45178) 2023-10-14T10:36:41.928 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.918+0000 7f0ba10b0700 1 -- 172.21.15.67:0/2326954883 learned_addr learned my addr 172.21.15.67:0/2326954883 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:41.928 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.918+0000 7f0ba18b1700 1 --2- 172.21.15.67:0/2326954883 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0b9c06a840 0x7f0b9c0c9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:41.929 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.918+0000 7f0ba20b2700 1 --2- 172.21.15.67:0/2326954883 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f0b9c0cede0 0x7f0b9c1f36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:41.929 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.918+0000 7f0ba10b0700 1 -- 172.21.15.67:0/2326954883 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f0b9c0cede0 msgr2=0x7f0b9c1f36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:41.929 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.918+0000 7f0ba10b0700 1 --2- 172.21.15.67:0/2326954883 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f0b9c0cede0 0x7f0b9c1f36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:41.929 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.918+0000 7f0ba10b0700 1 -- 172.21.15.67:0/2326954883 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0b9c06a840 msgr2=0x7f0b9c0c9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:41.929 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.918+0000 7f0ba10b0700 1 --2- 172.21.15.67:0/2326954883 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0b9c06a840 0x7f0b9c0c9260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:41.929 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.918+0000 7f0ba10b0700 1 -- 172.21.15.67:0/2326954883 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b98009500 con 0x7f0b9c0cd950 2023-10-14T10:36:41.930 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.918+0000 7f0ba10b0700 1 --2- 172.21.15.67:0/2326954883 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b9c0cd950 0x7f0b9c0c97a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f0b98009130 tx=0x7f0b98004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:41.930 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.918+0000 7f0b927fc700 1 -- 172.21.15.67:0/2326954883 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b98003c70 con 0x7f0b9c0cd950 2023-10-14T10:36:41.930 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.918+0000 7f0b927fc700 1 -- 172.21.15.67:0/2326954883 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b98003dd0 con 0x7f0b9c0cd950 2023-10-14T10:36:41.930 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.918+0000 7f0ba3b15700 1 -- 172.21.15.67:0/2326954883 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b9c0c42c0 con 0x7f0b9c0cd950 2023-10-14T10:36:41.930 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.918+0000 7f0b927fc700 1 -- 172.21.15.67:0/2326954883 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b98011660 con 0x7f0b9c0cd950 2023-10-14T10:36:41.931 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.919+0000 7f0ba3b15700 1 -- 172.21.15.67:0/2326954883 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b9c0c8d40 con 0x7f0b9c0cd950 2023-10-14T10:36:41.931 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.919+0000 7f0ba3b15700 1 -- 172.21.15.67:0/2326954883 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b9c067d20 con 0x7f0b9c0cd950 2023-10-14T10:36:41.931 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.920+0000 7f0b927fc700 1 -- 172.21.15.67:0/2326954883 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f0b980117c0 con 0x7f0b9c0cd950 2023-10-14T10:36:41.931 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.920+0000 7f0b927fc700 1 --2- 172.21.15.67:0/2326954883 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f0b8006af00 0x7f0b8006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:41.932 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.920+0000 7f0b927fc700 1 -- 172.21.15.67:0/2326954883 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0b9808a6d0 con 0x7f0b9c0cd950 2023-10-14T10:36:41.932 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.920+0000 7f0ba18b1700 1 --2- 172.21.15.67:0/2326954883 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f0b8006af00 0x7f0b8006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:41.932 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.921+0000 7f0ba18b1700 1 --2- 172.21.15.67:0/2326954883 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f0b8006af00 0x7f0b8006d3c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c0ceb60 tx=0x7f0b8c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:41.932 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:41.925+0000 7f0b927fc700 1 -- 172.21.15.67:0/2326954883 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0b98055790 con 0x7f0b9c0cd950 2023-10-14T10:36:42.126 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.124+0000 7f0ba3b15700 1 -- 172.21.15.67:0/2326954883 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0b9c0b2100 con 0x7f0b9c0cd950 2023-10-14T10:36:42.127 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.124+0000 7f0b927fc700 1 -- 172.21.15.67:0/2326954883 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0b98058db0 con 0x7f0b9c0cd950 2023-10-14T10:36:42.127 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2023-10-14T10:36:42.129 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.126+0000 7f0ba3b15700 1 -- 172.21.15.67:0/2326954883 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f0b8006af00 msgr2=0x7f0b8006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:42.129 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.126+0000 7f0ba3b15700 1 --2- 172.21.15.67:0/2326954883 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f0b8006af00 0x7f0b8006d3c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c0ceb60 tx=0x7f0b8c008040 comp rx=0 tx=0).stop 2023-10-14T10:36:42.129 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.126+0000 7f0ba3b15700 1 -- 172.21.15.67:0/2326954883 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b9c0cd950 msgr2=0x7f0b9c0c97a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:42.129 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.126+0000 7f0ba3b15700 1 --2- 172.21.15.67:0/2326954883 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b9c0cd950 0x7f0b9c0c97a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f0b98009130 tx=0x7f0b98004750 comp rx=0 tx=0).stop 2023-10-14T10:36:42.129 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.126+0000 7f0ba3b15700 1 -- 172.21.15.67:0/2326954883 shutdown_connections 2023-10-14T10:36:42.130 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.126+0000 7f0ba3b15700 1 --2- 172.21.15.67:0/2326954883 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f0b8006af00 0x7f0b8006d3c0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:42.130 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.126+0000 7f0ba3b15700 1 --2- 172.21.15.67:0/2326954883 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f0b9c0cede0 0x7f0b9c1f36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:42.130 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.126+0000 7f0ba3b15700 1 --2- 172.21.15.67:0/2326954883 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0b9c06a840 0x7f0b9c0c9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:42.130 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.126+0000 7f0ba3b15700 1 --2- 172.21.15.67:0/2326954883 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b9c0cd950 0x7f0b9c0c97a0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:42.130 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.126+0000 7f0ba3b15700 1 -- 172.21.15.67:0/2326954883 >> 172.21.15.67:0/2326954883 conn(0x7f0b9c06dae0 msgr2=0x7f0b9c15ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:42.131 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.126+0000 7f0ba3b15700 1 -- 172.21.15.67:0/2326954883 shutdown_connections 2023-10-14T10:36:42.131 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.126+0000 7f0ba3b15700 1 -- 172.21.15.67:0/2326954883 wait complete. 2023-10-14T10:36:42.141 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-14T10:36:42.267 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.265+0000 7f0c263c1700 1 Processor -- start 2023-10-14T10:36:42.267 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.265+0000 7f0c263c1700 1 -- start start 2023-10-14T10:36:42.270 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.265+0000 7f0c263c1700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f0c2015a850 0x7f0c200cbc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:42.271 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.266+0000 7f0c263c1700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c200ce3b0 con 0x7f0c200cd4b0 2023-10-14T10:36:42.271 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.266+0000 7f0c263c1700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c200b2dc0 con 0x7f0c200b2940 2023-10-14T10:36:42.271 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.266+0000 7f0c263c1700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c200b6290 con 0x7f0c2015a850 2023-10-14T10:36:42.271 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.266+0000 7f0c2495e700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f0c200cd4b0 0x7f0c200ce840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:52956/0 (socket says 172.21.15.67:52956) 2023-10-14T10:36:42.272 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.266+0000 7f0c2495e700 1 -- 172.21.15.67:0/4048995755 learned_addr learned my addr 172.21.15.67:0/4048995755 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:36:42.272 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.266+0000 7f0c1effd700 1 -- 172.21.15.67:0/4048995755 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2633513696 0 0) 0x7f0c200ce3b0 con 0x7f0c200cd4b0 2023-10-14T10:36:42.272 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.266+0000 7f0c1ffff700 1 --2- 172.21.15.67:0/4048995755 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f0c2015a850 0x7f0c200cbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:42.272 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.266+0000 7f0c1effd700 1 -- 172.21.15.67:0/4048995755 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c080035b0 con 0x7f0c200cd4b0 2023-10-14T10:36:42.272 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.266+0000 7f0c1effd700 1 -- 172.21.15.67:0/4048995755 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 814398785 0 0) 0x7f0c080035b0 con 0x7f0c200cd4b0 2023-10-14T10:36:42.273 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.266+0000 7f0c1effd700 1 -- 172.21.15.67:0/4048995755 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f0c2015a850 msgr2=0x7f0c200cbc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:42.273 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.266+0000 7f0c1effd700 1 --2- 172.21.15.67:0/4048995755 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f0c2015a850 0x7f0c200cbc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:42.273 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.266+0000 7f0c1effd700 1 -- 172.21.15.67:0/4048995755 >> v1:172.21.15.72:6789/0 conn(0x7f0c200b2940 legacy=0x7f0c200ccda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:42.273 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.266+0000 7f0c1effd700 1 -- 172.21.15.67:0/4048995755 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c200cc220 con 0x7f0c200cd4b0 2023-10-14T10:36:42.274 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.266+0000 7f0c1effd700 1 -- 172.21.15.67:0/4048995755 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3180043755 0 0) 0x7f0c180019f0 con 0x7f0c200cd4b0 2023-10-14T10:36:42.274 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.266+0000 7f0c1effd700 1 -- 172.21.15.67:0/4048995755 >> v1:172.21.15.67:6789/0 conn(0x7f0c200cd4b0 legacy=0x7f0c200ce840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:42.274 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.266+0000 7f0c1effd700 1 --2- 172.21.15.67:0/4048995755 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f0c2015a850 0x7f0c08003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:42.274 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.266+0000 7f0c1effd700 1 --2- 172.21.15.67:0/4048995755 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0c08003fb0 0x7f0c08004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:42.274 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.266+0000 7f0c1effd700 1 --2- 172.21.15.67:0/4048995755 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c08004d90 0x7f0c08005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:42.275 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.266+0000 7f0c1effd700 1 -- 172.21.15.67:0/4048995755 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c200b6290 con 0x7f0c08004d90 2023-10-14T10:36:42.275 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.266+0000 7f0c1effd700 1 -- 172.21.15.67:0/4048995755 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c0800baf0 con 0x7f0c08003fb0 2023-10-14T10:36:42.275 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.266+0000 7f0c1effd700 1 -- 172.21.15.67:0/4048995755 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c0800dec0 con 0x7f0c2015a850 2023-10-14T10:36:42.275 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.266+0000 7f0c2495e700 1 --2- 172.21.15.67:0/4048995755 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c08004d90 0x7f0c08005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:42.275 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.266+0000 7f0c1ffff700 1 --2- 172.21.15.67:0/4048995755 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f0c2015a850 0x7f0c08003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:42.275 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.267+0000 7f0c1f7fe700 1 --2- 172.21.15.67:0/4048995755 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0c08003fb0 0x7f0c08004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:42.276 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.267+0000 7f0c2495e700 1 -- 172.21.15.67:0/4048995755 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f0c2015a850 msgr2=0x7f0c08003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:42.276 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.267+0000 7f0c2495e700 1 --2- 172.21.15.67:0/4048995755 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f0c2015a850 0x7f0c08003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:42.276 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.267+0000 7f0c2495e700 1 -- 172.21.15.67:0/4048995755 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0c08003fb0 msgr2=0x7f0c08004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:42.276 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.267+0000 7f0c2495e700 1 --2- 172.21.15.67:0/4048995755 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0c08003fb0 0x7f0c08004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:42.276 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.267+0000 7f0c2495e700 1 -- 172.21.15.67:0/4048995755 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c080035b0 con 0x7f0c08004d90 2023-10-14T10:36:42.277 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.267+0000 7f0c2495e700 1 --2- 172.21.15.67:0/4048995755 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c08004d90 0x7f0c08005210 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0c1800a830 tx=0x7f0c08005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=211ca19e38fb36ef server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:42.277 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.267+0000 7f0c1effd700 1 -- 172.21.15.67:0/4048995755 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c1800a2f0 con 0x7f0c08004d90 2023-10-14T10:36:42.277 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.267+0000 7f0c1effd700 1 -- 172.21.15.67:0/4048995755 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c18012620 con 0x7f0c08004d90 2023-10-14T10:36:42.277 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.267+0000 7f0c1effd700 1 -- 172.21.15.67:0/4048995755 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c180129d0 con 0x7f0c08004d90 2023-10-14T10:36:42.277 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.267+0000 7f0c263c1700 1 -- 172.21.15.67:0/4048995755 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c08004d90 msgr2=0x7f0c08005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:42.278 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.267+0000 7f0c263c1700 1 --2- 172.21.15.67:0/4048995755 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c08004d90 0x7f0c08005210 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0c1800a830 tx=0x7f0c08005880 comp rx=0 tx=0).stop 2023-10-14T10:36:42.278 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.268+0000 7f0c2495e700 1 -- 172.21.15.67:0/4048995755 reap_dead start 2023-10-14T10:36:42.278 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.268+0000 7f0c263c1700 1 -- 172.21.15.67:0/4048995755 shutdown_connections 2023-10-14T10:36:42.278 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.268+0000 7f0c263c1700 1 -- 172.21.15.67:0/4048995755 >> 172.21.15.67:0/4048995755 conn(0x7f0c2006dac0 msgr2=0x7f0c2006dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:42.279 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.268+0000 7f0c263c1700 1 -- 172.21.15.67:0/4048995755 shutdown_connections 2023-10-14T10:36:42.279 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.268+0000 7f0c263c1700 1 -- 172.21.15.67:0/4048995755 wait complete. 2023-10-14T10:36:42.279 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.269+0000 7f0c263c1700 1 Processor -- start 2023-10-14T10:36:42.281 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.269+0000 7f0c263c1700 1 -- start start 2023-10-14T10:36:42.281 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.270+0000 7f0c263c1700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c2015cf20 0x7f0c2015d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:42.282 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.270+0000 7f0c263c1700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0c2014b010 0x7f0c2014b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:42.282 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.270+0000 7f0c263c1700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f0c20158250 0x7f0c201586d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:42.282 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.270+0000 7f0c263c1700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c200c8a50 con 0x7f0c2015cf20 2023-10-14T10:36:42.282 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.270+0000 7f0c263c1700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c2015d840 con 0x7f0c2014b010 2023-10-14T10:36:42.282 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.270+0000 7f0c263c1700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c200c7dd0 con 0x7f0c20158250 2023-10-14T10:36:42.283 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.270+0000 7f0c1ffff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c2015cf20 0x7f0c2015d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:42.283 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.270+0000 7f0c1ffff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c2015cf20 0x7f0c2015d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45196/0 (socket says 172.21.15.67:45196) 2023-10-14T10:36:42.283 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.270+0000 7f0c1ffff700 1 -- 172.21.15.67:0/1479652723 learned_addr learned my addr 172.21.15.67:0/1479652723 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:42.283 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.270+0000 7f0c1f7fe700 1 --2- 172.21.15.67:0/1479652723 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0c2014b010 0x7f0c2014b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:42.283 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.270+0000 7f0c2495e700 1 --2- 172.21.15.67:0/1479652723 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f0c20158250 0x7f0c201586d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:42.284 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.270+0000 7f0c1ffff700 1 -- 172.21.15.67:0/1479652723 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f0c20158250 msgr2=0x7f0c201586d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:42.284 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.270+0000 7f0c1ffff700 1 --2- 172.21.15.67:0/1479652723 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f0c20158250 0x7f0c201586d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:42.284 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.270+0000 7f0c1ffff700 1 -- 172.21.15.67:0/1479652723 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0c2014b010 msgr2=0x7f0c2014b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:42.284 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.270+0000 7f0c1ffff700 1 --2- 172.21.15.67:0/1479652723 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0c2014b010 0x7f0c2014b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:42.284 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.270+0000 7f0c1ffff700 1 -- 172.21.15.67:0/1479652723 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c200c80b0 con 0x7f0c2015cf20 2023-10-14T10:36:42.285 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.270+0000 7f0c2495e700 1 --2- 172.21.15.67:0/1479652723 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f0c20158250 0x7f0c201586d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-14T10:36:42.285 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.270+0000 7f0c1ffff700 1 --2- 172.21.15.67:0/1479652723 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c2015cf20 0x7f0c2015d300 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f0c10011aa0 tx=0x7f0c10011db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:42.285 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.270+0000 7f0c1cff9700 1 -- 172.21.15.67:0/1479652723 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c1000f940 con 0x7f0c2015cf20 2023-10-14T10:36:42.285 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.270+0000 7f0c1cff9700 1 -- 172.21.15.67:0/1479652723 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c1000faa0 con 0x7f0c2015cf20 2023-10-14T10:36:42.285 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.270+0000 7f0c263c1700 1 -- 172.21.15.67:0/1479652723 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c200c8390 con 0x7f0c2015cf20 2023-10-14T10:36:42.286 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.271+0000 7f0c1cff9700 1 -- 172.21.15.67:0/1479652723 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c1000fe50 con 0x7f0c2015cf20 2023-10-14T10:36:42.286 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.271+0000 7f0c263c1700 1 -- 172.21.15.67:0/1479652723 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c2014b9d0 con 0x7f0c2015cf20 2023-10-14T10:36:42.286 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.271+0000 7f0c263c1700 1 -- 172.21.15.67:0/1479652723 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c20067d20 con 0x7f0c2015cf20 2023-10-14T10:36:42.286 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.272+0000 7f0c1cff9700 1 -- 172.21.15.67:0/1479652723 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f0c10014440 con 0x7f0c2015cf20 2023-10-14T10:36:42.286 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.272+0000 7f0c1cff9700 1 --2- 172.21.15.67:0/1479652723 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f0c0c06af30 0x7f0c0c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:42.287 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.272+0000 7f0c1cff9700 1 -- 172.21.15.67:0/1479652723 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0c10089d60 con 0x7f0c2015cf20 2023-10-14T10:36:42.287 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.273+0000 7f0c1f7fe700 1 --2- 172.21.15.67:0/1479652723 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f0c0c06af30 0x7f0c0c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:42.287 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.273+0000 7f0c1f7fe700 1 --2- 172.21.15.67:0/1479652723 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f0c0c06af30 0x7f0c0c06d3f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0c2015bb70 tx=0x7f0c14009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:42.287 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.277+0000 7f0c1cff9700 1 -- 172.21.15.67:0/1479652723 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0c10054e40 con 0x7f0c2015cf20 2023-10-14T10:36:42.498 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.496+0000 7f0c263c1700 1 -- 172.21.15.67:0/1479652723 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f0c2006b460 con 0x7f0c2015cf20 2023-10-14T10:36:42.498 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.496+0000 7f0c1cff9700 1 -- 172.21.15.67:0/1479652723 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7f0c10058460 con 0x7f0c2015cf20 2023-10-14T10:36:42.500 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.498+0000 7f0c263c1700 1 -- 172.21.15.67:0/1479652723 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f0c0c06af30 msgr2=0x7f0c0c06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:42.500 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.498+0000 7f0c263c1700 1 --2- 172.21.15.67:0/1479652723 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f0c0c06af30 0x7f0c0c06d3f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0c2015bb70 tx=0x7f0c14009040 comp rx=0 tx=0).stop 2023-10-14T10:36:42.501 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.498+0000 7f0c263c1700 1 -- 172.21.15.67:0/1479652723 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c2015cf20 msgr2=0x7f0c2015d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:42.501 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.498+0000 7f0c263c1700 1 --2- 172.21.15.67:0/1479652723 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c2015cf20 0x7f0c2015d300 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f0c10011aa0 tx=0x7f0c10011db0 comp rx=0 tx=0).stop 2023-10-14T10:36:42.501 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.498+0000 7f0c263c1700 1 -- 172.21.15.67:0/1479652723 shutdown_connections 2023-10-14T10:36:42.501 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.498+0000 7f0c263c1700 1 --2- 172.21.15.67:0/1479652723 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f0c0c06af30 0x7f0c0c06d3f0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:42.501 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.498+0000 7f0c263c1700 1 --2- 172.21.15.67:0/1479652723 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f0c20158250 0x7f0c201586d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:42.502 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.498+0000 7f0c263c1700 1 --2- 172.21.15.67:0/1479652723 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c2015cf20 0x7f0c2015d300 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:42.502 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.498+0000 7f0c263c1700 1 --2- 172.21.15.67:0/1479652723 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0c2014b010 0x7f0c2014b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:42.502 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.498+0000 7f0c263c1700 1 -- 172.21.15.67:0/1479652723 >> 172.21.15.67:0/1479652723 conn(0x7f0c2006dac0 msgr2=0x7f0c20159740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:42.502 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.498+0000 7f0c263c1700 1 -- 172.21.15.67:0/1479652723 shutdown_connections 2023-10-14T10:36:42.502 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.498+0000 7f0c263c1700 1 -- 172.21.15.67:0/1479652723 wait complete. 2023-10-14T10:36:42.503 INFO:teuthology.orchestra.run.smithi067.stderr:listed 0 entries 2023-10-14T10:36:42.512 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2023-10-14T10:36:42.512 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:36:42.513 DEBUG:teuthology.orchestra.run.smithi067:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2023-10-14T10:36:42.553 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi067.front.sepia.ceph.com 2023-10-14T10:36:42.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:36:42.554 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-14T10:36:42.611 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-10-14T10:36:42.611 DEBUG:teuthology.orchestra.run.smithi072:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-14T10:36:42.633 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-10-14T10:36:42.633 DEBUG:teuthology.orchestra.run.smithi072:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-14T10:36:42.693 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-10-14T10:36:42.693 DEBUG:teuthology.orchestra.run.smithi072:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-14T10:36:42.752 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-14T10:36:42.878 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.876+0000 7f5f497fe700 1 Processor -- start 2023-10-14T10:36:42.879 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.876+0000 7f5f497fe700 1 -- start start 2023-10-14T10:36:42.881 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.877+0000 7f5f497fe700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f5f440d3fc0 0x7f5f4415dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:42.881 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.877+0000 7f5f497fe700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f5f440b7050 0x7f5f440d21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:42.881 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.877+0000 7f5f497fe700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f4415e4e0 con 0x7f5f4415dbc0 2023-10-14T10:36:42.882 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.877+0000 7f5f497fe700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f440b7560 con 0x7f5f440b7050 2023-10-14T10:36:42.882 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.877+0000 7f5f497fe700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f440b7840 con 0x7f5f440d3fc0 2023-10-14T10:36:42.882 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.877+0000 7f5f42ffd700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f5f4415dbc0 0x7f5f440d1ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:52972/0 (socket says 172.21.15.67:52972) 2023-10-14T10:36:42.882 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.877+0000 7f5f42ffd700 1 -- 172.21.15.67:0/2028344732 learned_addr learned my addr 172.21.15.67:0/2028344732 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:36:42.882 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.877+0000 7f5f437fe700 1 --2- 172.21.15.67:0/2028344732 >> v2:172.21.15.67:3300/0 conn(0x7f5f440b7050 0x7f5f440d21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:42.883 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.877+0000 7f5f427fc700 1 --2- 172.21.15.67:0/2028344732 >> v2:172.21.15.72:3300/0 conn(0x7f5f440d3fc0 0x7f5f4415dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:42.883 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.877+0000 7f5f41ffb700 1 -- 172.21.15.67:0/2028344732 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3676619252 0 0) 0x7f5f4415e4e0 con 0x7f5f4415dbc0 2023-10-14T10:36:42.883 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.877+0000 7f5f41ffb700 1 -- 172.21.15.67:0/2028344732 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f300035b0 con 0x7f5f4415dbc0 2023-10-14T10:36:42.883 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.878+0000 7f5f437fe700 1 -- 172.21.15.67:0/2028344732 >> v2:172.21.15.72:3300/0 conn(0x7f5f440d3fc0 msgr2=0x7f5f4415dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:42.883 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.878+0000 7f5f437fe700 1 --2- 172.21.15.67:0/2028344732 >> v2:172.21.15.72:3300/0 conn(0x7f5f440d3fc0 0x7f5f4415dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:42.884 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.878+0000 7f5f437fe700 1 -- 172.21.15.67:0/2028344732 >> v1:172.21.15.67:6789/0 conn(0x7f5f4415dbc0 legacy=0x7f5f440d1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:42.884 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.878+0000 7f5f437fe700 1 -- 172.21.15.67:0/2028344732 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f440b7b20 con 0x7f5f440b7050 2023-10-14T10:36:42.884 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.878+0000 7f5f437fe700 1 --2- 172.21.15.67:0/2028344732 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5f440b7050 0x7f5f440d21d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5f440d14e0 tx=0x7f5f3c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=209f8ada922c976c server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:42.884 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.878+0000 7f5f41ffb700 1 -- 172.21.15.67:0/2028344732 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f3c00e070 con 0x7f5f440b7050 2023-10-14T10:36:42.884 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.878+0000 7f5f41ffb700 1 -- 172.21.15.67:0/2028344732 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f3c00bb20 con 0x7f5f440b7050 2023-10-14T10:36:42.885 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.878+0000 7f5f41ffb700 1 -- 172.21.15.67:0/2028344732 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f3c012430 con 0x7f5f440b7050 2023-10-14T10:36:42.885 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.878+0000 7f5f497fe700 1 -- 172.21.15.67:0/2028344732 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5f440b7050 msgr2=0x7f5f440d21d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:42.885 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.878+0000 7f5f497fe700 1 --2- 172.21.15.67:0/2028344732 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5f440b7050 0x7f5f440d21d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5f440d14e0 tx=0x7f5f3c00b330 comp rx=0 tx=0).stop 2023-10-14T10:36:42.885 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.878+0000 7f5f497fe700 1 -- 172.21.15.67:0/2028344732 shutdown_connections 2023-10-14T10:36:42.886 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.878+0000 7f5f497fe700 1 --2- 172.21.15.67:0/2028344732 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5f440b7050 0x7f5f440d21d0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:42.886 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.878+0000 7f5f497fe700 1 --2- 172.21.15.67:0/2028344732 >> v2:172.21.15.72:3300/0 conn(0x7f5f440d3fc0 0x7f5f4415dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:42.886 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.878+0000 7f5f497fe700 1 -- 172.21.15.67:0/2028344732 >> 172.21.15.67:0/2028344732 conn(0x7f5f4409cb40 msgr2=0x7f5f440d3820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:42.886 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.879+0000 7f5f497fe700 1 -- 172.21.15.67:0/2028344732 shutdown_connections 2023-10-14T10:36:42.886 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.879+0000 7f5f497fe700 1 -- 172.21.15.67:0/2028344732 wait complete. 2023-10-14T10:36:42.889 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.880+0000 7f5f497fe700 1 Processor -- start 2023-10-14T10:36:42.889 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.880+0000 7f5f497fe700 1 -- start start 2023-10-14T10:36:42.890 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.881+0000 7f5f497fe700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f5f440b7050 0x7f5f441e8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:42.890 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.881+0000 7f5f497fe700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5f440d3fc0 0x7f5f441555d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:42.890 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.881+0000 7f5f497fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5f4415dbc0 0x7f5f441543e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:42.890 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.881+0000 7f5f497fe700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f44159e60 con 0x7f5f4415dbc0 2023-10-14T10:36:42.890 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.881+0000 7f5f497fe700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f44155e00 con 0x7f5f440d3fc0 2023-10-14T10:36:42.891 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.881+0000 7f5f497fe700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f441560e0 con 0x7f5f440b7050 2023-10-14T10:36:42.891 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.881+0000 7f5f437fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5f4415dbc0 0x7f5f441543e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:42.891 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.881+0000 7f5f437fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5f4415dbc0 0x7f5f441543e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45214/0 (socket says 172.21.15.67:45214) 2023-10-14T10:36:42.891 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.881+0000 7f5f437fe700 1 -- 172.21.15.67:0/2602377362 learned_addr learned my addr 172.21.15.67:0/2602377362 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:42.891 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.881+0000 7f5f42ffd700 1 --2- 172.21.15.67:0/2602377362 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f5f440b7050 0x7f5f441e8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:42.892 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.881+0000 7f5f427fc700 1 --2- 172.21.15.67:0/2602377362 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5f440d3fc0 0x7f5f441555d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:42.892 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.881+0000 7f5f437fe700 1 -- 172.21.15.67:0/2602377362 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f5f440b7050 msgr2=0x7f5f441e8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:42.892 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.881+0000 7f5f437fe700 1 --2- 172.21.15.67:0/2602377362 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f5f440b7050 0x7f5f441e8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:42.892 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.881+0000 7f5f437fe700 1 -- 172.21.15.67:0/2602377362 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5f440d3fc0 msgr2=0x7f5f441555d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:42.892 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.881+0000 7f5f437fe700 1 --2- 172.21.15.67:0/2602377362 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5f440d3fc0 0x7f5f441555d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:42.893 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.881+0000 7f5f437fe700 1 -- 172.21.15.67:0/2602377362 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f3c009a70 con 0x7f5f4415dbc0 2023-10-14T10:36:42.893 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.881+0000 7f5f437fe700 1 --2- 172.21.15.67:0/2602377362 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5f4415dbc0 0x7f5f441543e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c014040 tx=0x7f5f3c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:42.893 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.881+0000 7f5f3b7fe700 1 -- 172.21.15.67:0/2602377362 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f3c012430 con 0x7f5f4415dbc0 2023-10-14T10:36:42.893 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.881+0000 7f5f3b7fe700 1 -- 172.21.15.67:0/2602377362 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f3c020070 con 0x7f5f4415dbc0 2023-10-14T10:36:42.893 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.881+0000 7f5f497fe700 1 -- 172.21.15.67:0/2602377362 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f4415a070 con 0x7f5f4415dbc0 2023-10-14T10:36:42.894 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.881+0000 7f5f3b7fe700 1 -- 172.21.15.67:0/2602377362 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f3c0098f0 con 0x7f5f4415dbc0 2023-10-14T10:36:42.894 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.882+0000 7f5f497fe700 1 -- 172.21.15.67:0/2602377362 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f44155b10 con 0x7f5f4415dbc0 2023-10-14T10:36:42.894 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.882+0000 7f5f497fe700 1 -- 172.21.15.67:0/2602377362 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f4404ebb0 con 0x7f5f4415dbc0 2023-10-14T10:36:42.894 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.883+0000 7f5f3b7fe700 1 -- 172.21.15.67:0/2602377362 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f5f3c004040 con 0x7f5f4415dbc0 2023-10-14T10:36:42.894 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.883+0000 7f5f3b7fe700 1 --2- 172.21.15.67:0/2602377362 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f5f2406af00 0x7f5f2406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:42.895 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.883+0000 7f5f3b7fe700 1 -- 172.21.15.67:0/2602377362 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5f3c022070 con 0x7f5f4415dbc0 2023-10-14T10:36:42.895 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.884+0000 7f5f42ffd700 1 --2- 172.21.15.67:0/2602377362 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f5f2406af00 0x7f5f2406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:42.895 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.884+0000 7f5f42ffd700 1 --2- 172.21.15.67:0/2602377362 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f5f2406af00 0x7f5f2406d3c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5f340010e0 tx=0x7f5f34007c70 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:42.896 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:42.888+0000 7f5f3b7fe700 1 -- 172.21.15.67:0/2602377362 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5f3c059c40 con 0x7f5f4415dbc0 2023-10-14T10:36:43.092 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.090+0000 7f5f497fe700 1 -- 172.21.15.67:0/2602377362 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5f440cdc50 con 0x7f5f4415dbc0 2023-10-14T10:36:43.093 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.091+0000 7f5f3b7fe700 1 -- 172.21.15.67:0/2602377362 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16457 (secure 0 0 0) 0x7f5f3c0597d0 con 0x7f5f4415dbc0 2023-10-14T10:36:43.094 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-14T10:36:43.094 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":22,"fsid":"37f9ab46-3082-4b71-8b24-08d39897b008","created":"2023-10-14T10:35:46.067554+0000","modified":"2023-10-14T10:36:04.837296+0000","last_up_change":"2023-10-14T10:35:56.668142+0000","last_in_change":"2023-10-14T10:35:48.551490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-14T10:35:57.809566+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-14T10:36:01.667146+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-14T10:36:02.219700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2b59a07e-11ef-440c-b766-9f64ac885838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6801","nonce":1287255499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6803","nonce":1287255499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6807","nonce":1287255499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6805","nonce":1287255499}]},"public_addr":"172.21.15.67:6801/1287255499","cluster_addr":"172.21.15.67:6803/1287255499","heartbeat_back_addr":"172.21.15.67:6807/1287255499","heartbeat_front_addr":"172.21.15.67:6805/1287255499","state":["exists","up"]},{"osd":1,"uuid":"6f813bd6-fa0b-487d-a204-431eb1276a60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6818","nonce":2435538787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6822","nonce":2435538787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6830","nonce":2435538787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6826","nonce":2435538787}]},"public_addr":"172.21.15.67:6818/2435538787","cluster_addr":"172.21.15.67:6822/2435538787","heartbeat_back_addr":"172.21.15.67:6830/2435538787","heartbeat_front_addr":"172.21.15.67:6826/2435538787","state":["exists","up"]},{"osd":2,"uuid":"6af42c6a-1b59-4669-9c04-b6876788446d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6809","nonce":2827491791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6811","nonce":2827491791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6815","nonce":2827491791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6813","nonce":2827491791}]},"public_addr":"172.21.15.67:6809/2827491791","cluster_addr":"172.21.15.67:6811/2827491791","heartbeat_back_addr":"172.21.15.67:6815/2827491791","heartbeat_front_addr":"172.21.15.67:6813/2827491791","state":["exists","up"]},{"osd":3,"uuid":"1d889d76-a646-499d-bfff-ed5bae5434ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6819","nonce":1848001176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6821","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6823","nonce":1848001176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6831","nonce":1848001176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6825","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6827","nonce":1848001176}]},"public_addr":"172.21.15.67:6819/1848001176","cluster_addr":"172.21.15.67:6823/1848001176","heartbeat_back_addr":"172.21.15.67:6831/1848001176","heartbeat_front_addr":"172.21.15.67:6827/1848001176","state":["exists","up"]},{"osd":4,"uuid":"c3271dff-b437-4fdd-b13b-6e49a35473af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6804","nonce":1891230785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6808","nonce":1891230785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6815","nonce":1891230785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6811","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6813","nonce":1891230785}]},"public_addr":"172.21.15.72:6804/1891230785","cluster_addr":"172.21.15.72:6808/1891230785","heartbeat_back_addr":"172.21.15.72:6815/1891230785","heartbeat_front_addr":"172.21.15.72:6813/1891230785","state":["exists","up"]},{"osd":5,"uuid":"0337bb18-4b68-4eff-b528-a667275e6926","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6801","nonce":569686991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6803","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6805","nonce":569686991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6812","nonce":569686991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6807","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6809","nonce":569686991}]},"public_addr":"172.21.15.72:6801/569686991","cluster_addr":"172.21.15.72:6805/569686991","heartbeat_back_addr":"172.21.15.72:6812/569686991","heartbeat_front_addr":"172.21.15.72:6809/569686991","state":["exists","up"]},{"osd":6,"uuid":"8e92d9cf-0c8c-46eb-89a7-db2023ef1509","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6817","nonce":3767685934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6819","nonce":3767685934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6823","nonce":3767685934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6821","nonce":3767685934}]},"public_addr":"172.21.15.72:6817/3767685934","cluster_addr":"172.21.15.72:6819/3767685934","heartbeat_back_addr":"172.21.15.72:6823/3767685934","heartbeat_front_addr":"172.21.15.72:6821/3767685934","state":["exists","up"]},{"osd":7,"uuid":"3fd6aa7d-50a9-45c9-9561-5902a044a119","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6825","nonce":3567465926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6827","nonce":3567465926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6831","nonce":3567465926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6829","nonce":3567465926}]},"public_addr":"172.21.15.72:6825/3567465926","cluster_addr":"172.21.15.72:6827/3567465926","heartbeat_back_addr":"172.21.15.72:6831/3567465926","heartbeat_front_addr":"172.21.15.72:6829/3567465926","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.511466+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.795107+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.858674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.728895+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.794842+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.750064+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.732122+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.765185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-14T10:36:43.096 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.094+0000 7f5f497fe700 1 -- 172.21.15.67:0/2602377362 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f5f2406af00 msgr2=0x7f5f2406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:43.096 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.094+0000 7f5f497fe700 1 --2- 172.21.15.67:0/2602377362 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f5f2406af00 0x7f5f2406d3c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5f340010e0 tx=0x7f5f34007c70 comp rx=0 tx=0).stop 2023-10-14T10:36:43.097 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.094+0000 7f5f497fe700 1 -- 172.21.15.67:0/2602377362 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5f4415dbc0 msgr2=0x7f5f441543e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:43.097 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.094+0000 7f5f497fe700 1 --2- 172.21.15.67:0/2602377362 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5f4415dbc0 0x7f5f441543e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c014040 tx=0x7f5f3c004ad0 comp rx=0 tx=0).stop 2023-10-14T10:36:43.097 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.094+0000 7f5f497fe700 1 -- 172.21.15.67:0/2602377362 shutdown_connections 2023-10-14T10:36:43.097 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.094+0000 7f5f497fe700 1 --2- 172.21.15.67:0/2602377362 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f5f2406af00 0x7f5f2406d3c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:43.097 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.094+0000 7f5f497fe700 1 --2- 172.21.15.67:0/2602377362 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5f4415dbc0 0x7f5f441543e0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:43.098 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.094+0000 7f5f497fe700 1 --2- 172.21.15.67:0/2602377362 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f5f440b7050 0x7f5f441e8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:43.098 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.094+0000 7f5f497fe700 1 --2- 172.21.15.67:0/2602377362 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5f440d3fc0 0x7f5f441555d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:43.098 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.094+0000 7f5f497fe700 1 -- 172.21.15.67:0/2602377362 >> 172.21.15.67:0/2602377362 conn(0x7f5f4409cb40 msgr2=0x7f5f440ce5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:43.098 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.094+0000 7f5f497fe700 1 -- 172.21.15.67:0/2602377362 shutdown_connections 2023-10-14T10:36:43.098 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.094+0000 7f5f497fe700 1 -- 172.21.15.67:0/2602377362 wait complete. 2023-10-14T10:36:43.108 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-14T10:35:57.809566+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-14T10:36:01.667146+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-14T10:36:02.219700+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-14T10:36:43.109 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-14T10:36:43.236 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.234+0000 7fde28432700 1 Processor -- start 2023-10-14T10:36:43.237 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.234+0000 7fde28432700 1 -- start start 2023-10-14T10:36:43.239 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.235+0000 7fde28432700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fde2006ef20 0x7fde200cdb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:43.239 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.235+0000 7fde28432700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fde200ce0b0 0x7fde2006a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:43.239 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.235+0000 7fde28432700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde2015bf30 con 0x7fde2015bb50 2023-10-14T10:36:43.239 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.235+0000 7fde28432700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde200ce540 con 0x7fde200ce0b0 2023-10-14T10:36:43.240 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.235+0000 7fde28432700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde200b2d70 con 0x7fde2006ef20 2023-10-14T10:36:43.240 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.235+0000 7fde269cf700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fde200ce0b0 0x7fde2006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:43.240 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.235+0000 7fde261ce700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fde2015bb50 0x7fde2006e810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:52986/0 (socket says 172.21.15.67:52986) 2023-10-14T10:36:43.240 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.235+0000 7fde261ce700 1 -- 172.21.15.67:0/3677475728 learned_addr learned my addr 172.21.15.67:0/3677475728 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:36:43.241 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.235+0000 7fde259cd700 1 --2- 172.21.15.67:0/3677475728 >> v2:172.21.15.72:3300/0 conn(0x7fde2006ef20 0x7fde200cdb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:43.241 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.235+0000 7fde269cf700 1 -- 172.21.15.67:0/3677475728 >> v2:172.21.15.72:3300/0 conn(0x7fde2006ef20 msgr2=0x7fde200cdb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:43.241 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.236+0000 7fde269cf700 1 --2- 172.21.15.67:0/3677475728 >> v2:172.21.15.72:3300/0 conn(0x7fde2006ef20 0x7fde200cdb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:43.241 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.236+0000 7fde269cf700 1 -- 172.21.15.67:0/3677475728 >> v1:172.21.15.67:6789/0 conn(0x7fde2015bb50 legacy=0x7fde2006e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:43.242 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.236+0000 7fde269cf700 1 -- 172.21.15.67:0/3677475728 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde2006d500 con 0x7fde200ce0b0 2023-10-14T10:36:43.242 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.236+0000 7fde269cf700 1 --2- 172.21.15.67:0/3677475728 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde200ce0b0 0x7fde2006a840 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fde20070e90 tx=0x7fde1800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfcf03f8a0e3070a server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:43.242 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.236+0000 7fde251cc700 1 -- 172.21.15.67:0/3677475728 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde1800e070 con 0x7fde200ce0b0 2023-10-14T10:36:43.243 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.236+0000 7fde251cc700 1 -- 172.21.15.67:0/3677475728 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde1800bbd0 con 0x7fde200ce0b0 2023-10-14T10:36:43.243 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.236+0000 7fde251cc700 1 -- 172.21.15.67:0/3677475728 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde18012430 con 0x7fde200ce0b0 2023-10-14T10:36:43.243 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.236+0000 7fde28432700 1 -- 172.21.15.67:0/3677475728 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde200ce0b0 msgr2=0x7fde2006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:43.243 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.236+0000 7fde28432700 1 --2- 172.21.15.67:0/3677475728 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde200ce0b0 0x7fde2006a840 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fde20070e90 tx=0x7fde1800b330 comp rx=0 tx=0).stop 2023-10-14T10:36:43.243 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.236+0000 7fde28432700 1 -- 172.21.15.67:0/3677475728 shutdown_connections 2023-10-14T10:36:43.244 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.236+0000 7fde28432700 1 --2- 172.21.15.67:0/3677475728 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde200ce0b0 0x7fde2006a840 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:43.244 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.236+0000 7fde28432700 1 --2- 172.21.15.67:0/3677475728 >> v2:172.21.15.72:3300/0 conn(0x7fde2006ef20 0x7fde200cdb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:43.244 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.236+0000 7fde28432700 1 -- 172.21.15.67:0/3677475728 >> 172.21.15.67:0/3677475728 conn(0x7fde200cc6f0 msgr2=0x7fde200ccb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:43.244 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.236+0000 7fde28432700 1 -- 172.21.15.67:0/3677475728 shutdown_connections 2023-10-14T10:36:43.244 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.237+0000 7fde28432700 1 -- 172.21.15.67:0/3677475728 wait complete. 2023-10-14T10:36:43.248 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.238+0000 7fde28432700 1 Processor -- start 2023-10-14T10:36:43.248 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.238+0000 7fde28432700 1 -- start start 2023-10-14T10:36:43.248 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.238+0000 7fde28432700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fde2006ef20 0x7fde201510b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:43.248 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.238+0000 7fde28432700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fde200ce0b0 0x7fde200cacd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:43.249 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.238+0000 7fde28432700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde2015bb50 0x7fde201525f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:43.249 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.239+0000 7fde28432700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde200c8580 con 0x7fde2015bb50 2023-10-14T10:36:43.249 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.239+0000 7fde28432700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde200c8b40 con 0x7fde200ce0b0 2023-10-14T10:36:43.249 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.239+0000 7fde28432700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde200cb750 con 0x7fde2006ef20 2023-10-14T10:36:43.250 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.239+0000 7fde269cf700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde2015bb50 0x7fde201525f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:43.250 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.239+0000 7fde269cf700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde2015bb50 0x7fde201525f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45228/0 (socket says 172.21.15.67:45228) 2023-10-14T10:36:43.250 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.239+0000 7fde269cf700 1 -- 172.21.15.67:0/2116934724 learned_addr learned my addr 172.21.15.67:0/2116934724 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:43.250 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.239+0000 7fde259cd700 1 --2- 172.21.15.67:0/2116934724 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fde200ce0b0 0x7fde200cacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:43.250 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.239+0000 7fde261ce700 1 --2- 172.21.15.67:0/2116934724 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fde2006ef20 0x7fde201510b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:43.251 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.239+0000 7fde269cf700 1 -- 172.21.15.67:0/2116934724 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fde2006ef20 msgr2=0x7fde201510b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:43.251 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.239+0000 7fde269cf700 1 --2- 172.21.15.67:0/2116934724 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fde2006ef20 0x7fde201510b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:43.251 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.239+0000 7fde269cf700 1 -- 172.21.15.67:0/2116934724 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fde200ce0b0 msgr2=0x7fde200cacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:43.251 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.239+0000 7fde269cf700 1 --2- 172.21.15.67:0/2116934724 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fde200ce0b0 0x7fde200cacd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:43.252 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.239+0000 7fde269cf700 1 -- 172.21.15.67:0/2116934724 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde18009a70 con 0x7fde2015bb50 2023-10-14T10:36:43.252 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.239+0000 7fde261ce700 1 --2- 172.21.15.67:0/2116934724 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fde2006ef20 0x7fde201510b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-14T10:36:43.252 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.239+0000 7fde259cd700 1 --2- 172.21.15.67:0/2116934724 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fde200ce0b0 0x7fde200cacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-14T10:36:43.252 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.239+0000 7fde269cf700 1 --2- 172.21.15.67:0/2116934724 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde2015bb50 0x7fde201525f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fde20070fb0 tx=0x7fde18004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:43.252 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.239+0000 7fde12ffd700 1 -- 172.21.15.67:0/2116934724 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde18012430 con 0x7fde2015bb50 2023-10-14T10:36:43.253 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.239+0000 7fde12ffd700 1 -- 172.21.15.67:0/2116934724 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde18020070 con 0x7fde2015bb50 2023-10-14T10:36:43.253 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.239+0000 7fde12ffd700 1 -- 172.21.15.67:0/2116934724 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde18004280 con 0x7fde2015bb50 2023-10-14T10:36:43.253 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.239+0000 7fde28432700 1 -- 172.21.15.67:0/2116934724 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde200cba30 con 0x7fde2015bb50 2023-10-14T10:36:43.253 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.239+0000 7fde28432700 1 -- 172.21.15.67:0/2116934724 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde201515f0 con 0x7fde2015bb50 2023-10-14T10:36:43.253 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.240+0000 7fde28432700 1 -- 172.21.15.67:0/2116934724 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde200cbc70 con 0x7fde2015bb50 2023-10-14T10:36:43.253 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.241+0000 7fde12ffd700 1 -- 172.21.15.67:0/2116934724 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fde180043e0 con 0x7fde2015bb50 2023-10-14T10:36:43.254 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.241+0000 7fde12ffd700 1 --2- 172.21.15.67:0/2116934724 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fde0406af00 0x7fde0406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:43.254 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.241+0000 7fde12ffd700 1 -- 172.21.15.67:0/2116934724 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fde18022070 con 0x7fde2015bb50 2023-10-14T10:36:43.254 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.241+0000 7fde261ce700 1 --2- 172.21.15.67:0/2116934724 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fde0406af00 0x7fde0406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:43.254 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.242+0000 7fde261ce700 1 --2- 172.21.15.67:0/2116934724 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fde0406af00 0x7fde0406d3c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fde14007fd0 tx=0x7fde14009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:43.255 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.246+0000 7fde12ffd700 1 -- 172.21.15.67:0/2116934724 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fde18056540 con 0x7fde2015bb50 2023-10-14T10:36:43.446 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.444+0000 7fde28432700 1 -- 172.21.15.67:0/2116934724 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fde200c8cd0 con 0x7fde2015bb50 2023-10-14T10:36:43.447 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.445+0000 7fde12ffd700 1 -- 172.21.15.67:0/2116934724 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7fde18059b60 con 0x7fde2015bb50 2023-10-14T10:36:43.447 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2023-10-14T10:36:43.449 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.447+0000 7fde28432700 1 -- 172.21.15.67:0/2116934724 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fde0406af00 msgr2=0x7fde0406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:43.450 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.447+0000 7fde28432700 1 --2- 172.21.15.67:0/2116934724 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fde0406af00 0x7fde0406d3c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fde14007fd0 tx=0x7fde14009040 comp rx=0 tx=0).stop 2023-10-14T10:36:43.450 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.447+0000 7fde28432700 1 -- 172.21.15.67:0/2116934724 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde2015bb50 msgr2=0x7fde201525f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:43.450 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.447+0000 7fde28432700 1 --2- 172.21.15.67:0/2116934724 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde2015bb50 0x7fde201525f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fde20070fb0 tx=0x7fde18004ad0 comp rx=0 tx=0).stop 2023-10-14T10:36:43.450 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.447+0000 7fde28432700 1 -- 172.21.15.67:0/2116934724 shutdown_connections 2023-10-14T10:36:43.450 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.447+0000 7fde28432700 1 --2- 172.21.15.67:0/2116934724 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fde0406af00 0x7fde0406d3c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:43.451 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.447+0000 7fde28432700 1 --2- 172.21.15.67:0/2116934724 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde2015bb50 0x7fde201525f0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:43.451 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.447+0000 7fde28432700 1 --2- 172.21.15.67:0/2116934724 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fde2006ef20 0x7fde201510b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:43.451 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.447+0000 7fde28432700 1 --2- 172.21.15.67:0/2116934724 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fde200ce0b0 0x7fde200cacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:43.451 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.447+0000 7fde28432700 1 -- 172.21.15.67:0/2116934724 >> 172.21.15.67:0/2116934724 conn(0x7fde200cc6f0 msgr2=0x7fde200b4a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:43.451 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.447+0000 7fde28432700 1 -- 172.21.15.67:0/2116934724 shutdown_connections 2023-10-14T10:36:43.452 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.447+0000 7fde28432700 1 -- 172.21.15.67:0/2116934724 wait complete. 2023-10-14T10:36:43.461 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-14T10:36:43.588 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.586+0000 7f9dee5bc700 1 Processor -- start 2023-10-14T10:36:43.589 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.586+0000 7f9dee5bc700 1 -- start start 2023-10-14T10:36:43.591 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.587+0000 7f9dee5bc700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9de80ce280 0x7f9de806d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:43.592 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.587+0000 7f9dee5bc700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9de806adb0 con 0x7f9de806a840 2023-10-14T10:36:43.592 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.587+0000 7f9dee5bc700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9de806b090 con 0x7f9de815e030 2023-10-14T10:36:43.592 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.587+0000 7f9dee5bc700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9de80cbab0 con 0x7f9de80ce280 2023-10-14T10:36:43.592 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.587+0000 7f9decb59700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9de80ce280 0x7f9de806d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:43.593 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.587+0000 7f9de77fe700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f9de806a840 0x7f9de80cdb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:53000/0 (socket says 172.21.15.67:53000) 2023-10-14T10:36:43.593 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.587+0000 7f9de77fe700 1 -- 172.21.15.67:0/2068775534 learned_addr learned my addr 172.21.15.67:0/2068775534 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:36:43.593 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.588+0000 7f9decb59700 1 -- 172.21.15.67:0/2068775534 >> v1:172.21.15.72:6789/0 conn(0x7f9de815e030 legacy=0x7f9de80cb3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:43.593 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.588+0000 7f9decb59700 1 -- 172.21.15.67:0/2068775534 >> v1:172.21.15.67:6789/0 conn(0x7f9de806a840 legacy=0x7f9de80cdb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:43.593 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.588+0000 7f9decb59700 1 -- 172.21.15.67:0/2068775534 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9de806b370 con 0x7f9de80ce280 2023-10-14T10:36:43.594 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.588+0000 7f9de6ffd700 1 -- 172.21.15.67:0/2068775534 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2922906308 0 0) 0x7f9de806adb0 con 0x7f9de806a840 2023-10-14T10:36:43.594 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.588+0000 7f9decb59700 1 --2- 172.21.15.67:0/2068775534 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9de80ce280 0x7f9de806d3d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f9de0009d90 tx=0x7f9de000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aeaefd760a5ee3d server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:43.594 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.588+0000 7f9de6ffd700 1 -- 172.21.15.67:0/2068775534 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de000e070 con 0x7f9de80ce280 2023-10-14T10:36:43.594 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.588+0000 7f9de6ffd700 1 -- 172.21.15.67:0/2068775534 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9de000bbd0 con 0x7f9de80ce280 2023-10-14T10:36:43.594 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.588+0000 7f9de6ffd700 1 -- 172.21.15.67:0/2068775534 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de0012430 con 0x7f9de80ce280 2023-10-14T10:36:43.595 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.588+0000 7f9dee5bc700 1 -- 172.21.15.67:0/2068775534 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9de80ce280 msgr2=0x7f9de806d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:43.595 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.588+0000 7f9dee5bc700 1 --2- 172.21.15.67:0/2068775534 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9de80ce280 0x7f9de806d3d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f9de0009d90 tx=0x7f9de000b330 comp rx=0 tx=0).stop 2023-10-14T10:36:43.595 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.588+0000 7f9dee5bc700 1 -- 172.21.15.67:0/2068775534 shutdown_connections 2023-10-14T10:36:43.595 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.588+0000 7f9dee5bc700 1 --2- 172.21.15.67:0/2068775534 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9de80ce280 0x7f9de806d3d0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:43.595 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.588+0000 7f9dee5bc700 1 -- 172.21.15.67:0/2068775534 >> 172.21.15.67:0/2068775534 conn(0x7f9de80cc6f0 msgr2=0x7f9de80ccb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:43.596 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.589+0000 7f9dee5bc700 1 -- 172.21.15.67:0/2068775534 shutdown_connections 2023-10-14T10:36:43.596 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.589+0000 7f9dee5bc700 1 -- 172.21.15.67:0/2068775534 wait complete. 2023-10-14T10:36:43.596 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.590+0000 7f9dee5bc700 1 Processor -- start 2023-10-14T10:36:43.599 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.590+0000 7f9dee5bc700 1 -- start start 2023-10-14T10:36:43.599 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.591+0000 7f9dee5bc700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9de806a840 0x7f9de80b3140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:43.599 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.591+0000 7f9dee5bc700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9de80ce280 0x7f9de80b3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:43.599 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.591+0000 7f9dee5bc700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f9de815e030 0x7f9de81f3350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:43.600 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.591+0000 7f9dee5bc700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9de80b43c0 con 0x7f9de80ce280 2023-10-14T10:36:43.600 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.591+0000 7f9dee5bc700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9de80b3bc0 con 0x7f9de806a840 2023-10-14T10:36:43.600 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.591+0000 7f9dee5bc700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9de80b3e00 con 0x7f9de815e030 2023-10-14T10:36:43.600 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.591+0000 7f9de77fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9de80ce280 0x7f9de80b3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:43.600 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.591+0000 7f9de77fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9de80ce280 0x7f9de80b3680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45238/0 (socket says 172.21.15.67:45238) 2023-10-14T10:36:43.601 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.591+0000 7f9de77fe700 1 -- 172.21.15.67:0/727561895 learned_addr learned my addr 172.21.15.67:0/727561895 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:43.601 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.591+0000 7f9decb59700 1 --2- 172.21.15.67:0/727561895 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f9de815e030 0x7f9de81f3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:43.601 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.591+0000 7f9de7fff700 1 --2- 172.21.15.67:0/727561895 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9de806a840 0x7f9de80b3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:43.601 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.591+0000 7f9de77fe700 1 -- 172.21.15.67:0/727561895 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f9de815e030 msgr2=0x7f9de81f3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:43.602 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.591+0000 7f9de77fe700 1 --2- 172.21.15.67:0/727561895 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f9de815e030 0x7f9de81f3350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:43.602 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.591+0000 7f9de77fe700 1 -- 172.21.15.67:0/727561895 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9de806a840 msgr2=0x7f9de80b3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:43.602 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.591+0000 7f9de77fe700 1 --2- 172.21.15.67:0/727561895 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9de806a840 0x7f9de80b3140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:43.602 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.591+0000 7f9de77fe700 1 -- 172.21.15.67:0/727561895 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9de0009a70 con 0x7f9de80ce280 2023-10-14T10:36:43.602 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.591+0000 7f9de77fe700 1 --2- 172.21.15.67:0/727561895 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9de80ce280 0x7f9de80b3680 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f9dd000c520 tx=0x7f9dd000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:43.603 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.591+0000 7f9de4ff9700 1 -- 172.21.15.67:0/727561895 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dd0004c80 con 0x7f9de80ce280 2023-10-14T10:36:43.603 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.591+0000 7f9de4ff9700 1 -- 172.21.15.67:0/727561895 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dd0004e00 con 0x7f9de80ce280 2023-10-14T10:36:43.603 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.591+0000 7f9dee5bc700 1 -- 172.21.15.67:0/727561895 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9de80b40e0 con 0x7f9de80ce280 2023-10-14T10:36:43.603 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.591+0000 7f9de4ff9700 1 -- 172.21.15.67:0/727561895 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dd0007310 con 0x7f9de80ce280 2023-10-14T10:36:43.603 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.592+0000 7f9dee5bc700 1 -- 172.21.15.67:0/727561895 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9de80c3c00 con 0x7f9de80ce280 2023-10-14T10:36:43.604 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.592+0000 7f9dee5bc700 1 -- 172.21.15.67:0/727561895 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9de80c3760 con 0x7f9de80ce280 2023-10-14T10:36:43.604 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.593+0000 7f9de4ff9700 1 -- 172.21.15.67:0/727561895 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f9dd0007470 con 0x7f9de80ce280 2023-10-14T10:36:43.604 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.593+0000 7f9de4ff9700 1 --2- 172.21.15.67:0/727561895 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f9dc806af00 0x7f9dc806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:43.604 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.593+0000 7f9de4ff9700 1 -- 172.21.15.67:0/727561895 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9dd0089350 con 0x7f9de80ce280 2023-10-14T10:36:43.604 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.594+0000 7f9de7fff700 1 --2- 172.21.15.67:0/727561895 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f9dc806af00 0x7f9dc806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:43.605 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.594+0000 7f9de7fff700 1 --2- 172.21.15.67:0/727561895 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f9dc806af00 0x7f9dc806d3c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9dd8000f60 tx=0x7f9dd8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:43.605 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.598+0000 7f9de4ff9700 1 -- 172.21.15.67:0/727561895 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9dd0054360 con 0x7f9de80ce280 2023-10-14T10:36:43.797 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.795+0000 7f9dee5bc700 1 -- 172.21.15.67:0/727561895 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9de8067d20 con 0x7f9de80ce280 2023-10-14T10:36:43.797 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.795+0000 7f9de4ff9700 1 -- 172.21.15.67:0/727561895 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9dd0057980 con 0x7f9de80ce280 2023-10-14T10:36:43.798 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2023-10-14T10:36:43.799 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.797+0000 7f9dee5bc700 1 -- 172.21.15.67:0/727561895 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f9dc806af00 msgr2=0x7f9dc806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:43.800 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.797+0000 7f9dee5bc700 1 --2- 172.21.15.67:0/727561895 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f9dc806af00 0x7f9dc806d3c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9dd8000f60 tx=0x7f9dd8009040 comp rx=0 tx=0).stop 2023-10-14T10:36:43.800 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.797+0000 7f9dee5bc700 1 -- 172.21.15.67:0/727561895 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9de80ce280 msgr2=0x7f9de80b3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:43.800 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.797+0000 7f9dee5bc700 1 --2- 172.21.15.67:0/727561895 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9de80ce280 0x7f9de80b3680 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f9dd000c520 tx=0x7f9dd000c8e0 comp rx=0 tx=0).stop 2023-10-14T10:36:43.800 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.797+0000 7f9dee5bc700 1 -- 172.21.15.67:0/727561895 shutdown_connections 2023-10-14T10:36:43.800 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.797+0000 7f9dee5bc700 1 --2- 172.21.15.67:0/727561895 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f9dc806af00 0x7f9dc806d3c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:43.801 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.797+0000 7f9dee5bc700 1 --2- 172.21.15.67:0/727561895 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f9de815e030 0x7f9de81f3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:43.801 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.797+0000 7f9dee5bc700 1 --2- 172.21.15.67:0/727561895 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9de806a840 0x7f9de80b3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:43.801 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.797+0000 7f9dee5bc700 1 --2- 172.21.15.67:0/727561895 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9de80ce280 0x7f9de80b3680 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:43.801 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.797+0000 7f9dee5bc700 1 -- 172.21.15.67:0/727561895 >> 172.21.15.67:0/727561895 conn(0x7f9de80cc6f0 msgr2=0x7f9de80cdb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:43.801 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.797+0000 7f9dee5bc700 1 -- 172.21.15.67:0/727561895 shutdown_connections 2023-10-14T10:36:43.801 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.797+0000 7f9dee5bc700 1 -- 172.21.15.67:0/727561895 wait complete. 2023-10-14T10:36:43.811 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-14T10:36:43.941 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.939+0000 7fd4f5330700 1 Processor -- start 2023-10-14T10:36:43.941 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.939+0000 7fd4f5330700 1 -- start start 2023-10-14T10:36:43.945 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.939+0000 7fd4f5330700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fd4f00d0c70 0x7fd4f00cdfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:43.945 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.940+0000 7fd4f5330700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4f00d23e0 con 0x7fd4f015b740 2023-10-14T10:36:43.945 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.940+0000 7fd4f5330700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4f00cf190 con 0x7fd4f00c5110 2023-10-14T10:36:43.946 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.940+0000 7fd4f5330700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4f015bb20 con 0x7fd4f00d0c70 2023-10-14T10:36:43.946 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.940+0000 7fd4eeffd700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fd4f015b740 0x7fd4f00d1cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:53012/0 (socket says 172.21.15.67:53012) 2023-10-14T10:36:43.946 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.940+0000 7fd4eeffd700 1 -- 172.21.15.67:0/419525414 learned_addr learned my addr 172.21.15.67:0/419525414 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:36:43.946 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.940+0000 7fd4edffb700 1 -- 172.21.15.67:0/419525414 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 970807315 0 0) 0x7fd4e0001110 con 0x7fd4f015b740 2023-10-14T10:36:43.946 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.940+0000 7fd4ee7fc700 1 --2- 172.21.15.67:0/419525414 >> v2:172.21.15.72:3300/0 conn(0x7fd4f00d0c70 0x7fd4f00cdfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:43.947 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.940+0000 7fd4edffb700 1 -- 172.21.15.67:0/419525414 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4d80035b0 con 0x7fd4f015b740 2023-10-14T10:36:43.947 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.940+0000 7fd4edffb700 1 -- 172.21.15.67:0/419525414 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 627868878 0 0) 0x7fd4d80035b0 con 0x7fd4f015b740 2023-10-14T10:36:43.947 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.940+0000 7fd4edffb700 1 -- 172.21.15.67:0/419525414 >> v2:172.21.15.72:3300/0 conn(0x7fd4f00d0c70 msgr2=0x7fd4f00cdfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:43.947 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.940+0000 7fd4edffb700 1 --2- 172.21.15.67:0/419525414 >> v2:172.21.15.72:3300/0 conn(0x7fd4f00d0c70 0x7fd4f00cdfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:43.947 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.940+0000 7fd4edffb700 1 -- 172.21.15.67:0/419525414 >> v1:172.21.15.72:6789/0 conn(0x7fd4f00c5110 legacy=0x7fd4f00cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:43.948 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.940+0000 7fd4edffb700 1 -- 172.21.15.67:0/419525414 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4f00cbd40 con 0x7fd4f015b740 2023-10-14T10:36:43.948 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.940+0000 7fd4edffb700 1 -- 172.21.15.67:0/419525414 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3180043755 0 0) 0x7fd4e0001800 con 0x7fd4f015b740 2023-10-14T10:36:43.948 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.940+0000 7fd4edffb700 1 -- 172.21.15.67:0/419525414 >> v1:172.21.15.67:6789/0 conn(0x7fd4f015b740 legacy=0x7fd4f00d1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:43.948 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.940+0000 7fd4edffb700 1 --2- 172.21.15.67:0/419525414 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd4d8003a00 0x7fd4d8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:43.948 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.940+0000 7fd4edffb700 1 --2- 172.21.15.67:0/419525414 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4d8004850 0x7fd4d8004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:43.949 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.940+0000 7fd4edffb700 1 --2- 172.21.15.67:0/419525414 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fd4d8005330 0x7fd4d800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:43.949 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.940+0000 7fd4edffb700 1 -- 172.21.15.67:0/419525414 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f015bb20 con 0x7fd4d8004850 2023-10-14T10:36:43.949 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.940+0000 7fd4edffb700 1 -- 172.21.15.67:0/419525414 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d800c200 con 0x7fd4d8003a00 2023-10-14T10:36:43.949 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.940+0000 7fd4edffb700 1 -- 172.21.15.67:0/419525414 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d800c3c0 con 0x7fd4d8005330 2023-10-14T10:36:43.949 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.940+0000 7fd4ee7fc700 1 --2- 172.21.15.67:0/419525414 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4d8004850 0x7fd4d8004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:43.950 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.941+0000 7fd4eeffd700 1 --2- 172.21.15.67:0/419525414 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd4d8003a00 0x7fd4d8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:43.950 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.941+0000 7fd4ef7fe700 1 --2- 172.21.15.67:0/419525414 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fd4d8005330 0x7fd4d800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:43.950 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.941+0000 7fd4ee7fc700 1 -- 172.21.15.67:0/419525414 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fd4d8005330 msgr2=0x7fd4d800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:43.950 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.941+0000 7fd4ee7fc700 1 --2- 172.21.15.67:0/419525414 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fd4d8005330 0x7fd4d800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:43.950 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.941+0000 7fd4ee7fc700 1 -- 172.21.15.67:0/419525414 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd4d8003a00 msgr2=0x7fd4d8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:43.951 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.941+0000 7fd4ee7fc700 1 --2- 172.21.15.67:0/419525414 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd4d8003a00 0x7fd4d8003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:43.951 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.941+0000 7fd4ef7fe700 1 -- 172.21.15.67:0/419525414 reap_dead start 2023-10-14T10:36:43.951 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.941+0000 7fd4ee7fc700 1 -- 172.21.15.67:0/419525414 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4d80035b0 con 0x7fd4d8004850 2023-10-14T10:36:43.951 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.941+0000 7fd4ee7fc700 1 --2- 172.21.15.67:0/419525414 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4d8004850 0x7fd4d8004cd0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd4e4009b30 tx=0x7fd4d8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93aaf3832a08cfe1 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:43.951 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.941+0000 7fd4edffb700 1 -- 172.21.15.67:0/419525414 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e400e6c0 con 0x7fd4d8004850 2023-10-14T10:36:43.952 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.941+0000 7fd4edffb700 1 -- 172.21.15.67:0/419525414 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4e400e820 con 0x7fd4d8004850 2023-10-14T10:36:43.952 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.941+0000 7fd4edffb700 1 -- 172.21.15.67:0/419525414 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e400eb90 con 0x7fd4d8004850 2023-10-14T10:36:43.952 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.941+0000 7fd4f5330700 1 -- 172.21.15.67:0/419525414 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4d8004850 msgr2=0x7fd4d8004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:43.952 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.941+0000 7fd4f5330700 1 --2- 172.21.15.67:0/419525414 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4d8004850 0x7fd4d8004cd0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd4e4009b30 tx=0x7fd4d8005880 comp rx=0 tx=0).stop 2023-10-14T10:36:43.952 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.942+0000 7fd4f5330700 1 -- 172.21.15.67:0/419525414 shutdown_connections 2023-10-14T10:36:43.953 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.942+0000 7fd4f5330700 1 --2- 172.21.15.67:0/419525414 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4d8004850 0x7fd4d8004cd0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:43.953 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.942+0000 7fd4f5330700 1 -- 172.21.15.67:0/419525414 >> 172.21.15.67:0/419525414 conn(0x7fd4f00665a0 msgr2=0x7fd4f0066980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:43.953 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.942+0000 7fd4f5330700 1 -- 172.21.15.67:0/419525414 shutdown_connections 2023-10-14T10:36:43.953 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.942+0000 7fd4f5330700 1 -- 172.21.15.67:0/419525414 wait complete. 2023-10-14T10:36:43.956 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.943+0000 7fd4f5330700 1 Processor -- start 2023-10-14T10:36:43.957 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.943+0000 7fd4f5330700 1 -- start start 2023-10-14T10:36:43.957 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.944+0000 7fd4f5330700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4d8004850 0x7fd4f01f0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:43.957 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.944+0000 7fd4f5330700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fd4f0152f80 0x7fd4f0153400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:43.957 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.944+0000 7fd4f5330700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd4f0155070 0x7fd4f01554f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:43.957 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.944+0000 7fd4f5330700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f01f0c90 con 0x7fd4d8004850 2023-10-14T10:36:43.958 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.944+0000 7fd4f5330700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f0153940 con 0x7fd4f0155070 2023-10-14T10:36:43.958 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.944+0000 7fd4f5330700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f0155ea0 con 0x7fd4f0152f80 2023-10-14T10:36:43.958 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.944+0000 7fd4eeffd700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4d8004850 0x7fd4f01f0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:43.958 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.944+0000 7fd4eeffd700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4d8004850 0x7fd4f01f0750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45256/0 (socket says 172.21.15.67:45256) 2023-10-14T10:36:43.958 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.944+0000 7fd4eeffd700 1 -- 172.21.15.67:0/698442112 learned_addr learned my addr 172.21.15.67:0/698442112 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:43.959 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.945+0000 7fd4ee7fc700 1 --2- 172.21.15.67:0/698442112 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fd4f0152f80 0x7fd4f0153400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:43.959 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.945+0000 7fd4ef7fe700 1 --2- 172.21.15.67:0/698442112 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd4f0155070 0x7fd4f01554f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:43.959 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.945+0000 7fd4eeffd700 1 -- 172.21.15.67:0/698442112 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fd4f0152f80 msgr2=0x7fd4f0153400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:43.959 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.945+0000 7fd4eeffd700 1 --2- 172.21.15.67:0/698442112 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fd4f0152f80 0x7fd4f0153400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:43.959 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.945+0000 7fd4eeffd700 1 -- 172.21.15.67:0/698442112 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd4f0155070 msgr2=0x7fd4f01554f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:43.960 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.945+0000 7fd4eeffd700 1 --2- 172.21.15.67:0/698442112 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd4f0155070 0x7fd4f01554f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:43.960 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.945+0000 7fd4eeffd700 1 -- 172.21.15.67:0/698442112 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4f0156180 con 0x7fd4d8004850 2023-10-14T10:36:43.960 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.945+0000 7fd4ee7fc700 1 --2- 172.21.15.67:0/698442112 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fd4f0152f80 0x7fd4f0153400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-14T10:36:43.960 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.945+0000 7fd4eeffd700 1 --2- 172.21.15.67:0/698442112 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4d8004850 0x7fd4f01f0750 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd4e0004f80 tx=0x7fd4e00073f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:43.960 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.945+0000 7fd4d77fe700 1 -- 172.21.15.67:0/698442112 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e0011580 con 0x7fd4d8004850 2023-10-14T10:36:43.961 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.945+0000 7fd4d77fe700 1 -- 172.21.15.67:0/698442112 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4e00116e0 con 0x7fd4d8004850 2023-10-14T10:36:43.961 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.945+0000 7fd4f5330700 1 -- 172.21.15.67:0/698442112 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4f0156460 con 0x7fd4d8004850 2023-10-14T10:36:43.961 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.945+0000 7fd4d77fe700 1 -- 172.21.15.67:0/698442112 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e0011a90 con 0x7fd4d8004850 2023-10-14T10:36:43.961 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.945+0000 7fd4f5330700 1 -- 172.21.15.67:0/698442112 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4f01509e0 con 0x7fd4d8004850 2023-10-14T10:36:43.961 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.946+0000 7fd4f5330700 1 -- 172.21.15.67:0/698442112 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4f0067d20 con 0x7fd4d8004850 2023-10-14T10:36:43.962 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.947+0000 7fd4d77fe700 1 -- 172.21.15.67:0/698442112 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd4e0011bf0 con 0x7fd4d8004850 2023-10-14T10:36:43.962 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.947+0000 7fd4d77fe700 1 --2- 172.21.15.67:0/698442112 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fd4dc06af00 0x7fd4dc06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:43.962 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.947+0000 7fd4d77fe700 1 -- 172.21.15.67:0/698442112 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd4e0089330 con 0x7fd4d8004850 2023-10-14T10:36:43.962 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.947+0000 7fd4ee7fc700 1 --2- 172.21.15.67:0/698442112 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fd4dc06af00 0x7fd4dc06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:43.962 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.948+0000 7fd4ee7fc700 1 --2- 172.21.15.67:0/698442112 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fd4dc06af00 0x7fd4dc06d3c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd4e40090b0 tx=0x7fd4e4008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:43.963 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:43.952+0000 7fd4d77fe700 1 -- 172.21.15.67:0/698442112 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd4e00543e0 con 0x7fd4d8004850 2023-10-14T10:36:44.155 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.153+0000 7fd4f5330700 1 -- 172.21.15.67:0/698442112 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd4f00d23e0 con 0x7fd4d8004850 2023-10-14T10:36:44.156 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.153+0000 7fd4d77fe700 1 -- 172.21.15.67:0/698442112 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd4f00d23e0 con 0x7fd4d8004850 2023-10-14T10:36:44.156 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2023-10-14T10:36:44.158 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.155+0000 7fd4f5330700 1 -- 172.21.15.67:0/698442112 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fd4dc06af00 msgr2=0x7fd4dc06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:44.158 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.155+0000 7fd4f5330700 1 --2- 172.21.15.67:0/698442112 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fd4dc06af00 0x7fd4dc06d3c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd4e40090b0 tx=0x7fd4e4008030 comp rx=0 tx=0).stop 2023-10-14T10:36:44.158 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.155+0000 7fd4f5330700 1 -- 172.21.15.67:0/698442112 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4d8004850 msgr2=0x7fd4f01f0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:44.159 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.155+0000 7fd4f5330700 1 --2- 172.21.15.67:0/698442112 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4d8004850 0x7fd4f01f0750 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd4e0004f80 tx=0x7fd4e00073f0 comp rx=0 tx=0).stop 2023-10-14T10:36:44.159 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.156+0000 7fd4f5330700 1 -- 172.21.15.67:0/698442112 shutdown_connections 2023-10-14T10:36:44.159 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.156+0000 7fd4f5330700 1 --2- 172.21.15.67:0/698442112 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fd4dc06af00 0x7fd4dc06d3c0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:44.161 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.156+0000 7fd4f5330700 1 --2- 172.21.15.67:0/698442112 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd4f0155070 0x7fd4f01554f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:44.161 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.156+0000 7fd4f5330700 1 --2- 172.21.15.67:0/698442112 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4d8004850 0x7fd4f01f0750 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:44.161 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.156+0000 7fd4f5330700 1 --2- 172.21.15.67:0/698442112 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fd4f0152f80 0x7fd4f0153400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:44.161 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.156+0000 7fd4f5330700 1 -- 172.21.15.67:0/698442112 >> 172.21.15.67:0/698442112 conn(0x7fd4f00665a0 msgr2=0x7fd4f00c7340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:44.162 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.156+0000 7fd4f5330700 1 -- 172.21.15.67:0/698442112 shutdown_connections 2023-10-14T10:36:44.162 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.156+0000 7fd4f5330700 1 -- 172.21.15.67:0/698442112 wait complete. 2023-10-14T10:36:44.171 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-14T10:36:44.298 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.296+0000 7fa16cff1700 1 Processor -- start 2023-10-14T10:36:44.298 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.296+0000 7fa16cff1700 1 -- start start 2023-10-14T10:36:44.301 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.297+0000 7fa16cff1700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fa1680cc3f0 0x7fa1680cc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:44.301 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.297+0000 7fa16cff1700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa168071ab0 con 0x7fa168161670 2023-10-14T10:36:44.301 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.297+0000 7fa16cff1700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa168161bc0 con 0x7fa16806af50 2023-10-14T10:36:44.301 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.297+0000 7fa16cff1700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa168161ea0 con 0x7fa1680cc3f0 2023-10-14T10:36:44.302 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.297+0000 7fa165d9b700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fa168161670 0x7fa16806a840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:53032/0 (socket says 172.21.15.67:53032) 2023-10-14T10:36:44.302 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.297+0000 7fa165d9b700 1 -- 172.21.15.67:0/148721374 learned_addr learned my addr 172.21.15.67:0/148721374 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:36:44.302 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.297+0000 7fa16659c700 1 --2- 172.21.15.67:0/148721374 >> v2:172.21.15.72:3300/0 conn(0x7fa1680cc3f0 0x7fa1680cc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:44.302 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.297+0000 7fa16559a700 1 -- 172.21.15.67:0/148721374 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2899963203 0 0) 0x7fa168071ab0 con 0x7fa168161670 2023-10-14T10:36:44.302 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.297+0000 7fa16559a700 1 -- 172.21.15.67:0/148721374 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1500035b0 con 0x7fa168161670 2023-10-14T10:36:44.303 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.297+0000 7fa16559a700 1 -- 172.21.15.67:0/148721374 <== mon.1 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3038119776 0 0) 0x7fa168161bc0 con 0x7fa16806af50 2023-10-14T10:36:44.303 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.297+0000 7fa16559a700 1 -- 172.21.15.67:0/148721374 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa168071ab0 con 0x7fa16806af50 2023-10-14T10:36:44.303 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.297+0000 7fa16559a700 1 -- 172.21.15.67:0/148721374 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3347177013 0 0) 0x7fa1500035b0 con 0x7fa168161670 2023-10-14T10:36:44.303 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.297+0000 7fa16659c700 1 -- 172.21.15.67:0/148721374 >> v1:172.21.15.72:6789/0 conn(0x7fa16806af50 legacy=0x7fa168071f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:44.303 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.297+0000 7fa16659c700 1 -- 172.21.15.67:0/148721374 >> v1:172.21.15.67:6789/0 conn(0x7fa168161670 legacy=0x7fa16806a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:44.304 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.297+0000 7fa16659c700 1 -- 172.21.15.67:0/148721374 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa168162180 con 0x7fa1680cc3f0 2023-10-14T10:36:44.304 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.298+0000 7fa16659c700 1 --2- 172.21.15.67:0/148721374 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa1680cc3f0 0x7fa1680cc7d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa158009840 tx=0x7fa158009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab6e88c59df356a0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:44.304 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.298+0000 7fa16559a700 1 -- 172.21.15.67:0/148721374 <== mon.1 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15c001670 con 0x7fa1680cc3f0 2023-10-14T10:36:44.304 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.298+0000 7fa16559a700 1 -- 172.21.15.67:0/148721374 <== mon.1 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa158004030 con 0x7fa1680cc3f0 2023-10-14T10:36:44.304 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.298+0000 7fa16559a700 1 -- 172.21.15.67:0/148721374 <== mon.1 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1580043c0 con 0x7fa1680cc3f0 2023-10-14T10:36:44.304 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.298+0000 7fa16cff1700 1 -- 172.21.15.67:0/148721374 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa1680cc3f0 msgr2=0x7fa1680cc7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:44.305 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.298+0000 7fa16cff1700 1 --2- 172.21.15.67:0/148721374 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa1680cc3f0 0x7fa1680cc7d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa158009840 tx=0x7fa158009b50 comp rx=0 tx=0).stop 2023-10-14T10:36:44.305 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.298+0000 7fa16cff1700 1 -- 172.21.15.67:0/148721374 shutdown_connections 2023-10-14T10:36:44.305 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.298+0000 7fa16cff1700 1 --2- 172.21.15.67:0/148721374 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa1680cc3f0 0x7fa1680cc7d0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:44.305 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.298+0000 7fa16cff1700 1 -- 172.21.15.67:0/148721374 >> 172.21.15.67:0/148721374 conn(0x7fa16806dac0 msgr2=0x7fa16806dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:44.305 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.298+0000 7fa16cff1700 1 -- 172.21.15.67:0/148721374 shutdown_connections 2023-10-14T10:36:44.306 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.298+0000 7fa16cff1700 1 -- 172.21.15.67:0/148721374 wait complete. 2023-10-14T10:36:44.308 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.300+0000 7fa16cff1700 1 Processor -- start 2023-10-14T10:36:44.309 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.300+0000 7fa16cff1700 1 -- start start 2023-10-14T10:36:44.309 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.300+0000 7fa16cff1700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fa16806af50 0x7fa1680c7e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:44.309 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.300+0000 7fa16cff1700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa1680cc3f0 0x7fa1680ad430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:44.309 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.300+0000 7fa16cff1700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa168161670 0x7fa1680caef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:44.309 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.301+0000 7fa16cff1700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1680ad010 con 0x7fa1680cc3f0 2023-10-14T10:36:44.310 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.301+0000 7fa16cff1700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa16814f410 con 0x7fa168161670 2023-10-14T10:36:44.310 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.301+0000 7fa16cff1700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1680c68c0 con 0x7fa16806af50 2023-10-14T10:36:44.310 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.301+0000 7fa165d9b700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa1680cc3f0 0x7fa1680ad430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:44.310 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.301+0000 7fa165d9b700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa1680cc3f0 0x7fa1680ad430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45272/0 (socket says 172.21.15.67:45272) 2023-10-14T10:36:44.310 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.301+0000 7fa165d9b700 1 -- 172.21.15.67:0/2448310737 learned_addr learned my addr 172.21.15.67:0/2448310737 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:44.311 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.301+0000 7fa16659c700 1 --2- 172.21.15.67:0/2448310737 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fa16806af50 0x7fa1680c7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:44.311 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.301+0000 7fa166d9d700 1 --2- 172.21.15.67:0/2448310737 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa168161670 0x7fa1680caef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:44.311 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.301+0000 7fa165d9b700 1 -- 172.21.15.67:0/2448310737 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fa16806af50 msgr2=0x7fa1680c7e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:44.311 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.301+0000 7fa165d9b700 1 --2- 172.21.15.67:0/2448310737 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fa16806af50 0x7fa1680c7e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:44.311 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.301+0000 7fa165d9b700 1 -- 172.21.15.67:0/2448310737 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa168161670 msgr2=0x7fa1680caef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:44.312 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.301+0000 7fa165d9b700 1 --2- 172.21.15.67:0/2448310737 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa168161670 0x7fa1680caef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:44.312 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.301+0000 7fa165d9b700 1 -- 172.21.15.67:0/2448310737 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1580094a0 con 0x7fa1680cc3f0 2023-10-14T10:36:44.312 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.301+0000 7fa16659c700 1 --2- 172.21.15.67:0/2448310737 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fa16806af50 0x7fa1680c7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-14T10:36:44.312 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.301+0000 7fa165d9b700 1 --2- 172.21.15.67:0/2448310737 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa1680cc3f0 0x7fa1680ad430 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa15c00c750 tx=0x7fa15c00cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:44.312 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.301+0000 7fa156ffd700 1 -- 172.21.15.67:0/2448310737 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15c007560 con 0x7fa1680cc3f0 2023-10-14T10:36:44.312 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.301+0000 7fa156ffd700 1 -- 172.21.15.67:0/2448310737 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa15c0076c0 con 0x7fa1680cc3f0 2023-10-14T10:36:44.313 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.301+0000 7fa16cff1700 1 -- 172.21.15.67:0/2448310737 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1680ad970 con 0x7fa1680cc3f0 2023-10-14T10:36:44.313 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.301+0000 7fa156ffd700 1 -- 172.21.15.67:0/2448310737 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15c004a70 con 0x7fa1680cc3f0 2023-10-14T10:36:44.313 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.301+0000 7fa16cff1700 1 -- 172.21.15.67:0/2448310737 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1680ac950 con 0x7fa1680cc3f0 2023-10-14T10:36:44.313 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.302+0000 7fa16cff1700 1 -- 172.21.15.67:0/2448310737 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1680ad150 con 0x7fa1680cc3f0 2023-10-14T10:36:44.313 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.303+0000 7fa156ffd700 1 -- 172.21.15.67:0/2448310737 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fa15c004bd0 con 0x7fa1680cc3f0 2023-10-14T10:36:44.314 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.303+0000 7fa156ffd700 1 --2- 172.21.15.67:0/2448310737 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fa14806af00 0x7fa14806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:44.314 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.303+0000 7fa156ffd700 1 -- 172.21.15.67:0/2448310737 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa15c088790 con 0x7fa1680cc3f0 2023-10-14T10:36:44.314 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.303+0000 7fa16659c700 1 --2- 172.21.15.67:0/2448310737 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fa14806af00 0x7fa14806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:44.314 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.304+0000 7fa16659c700 1 --2- 172.21.15.67:0/2448310737 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fa14806af00 0x7fa14806d3c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa1580090b0 tx=0x7fa158008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:44.315 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.308+0000 7fa156ffd700 1 -- 172.21.15.67:0/2448310737 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa1680ad150 con 0x7fa1680cc3f0 2023-10-14T10:36:44.525 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.523+0000 7fa16cff1700 1 -- 172.21.15.67:0/2448310737 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa1680ad150 con 0x7fa1680cc3f0 2023-10-14T10:36:44.526 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.524+0000 7fa156ffd700 1 -- 172.21.15.67:0/2448310737 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7fa1680ad150 con 0x7fa1680cc3f0 2023-10-14T10:36:44.528 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.526+0000 7fa16cff1700 1 -- 172.21.15.67:0/2448310737 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fa14806af00 msgr2=0x7fa14806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:44.528 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.526+0000 7fa16cff1700 1 --2- 172.21.15.67:0/2448310737 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fa14806af00 0x7fa14806d3c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa1580090b0 tx=0x7fa158008030 comp rx=0 tx=0).stop 2023-10-14T10:36:44.529 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.526+0000 7fa16cff1700 1 -- 172.21.15.67:0/2448310737 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa1680cc3f0 msgr2=0x7fa1680ad430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:44.529 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.526+0000 7fa16cff1700 1 --2- 172.21.15.67:0/2448310737 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa1680cc3f0 0x7fa1680ad430 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa15c00c750 tx=0x7fa15c00cb10 comp rx=0 tx=0).stop 2023-10-14T10:36:44.529 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.526+0000 7fa16cff1700 1 -- 172.21.15.67:0/2448310737 shutdown_connections 2023-10-14T10:36:44.529 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.526+0000 7fa16cff1700 1 --2- 172.21.15.67:0/2448310737 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fa14806af00 0x7fa14806d3c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:44.529 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.526+0000 7fa16cff1700 1 --2- 172.21.15.67:0/2448310737 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa168161670 0x7fa1680caef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:44.530 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.526+0000 7fa16cff1700 1 --2- 172.21.15.67:0/2448310737 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fa16806af50 0x7fa1680c7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:44.530 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.526+0000 7fa16cff1700 1 --2- 172.21.15.67:0/2448310737 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa1680cc3f0 0x7fa1680ad430 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:44.530 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.526+0000 7fa16cff1700 1 -- 172.21.15.67:0/2448310737 >> 172.21.15.67:0/2448310737 conn(0x7fa16806dac0 msgr2=0x7fa1680cc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:44.530 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.526+0000 7fa16cff1700 1 -- 172.21.15.67:0/2448310737 shutdown_connections 2023-10-14T10:36:44.531 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.526+0000 7fa16cff1700 1 -- 172.21.15.67:0/2448310737 wait complete. 2023-10-14T10:36:44.531 INFO:teuthology.orchestra.run.smithi067.stderr:listed 0 entries 2023-10-14T10:36:44.540 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2023-10-14T10:36:44.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:36:44.541 DEBUG:teuthology.orchestra.run.smithi072:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2023-10-14T10:36:44.596 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi072.front.sepia.ceph.com 2023-10-14T10:36:44.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:36:44.597 DEBUG:teuthology.orchestra.run.smithi072:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-14T10:36:44.615 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-14T10:36:44.616 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-14T10:36:44.741 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.738+0000 7f7d899b4700 1 Processor -- start 2023-10-14T10:36:44.741 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.738+0000 7f7d899b4700 1 -- start start 2023-10-14T10:36:44.745 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.739+0000 7f7d899b4700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f7d8415ab10 0x7f7d8415f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:44.745 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.739+0000 7f7d899b4700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d8415e7a0 con 0x7f7d840cfe30 2023-10-14T10:36:44.745 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.739+0000 7f7d899b4700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d8415f730 con 0x7f7d840b1f50 2023-10-14T10:36:44.745 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.739+0000 7f7d899b4700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d840b29a0 con 0x7f7d8415ab10 2023-10-14T10:36:44.745 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.739+0000 7f7d837fe700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f7d840cfe30 0x7f7d840cbf40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:53050/0 (socket says 172.21.15.67:53050) 2023-10-14T10:36:44.746 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.739+0000 7f7d837fe700 1 -- 172.21.15.67:0/1986419211 learned_addr learned my addr 172.21.15.67:0/1986419211 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:36:44.746 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.740+0000 7f7d82ffd700 1 --2- 172.21.15.67:0/1986419211 >> v2:172.21.15.72:3300/0 conn(0x7f7d8415ab10 0x7f7d8415f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:44.746 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.740+0000 7f7d81ffb700 1 -- 172.21.15.67:0/1986419211 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 212424626 0 0) 0x7f7d8415e7a0 con 0x7f7d840cfe30 2023-10-14T10:36:44.746 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.740+0000 7f7d81ffb700 1 -- 172.21.15.67:0/1986419211 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d6c0035b0 con 0x7f7d840cfe30 2023-10-14T10:36:44.746 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.740+0000 7f7d81ffb700 1 -- 172.21.15.67:0/1986419211 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1793135799 0 0) 0x7f7d6c0035b0 con 0x7f7d840cfe30 2023-10-14T10:36:44.747 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.740+0000 7f7d81ffb700 1 -- 172.21.15.67:0/1986419211 >> v2:172.21.15.72:3300/0 conn(0x7f7d8415ab10 msgr2=0x7f7d8415f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:44.747 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.740+0000 7f7d81ffb700 1 --2- 172.21.15.67:0/1986419211 >> v2:172.21.15.72:3300/0 conn(0x7f7d8415ab10 0x7f7d8415f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:44.747 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.740+0000 7f7d81ffb700 1 -- 172.21.15.67:0/1986419211 >> v1:172.21.15.72:6789/0 conn(0x7f7d840b1f50 legacy=0x7f7d840cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:44.747 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.740+0000 7f7d81ffb700 1 -- 172.21.15.67:0/1986419211 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d840b2c80 con 0x7f7d840cfe30 2023-10-14T10:36:44.747 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.740+0000 7f7d81ffb700 1 -- 172.21.15.67:0/1986419211 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3180043755 0 0) 0x7f7d7c0019f0 con 0x7f7d840cfe30 2023-10-14T10:36:44.748 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.740+0000 7f7d81ffb700 1 -- 172.21.15.67:0/1986419211 >> v1:172.21.15.67:6789/0 conn(0x7f7d840cfe30 legacy=0x7f7d840cbf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:44.748 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.740+0000 7f7d81ffb700 1 --2- 172.21.15.67:0/1986419211 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f7d6c003a00 0x7f7d6c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:44.748 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.740+0000 7f7d81ffb700 1 --2- 172.21.15.67:0/1986419211 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7d6c004400 0x7f7d6c0048d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:44.748 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.740+0000 7f7d81ffb700 1 --2- 172.21.15.67:0/1986419211 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d6c0050d0 0x7f7d6c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:44.748 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.740+0000 7f7d81ffb700 1 -- 172.21.15.67:0/1986419211 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d840b29a0 con 0x7f7d6c0050d0 2023-10-14T10:36:44.749 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.740+0000 7f7d81ffb700 1 -- 172.21.15.67:0/1986419211 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6c00c030 con 0x7f7d6c004400 2023-10-14T10:36:44.749 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.740+0000 7f7d81ffb700 1 -- 172.21.15.67:0/1986419211 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6c00c1f0 con 0x7f7d6c003a00 2023-10-14T10:36:44.749 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.740+0000 7f7d837fe700 1 --2- 172.21.15.67:0/1986419211 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d6c0050d0 0x7f7d6c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:44.749 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.740+0000 7f7d82ffd700 1 --2- 172.21.15.67:0/1986419211 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f7d6c003a00 0x7f7d6c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:44.749 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.740+0000 7f7d827fc700 1 --2- 172.21.15.67:0/1986419211 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7d6c004400 0x7f7d6c0048d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:44.750 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.741+0000 7f7d837fe700 1 -- 172.21.15.67:0/1986419211 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f7d6c003a00 msgr2=0x7f7d6c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:44.750 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.741+0000 7f7d837fe700 1 --2- 172.21.15.67:0/1986419211 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f7d6c003a00 0x7f7d6c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:44.750 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.741+0000 7f7d837fe700 1 -- 172.21.15.67:0/1986419211 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7d6c004400 msgr2=0x7f7d6c0048d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:44.750 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.741+0000 7f7d837fe700 1 --2- 172.21.15.67:0/1986419211 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7d6c004400 0x7f7d6c0048d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:44.750 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.741+0000 7f7d837fe700 1 -- 172.21.15.67:0/1986419211 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d6c0035b0 con 0x7f7d6c0050d0 2023-10-14T10:36:44.751 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.741+0000 7f7d837fe700 1 -- 172.21.15.67:0/1986419211 reap_dead start 2023-10-14T10:36:44.751 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.741+0000 7f7d837fe700 1 --2- 172.21.15.67:0/1986419211 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d6c0050d0 0x7f7d6c00ba10 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c00a810 tx=0x7f7d6c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ab234d0e88a9473 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:44.751 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.741+0000 7f7d81ffb700 1 -- 172.21.15.67:0/1986419211 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d7c012600 con 0x7f7d6c0050d0 2023-10-14T10:36:44.751 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.741+0000 7f7d81ffb700 1 -- 172.21.15.67:0/1986419211 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d7c012780 con 0x7f7d6c0050d0 2023-10-14T10:36:44.751 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.741+0000 7f7d81ffb700 1 -- 172.21.15.67:0/1986419211 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d7c012b30 con 0x7f7d6c0050d0 2023-10-14T10:36:44.752 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.741+0000 7f7d899b4700 1 -- 172.21.15.67:0/1986419211 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d6c0050d0 msgr2=0x7f7d6c00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:44.752 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.741+0000 7f7d899b4700 1 --2- 172.21.15.67:0/1986419211 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d6c0050d0 0x7f7d6c00ba10 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c00a810 tx=0x7f7d6c005880 comp rx=0 tx=0).stop 2023-10-14T10:36:44.752 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.741+0000 7f7d899b4700 1 -- 172.21.15.67:0/1986419211 shutdown_connections 2023-10-14T10:36:44.752 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.741+0000 7f7d899b4700 1 --2- 172.21.15.67:0/1986419211 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d6c0050d0 0x7f7d6c00ba10 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:44.752 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.741+0000 7f7d899b4700 1 -- 172.21.15.67:0/1986419211 >> 172.21.15.67:0/1986419211 conn(0x7f7d8406dae0 msgr2=0x7f7d8406def0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:44.753 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.741+0000 7f7d899b4700 1 -- 172.21.15.67:0/1986419211 shutdown_connections 2023-10-14T10:36:44.753 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.742+0000 7f7d899b4700 1 -- 172.21.15.67:0/1986419211 wait complete. 2023-10-14T10:36:44.753 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.743+0000 7f7d899b4700 1 Processor -- start 2023-10-14T10:36:44.753 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.743+0000 7f7d899b4700 1 -- start start 2023-10-14T10:36:44.756 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.744+0000 7f7d899b4700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7d6c0050d0 0x7f7d8414e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:44.756 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.744+0000 7f7d899b4700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f7d8414ed20 0x7f7d840b39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:44.756 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.744+0000 7f7d899b4700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d8415d440 0x7f7d84158600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:44.756 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.744+0000 7f7d899b4700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d8415c130 con 0x7f7d8415d440 2023-10-14T10:36:44.756 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.744+0000 7f7d899b4700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d8415c3a0 con 0x7f7d6c0050d0 2023-10-14T10:36:44.757 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.744+0000 7f7d899b4700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d8415c680 con 0x7f7d8414ed20 2023-10-14T10:36:44.757 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.744+0000 7f7d837fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d8415d440 0x7f7d84158600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:44.757 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.744+0000 7f7d837fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d8415d440 0x7f7d84158600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45294/0 (socket says 172.21.15.67:45294) 2023-10-14T10:36:44.757 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.744+0000 7f7d837fe700 1 -- 172.21.15.67:0/1353774013 learned_addr learned my addr 172.21.15.67:0/1353774013 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:44.757 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.744+0000 7f7d82ffd700 1 --2- 172.21.15.67:0/1353774013 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7d6c0050d0 0x7f7d8414e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:44.758 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.744+0000 7f7d827fc700 1 --2- 172.21.15.67:0/1353774013 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f7d8414ed20 0x7f7d840b39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:44.758 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.744+0000 7f7d827fc700 1 -- 172.21.15.67:0/1353774013 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7d6c0050d0 msgr2=0x7f7d8414e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:44.758 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.744+0000 7f7d827fc700 1 --2- 172.21.15.67:0/1353774013 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7d6c0050d0 0x7f7d8414e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:44.758 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.744+0000 7f7d827fc700 1 -- 172.21.15.67:0/1353774013 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d8415d440 msgr2=0x7f7d84158600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:44.758 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.744+0000 7f7d827fc700 1 --2- 172.21.15.67:0/1353774013 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d8415d440 0x7f7d84158600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:44.759 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.744+0000 7f7d827fc700 1 -- 172.21.15.67:0/1353774013 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d8415c960 con 0x7f7d8414ed20 2023-10-14T10:36:44.759 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.744+0000 7f7d827fc700 1 --2- 172.21.15.67:0/1353774013 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f7d8414ed20 0x7f7d840b39b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7d7400f7f0 tx=0x7f7d7400fb00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:44.759 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.745+0000 7f7d6bfff700 1 -- 172.21.15.67:0/1353774013 <== mon.2 v2:172.21.15.72:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d740053e0 con 0x7f7d8414ed20 2023-10-14T10:36:44.759 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.745+0000 7f7d6bfff700 1 -- 172.21.15.67:0/1353774013 <== mon.2 v2:172.21.15.72:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d74005560 con 0x7f7d8414ed20 2023-10-14T10:36:44.759 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.745+0000 7f7d899b4700 1 -- 172.21.15.67:0/1353774013 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d8415cc40 con 0x7f7d8414ed20 2023-10-14T10:36:44.760 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.745+0000 7f7d6bfff700 1 -- 172.21.15.67:0/1353774013 <== mon.2 v2:172.21.15.72:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d74006af0 con 0x7f7d8414ed20 2023-10-14T10:36:44.760 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.745+0000 7f7d899b4700 1 -- 172.21.15.67:0/1353774013 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d840b3ef0 con 0x7f7d8414ed20 2023-10-14T10:36:44.760 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.746+0000 7f7d899b4700 1 -- 172.21.15.67:0/1353774013 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d8415d160 con 0x7f7d8414ed20 2023-10-14T10:36:44.760 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.746+0000 7f7d6bfff700 1 -- 172.21.15.67:0/1353774013 <== mon.2 v2:172.21.15.72:3301/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f7d74006c50 con 0x7f7d8414ed20 2023-10-14T10:36:44.760 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.747+0000 7f7d6bfff700 1 --2- 172.21.15.67:0/1353774013 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f7d7006af00 0x7f7d7006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:44.761 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.747+0000 7f7d6bfff700 1 -- 172.21.15.67:0/1353774013 <== mon.2 v2:172.21.15.72:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7d74088c50 con 0x7f7d8414ed20 2023-10-14T10:36:44.761 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.747+0000 7f7d82ffd700 1 --2- 172.21.15.67:0/1353774013 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f7d7006af00 0x7f7d7006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:44.761 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.747+0000 7f7d82ffd700 1 --2- 172.21.15.67:0/1353774013 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f7d7006af00 0x7f7d7006d3c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7d78009be0 tx=0x7f7d78009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:44.761 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.752+0000 7f7d6bfff700 1 -- 172.21.15.67:0/1353774013 <== mon.2 v2:172.21.15.72:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7d8415d160 con 0x7f7d8414ed20 2023-10-14T10:36:44.999 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.997+0000 7f7d899b4700 1 -- 172.21.15.67:0/1353774013 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7d8415d160 con 0x7f7d8414ed20 2023-10-14T10:36:45.000 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:44.998+0000 7f7d6bfff700 1 -- 172.21.15.67:0/1353774013 <== mon.2 v2:172.21.15.72:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4901 (secure 0 0 0) 0x7f7d8415d160 con 0x7f7d8414ed20 2023-10-14T10:36:45.000 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-14T10:36:45.000 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4217,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6836/838298018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":838298018},{"type":"v1","addr":"172.21.15.72:6836","nonce":838298018}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4255,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/1687192726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6833","nonce":1687192726},{"type":"v1","addr":"172.21.15.67:6835","nonce":1687192726}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6834/3936231765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":3936231765},{"type":"v1","addr":"172.21.15.67:6834","nonce":3936231765}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-14T10:36:03.247976+0000","modified":"2023-10-14T10:36:06.283381+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4267},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4267":{"gid":4267,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.72:6837/757500960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6835","nonce":757500960},{"type":"v1","addr":"172.21.15.72:6837","nonce":757500960}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-14T10:36:45.002 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.000+0000 7f7d899b4700 1 -- 172.21.15.67:0/1353774013 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f7d7006af00 msgr2=0x7f7d7006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:45.003 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.000+0000 7f7d899b4700 1 --2- 172.21.15.67:0/1353774013 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f7d7006af00 0x7f7d7006d3c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7d78009be0 tx=0x7f7d78009210 comp rx=0 tx=0).stop 2023-10-14T10:36:45.003 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.000+0000 7f7d899b4700 1 -- 172.21.15.67:0/1353774013 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f7d8414ed20 msgr2=0x7f7d840b39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:45.003 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.000+0000 7f7d899b4700 1 --2- 172.21.15.67:0/1353774013 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f7d8414ed20 0x7f7d840b39b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7d7400f7f0 tx=0x7f7d7400fb00 comp rx=0 tx=0).stop 2023-10-14T10:36:45.003 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.000+0000 7f7d899b4700 1 -- 172.21.15.67:0/1353774013 shutdown_connections 2023-10-14T10:36:45.004 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.000+0000 7f7d899b4700 1 --2- 172.21.15.67:0/1353774013 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f7d7006af00 0x7f7d7006d3c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:45.004 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.000+0000 7f7d899b4700 1 --2- 172.21.15.67:0/1353774013 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d8415d440 0x7f7d84158600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:45.004 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.000+0000 7f7d899b4700 1 --2- 172.21.15.67:0/1353774013 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7d6c0050d0 0x7f7d8414e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:45.004 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.000+0000 7f7d899b4700 1 --2- 172.21.15.67:0/1353774013 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f7d8414ed20 0x7f7d840b39b0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:45.004 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.000+0000 7f7d899b4700 1 -- 172.21.15.67:0/1353774013 >> 172.21.15.67:0/1353774013 conn(0x7f7d8406dae0 msgr2=0x7f7d840cd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:45.005 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.000+0000 7f7d899b4700 1 -- 172.21.15.67:0/1353774013 shutdown_connections 2023-10-14T10:36:45.005 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.000+0000 7f7d899b4700 1 -- 172.21.15.67:0/1353774013 wait complete. 2023-10-14T10:36:45.005 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 8 2023-10-14T10:36:45.015 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-14T10:36:45.140 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.138+0000 7f394e919700 1 Processor -- start 2023-10-14T10:36:45.141 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.138+0000 7f394e919700 1 -- start start 2023-10-14T10:36:45.144 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.139+0000 7f394e919700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f39480cfe30 0x7f39480cbf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:45.144 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.139+0000 7f394e919700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f394815e7a0 con 0x7f394815ab10 2023-10-14T10:36:45.145 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.139+0000 7f394e919700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39480b29a0 con 0x7f394815f1f0 2023-10-14T10:36:45.145 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.139+0000 7f394e919700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39480b2c80 con 0x7f39480cfe30 2023-10-14T10:36:45.145 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.139+0000 7f3947fff700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f394815ab10 0x7f39480b1f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:53066/0 (socket says 172.21.15.67:53066) 2023-10-14T10:36:45.145 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.139+0000 7f3947fff700 1 -- 172.21.15.67:0/4261069110 learned_addr learned my addr 172.21.15.67:0/4261069110 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:36:45.145 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.139+0000 7f394ceb6700 1 --2- 172.21.15.67:0/4261069110 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f39480cfe30 0x7f39480cbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:45.146 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.139+0000 7f3946ffd700 1 -- 172.21.15.67:0/4261069110 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2226426276 0 0) 0x7f394815e7a0 con 0x7f394815ab10 2023-10-14T10:36:45.146 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.139+0000 7f3946ffd700 1 -- 172.21.15.67:0/4261069110 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39340035b0 con 0x7f394815ab10 2023-10-14T10:36:45.146 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.140+0000 7f3946ffd700 1 -- 172.21.15.67:0/4261069110 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2913713837 0 0) 0x7f39340035b0 con 0x7f394815ab10 2023-10-14T10:36:45.146 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.140+0000 7f3946ffd700 1 -- 172.21.15.67:0/4261069110 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f39480cfe30 msgr2=0x7f39480cbf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:45.146 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.140+0000 7f3946ffd700 1 --2- 172.21.15.67:0/4261069110 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f39480cfe30 0x7f39480cbf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:45.147 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.140+0000 7f3946ffd700 1 -- 172.21.15.67:0/4261069110 >> v1:172.21.15.72:6789/0 conn(0x7f394815f1f0 legacy=0x7f39480cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:45.147 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.140+0000 7f3946ffd700 1 -- 172.21.15.67:0/4261069110 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f394815e9f0 con 0x7f394815ab10 2023-10-14T10:36:45.147 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.140+0000 7f3946ffd700 1 -- 172.21.15.67:0/4261069110 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3180043755 0 0) 0x7f3938001670 con 0x7f394815ab10 2023-10-14T10:36:45.147 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.140+0000 7f3946ffd700 1 -- 172.21.15.67:0/4261069110 >> v1:172.21.15.67:6789/0 conn(0x7f394815ab10 legacy=0x7f39480b1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:45.148 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.140+0000 7f3946ffd700 1 --2- 172.21.15.67:0/4261069110 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3934003a30 0x7f3934003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:45.148 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.140+0000 7f3946ffd700 1 --2- 172.21.15.67:0/4261069110 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3934004850 0x7f3934004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:45.148 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.140+0000 7f3946ffd700 1 --2- 172.21.15.67:0/4261069110 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f39480cfe30 0x7f3934005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:45.148 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.140+0000 7f3946ffd700 1 -- 172.21.15.67:0/4261069110 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39480b2c80 con 0x7f3934004850 2023-10-14T10:36:45.148 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.140+0000 7f3946ffd700 1 -- 172.21.15.67:0/4261069110 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f393400baf0 con 0x7f3934003a30 2023-10-14T10:36:45.148 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.140+0000 7f3946ffd700 1 -- 172.21.15.67:0/4261069110 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f393400bcb0 con 0x7f39480cfe30 2023-10-14T10:36:45.149 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.140+0000 7f39477fe700 1 --2- 172.21.15.67:0/4261069110 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3934004850 0x7f3934004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:45.149 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.140+0000 7f3947fff700 1 --2- 172.21.15.67:0/4261069110 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3934003a30 0x7f3934003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:45.149 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.140+0000 7f394ceb6700 1 --2- 172.21.15.67:0/4261069110 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f39480cfe30 0x7f3934005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:45.149 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.140+0000 7f39477fe700 1 -- 172.21.15.67:0/4261069110 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f39480cfe30 msgr2=0x7f3934005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:45.149 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.140+0000 7f39477fe700 1 --2- 172.21.15.67:0/4261069110 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f39480cfe30 0x7f3934005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:45.150 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.140+0000 7f39477fe700 1 -- 172.21.15.67:0/4261069110 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3934003a30 msgr2=0x7f3934003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:45.150 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.140+0000 7f39477fe700 1 --2- 172.21.15.67:0/4261069110 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3934003a30 0x7f3934003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:45.150 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.140+0000 7f39477fe700 1 -- 172.21.15.67:0/4261069110 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39340035b0 con 0x7f3934004850 2023-10-14T10:36:45.150 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.140+0000 7f39477fe700 1 --2- 172.21.15.67:0/4261069110 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3934004850 0x7f3934004cd0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3930001400 tx=0x7f3934005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4096b84acbe0ade2 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:45.150 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.140+0000 7f3946ffd700 1 -- 172.21.15.67:0/4261069110 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39300052d0 con 0x7f3934004850 2023-10-14T10:36:45.151 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.141+0000 7f3946ffd700 1 -- 172.21.15.67:0/4261069110 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3930005430 con 0x7f3934004850 2023-10-14T10:36:45.151 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.141+0000 7f3946ffd700 1 -- 172.21.15.67:0/4261069110 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3930003910 con 0x7f3934004850 2023-10-14T10:36:45.151 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.141+0000 7f394e919700 1 -- 172.21.15.67:0/4261069110 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3934004850 msgr2=0x7f3934004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:45.151 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.141+0000 7f394e919700 1 --2- 172.21.15.67:0/4261069110 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3934004850 0x7f3934004cd0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3930001400 tx=0x7f3934005880 comp rx=0 tx=0).stop 2023-10-14T10:36:45.151 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.141+0000 7f394ceb6700 1 -- 172.21.15.67:0/4261069110 reap_dead start 2023-10-14T10:36:45.152 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.141+0000 7f394e919700 1 -- 172.21.15.67:0/4261069110 shutdown_connections 2023-10-14T10:36:45.152 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.141+0000 7f394e919700 1 -- 172.21.15.67:0/4261069110 >> 172.21.15.67:0/4261069110 conn(0x7f394806dae0 msgr2=0x7f394806def0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:45.152 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.141+0000 7f394e919700 1 -- 172.21.15.67:0/4261069110 shutdown_connections 2023-10-14T10:36:45.152 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.141+0000 7f394e919700 1 -- 172.21.15.67:0/4261069110 wait complete. 2023-10-14T10:36:45.152 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.142+0000 7f394e919700 1 Processor -- start 2023-10-14T10:36:45.153 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.143+0000 7f394e919700 1 -- start start 2023-10-14T10:36:45.155 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.143+0000 7f394e919700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3934004850 0x7f39480c6670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:45.156 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.143+0000 7f394e919700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f39480c6bb0 0x7f39481f2760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:45.156 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.143+0000 7f394e919700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f39481f2ca0 0x7f39481ee5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:45.156 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.143+0000 7f394e919700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39481f3150 con 0x7f3934004850 2023-10-14T10:36:45.156 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.143+0000 7f394e919700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39480ca920 con 0x7f39481f2ca0 2023-10-14T10:36:45.156 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.143+0000 7f394e919700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39480ca080 con 0x7f39480c6bb0 2023-10-14T10:36:45.156 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.143+0000 7f3947fff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3934004850 0x7f39480c6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:45.157 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.143+0000 7f3947fff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3934004850 0x7f39480c6670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45310/0 (socket says 172.21.15.67:45310) 2023-10-14T10:36:45.157 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.143+0000 7f3947fff700 1 -- 172.21.15.67:0/3199831204 learned_addr learned my addr 172.21.15.67:0/3199831204 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:45.157 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.144+0000 7f39477fe700 1 --2- 172.21.15.67:0/3199831204 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f39480c6bb0 0x7f39481f2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:45.157 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.144+0000 7f394ceb6700 1 --2- 172.21.15.67:0/3199831204 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f39481f2ca0 0x7f39481ee5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:45.157 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.144+0000 7f3947fff700 1 -- 172.21.15.67:0/3199831204 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f39480c6bb0 msgr2=0x7f39481f2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:45.158 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.144+0000 7f3947fff700 1 --2- 172.21.15.67:0/3199831204 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f39480c6bb0 0x7f39481f2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:45.158 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.144+0000 7f3947fff700 1 -- 172.21.15.67:0/3199831204 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f39481f2ca0 msgr2=0x7f39481ee5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:45.158 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.144+0000 7f3947fff700 1 --2- 172.21.15.67:0/3199831204 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f39481f2ca0 0x7f39481ee5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:45.158 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.144+0000 7f3947fff700 1 -- 172.21.15.67:0/3199831204 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39480ca360 con 0x7f3934004850 2023-10-14T10:36:45.158 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.144+0000 7f394ceb6700 1 --2- 172.21.15.67:0/3199831204 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f39481f2ca0 0x7f39481ee5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-14T10:36:45.158 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.144+0000 7f3947fff700 1 --2- 172.21.15.67:0/3199831204 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3934004850 0x7f39480c6670 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f39380072c0 tx=0x7f39380075d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:45.159 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.144+0000 7f3944ff9700 1 -- 172.21.15.67:0/3199831204 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f393800f600 con 0x7f3934004850 2023-10-14T10:36:45.159 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.144+0000 7f3944ff9700 1 -- 172.21.15.67:0/3199831204 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f393800f760 con 0x7f3934004850 2023-10-14T10:36:45.159 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.144+0000 7f394e919700 1 -- 172.21.15.67:0/3199831204 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39480ca640 con 0x7f3934004850 2023-10-14T10:36:45.159 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.144+0000 7f3944ff9700 1 -- 172.21.15.67:0/3199831204 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f393800fb10 con 0x7f3934004850 2023-10-14T10:36:45.159 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.144+0000 7f394e919700 1 -- 172.21.15.67:0/3199831204 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39481eeb10 con 0x7f3934004850 2023-10-14T10:36:45.160 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.145+0000 7f394e919700 1 -- 172.21.15.67:0/3199831204 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f394815c300 con 0x7f3934004850 2023-10-14T10:36:45.160 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.145+0000 7f3944ff9700 1 -- 172.21.15.67:0/3199831204 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f393800fc70 con 0x7f3934004850 2023-10-14T10:36:45.160 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.146+0000 7f3944ff9700 1 --2- 172.21.15.67:0/3199831204 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f392806af00 0x7f392806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:45.160 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.146+0000 7f3944ff9700 1 -- 172.21.15.67:0/3199831204 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3938088f60 con 0x7f3934004850 2023-10-14T10:36:45.160 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.146+0000 7f39477fe700 1 --2- 172.21.15.67:0/3199831204 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f392806af00 0x7f392806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:45.161 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.146+0000 7f39477fe700 1 --2- 172.21.15.67:0/3199831204 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f392806af00 0x7f392806d3c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3930001040 tx=0x7f393001f040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:45.161 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.150+0000 7f3944ff9700 1 -- 172.21.15.67:0/3199831204 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f394815c300 con 0x7f3934004850 2023-10-14T10:36:45.352 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.350+0000 7f394e919700 1 -- 172.21.15.67:0/3199831204 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f394815c300 con 0x7f3934004850 2023-10-14T10:36:45.354 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.352+0000 7f3944ff9700 1 -- 172.21.15.67:0/3199831204 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16457 (secure 0 0 0) 0x7f394815c300 con 0x7f3934004850 2023-10-14T10:36:45.354 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-14T10:36:45.354 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":22,"fsid":"37f9ab46-3082-4b71-8b24-08d39897b008","created":"2023-10-14T10:35:46.067554+0000","modified":"2023-10-14T10:36:04.837296+0000","last_up_change":"2023-10-14T10:35:56.668142+0000","last_in_change":"2023-10-14T10:35:48.551490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-14T10:35:57.809566+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-14T10:36:01.667146+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-14T10:36:02.219700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2b59a07e-11ef-440c-b766-9f64ac885838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6801","nonce":1287255499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6803","nonce":1287255499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6807","nonce":1287255499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6805","nonce":1287255499}]},"public_addr":"172.21.15.67:6801/1287255499","cluster_addr":"172.21.15.67:6803/1287255499","heartbeat_back_addr":"172.21.15.67:6807/1287255499","heartbeat_front_addr":"172.21.15.67:6805/1287255499","state":["exists","up"]},{"osd":1,"uuid":"6f813bd6-fa0b-487d-a204-431eb1276a60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6818","nonce":2435538787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6822","nonce":2435538787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6830","nonce":2435538787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6826","nonce":2435538787}]},"public_addr":"172.21.15.67:6818/2435538787","cluster_addr":"172.21.15.67:6822/2435538787","heartbeat_back_addr":"172.21.15.67:6830/2435538787","heartbeat_front_addr":"172.21.15.67:6826/2435538787","state":["exists","up"]},{"osd":2,"uuid":"6af42c6a-1b59-4669-9c04-b6876788446d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6809","nonce":2827491791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6811","nonce":2827491791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6815","nonce":2827491791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6813","nonce":2827491791}]},"public_addr":"172.21.15.67:6809/2827491791","cluster_addr":"172.21.15.67:6811/2827491791","heartbeat_back_addr":"172.21.15.67:6815/2827491791","heartbeat_front_addr":"172.21.15.67:6813/2827491791","state":["exists","up"]},{"osd":3,"uuid":"1d889d76-a646-499d-bfff-ed5bae5434ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6819","nonce":1848001176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6821","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6823","nonce":1848001176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6831","nonce":1848001176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6825","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6827","nonce":1848001176}]},"public_addr":"172.21.15.67:6819/1848001176","cluster_addr":"172.21.15.67:6823/1848001176","heartbeat_back_addr":"172.21.15.67:6831/1848001176","heartbeat_front_addr":"172.21.15.67:6827/1848001176","state":["exists","up"]},{"osd":4,"uuid":"c3271dff-b437-4fdd-b13b-6e49a35473af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6804","nonce":1891230785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6808","nonce":1891230785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6815","nonce":1891230785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6811","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6813","nonce":1891230785}]},"public_addr":"172.21.15.72:6804/1891230785","cluster_addr":"172.21.15.72:6808/1891230785","heartbeat_back_addr":"172.21.15.72:6815/1891230785","heartbeat_front_addr":"172.21.15.72:6813/1891230785","state":["exists","up"]},{"osd":5,"uuid":"0337bb18-4b68-4eff-b528-a667275e6926","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6801","nonce":569686991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6803","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6805","nonce":569686991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6812","nonce":569686991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6807","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6809","nonce":569686991}]},"public_addr":"172.21.15.72:6801/569686991","cluster_addr":"172.21.15.72:6805/569686991","heartbeat_back_addr":"172.21.15.72:6812/569686991","heartbeat_front_addr":"172.21.15.72:6809/569686991","state":["exists","up"]},{"osd":6,"uuid":"8e92d9cf-0c8c-46eb-89a7-db2023ef1509","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6817","nonce":3767685934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6819","nonce":3767685934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6823","nonce":3767685934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6821","nonce":3767685934}]},"public_addr":"172.21.15.72:6817/3767685934","cluster_addr":"172.21.15.72:6819/3767685934","heartbeat_back_addr":"172.21.15.72:6823/3767685934","heartbeat_front_addr":"172.21.15.72:6821/3767685934","state":["exists","up"]},{"osd":7,"uuid":"3fd6aa7d-50a9-45c9-9561-5902a044a119","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6825","nonce":3567465926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6827","nonce":3567465926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6831","nonce":3567465926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6829","nonce":3567465926}]},"public_addr":"172.21.15.72:6825/3567465926","cluster_addr":"172.21.15.72:6827/3567465926","heartbeat_back_addr":"172.21.15.72:6831/3567465926","heartbeat_front_addr":"172.21.15.72:6829/3567465926","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.511466+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.795107+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.858674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.728895+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.794842+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.750064+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.732122+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.765185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-14T10:36:45.356 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.354+0000 7f394e919700 1 -- 172.21.15.67:0/3199831204 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f392806af00 msgr2=0x7f392806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:45.356 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.354+0000 7f394e919700 1 --2- 172.21.15.67:0/3199831204 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f392806af00 0x7f392806d3c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3930001040 tx=0x7f393001f040 comp rx=0 tx=0).stop 2023-10-14T10:36:45.357 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.354+0000 7f394e919700 1 -- 172.21.15.67:0/3199831204 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3934004850 msgr2=0x7f39480c6670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:45.357 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.354+0000 7f394e919700 1 --2- 172.21.15.67:0/3199831204 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3934004850 0x7f39480c6670 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f39380072c0 tx=0x7f39380075d0 comp rx=0 tx=0).stop 2023-10-14T10:36:45.357 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.354+0000 7f394e919700 1 -- 172.21.15.67:0/3199831204 shutdown_connections 2023-10-14T10:36:45.357 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.354+0000 7f394e919700 1 --2- 172.21.15.67:0/3199831204 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f392806af00 0x7f392806d3c0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:45.357 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.354+0000 7f394e919700 1 --2- 172.21.15.67:0/3199831204 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f39481f2ca0 0x7f39481ee5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:45.358 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.354+0000 7f394e919700 1 --2- 172.21.15.67:0/3199831204 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3934004850 0x7f39480c6670 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:45.358 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.354+0000 7f394e919700 1 --2- 172.21.15.67:0/3199831204 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f39480c6bb0 0x7f39481f2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:45.358 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.354+0000 7f394e919700 1 -- 172.21.15.67:0/3199831204 >> 172.21.15.67:0/3199831204 conn(0x7f394806dae0 msgr2=0x7f394806def0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:45.358 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.354+0000 7f394e919700 1 -- 172.21.15.67:0/3199831204 shutdown_connections 2023-10-14T10:36:45.358 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.354+0000 7f394e919700 1 -- 172.21.15.67:0/3199831204 wait complete. 2023-10-14T10:36:45.368 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-14T10:35:57.809566+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-14T10:36:01.667146+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-14T10:36:02.219700+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-14T10:36:45.369 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-14T10:36:45.497 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.495+0000 7fe873c76700 1 Processor -- start 2023-10-14T10:36:45.497 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.495+0000 7fe873c76700 1 -- start start 2023-10-14T10:36:45.501 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.496+0000 7fe873c76700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fe86c15ab10 0x7fe86c15f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:45.501 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.496+0000 7fe873c76700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fe86c0cfe30 0x7fe86c0cbf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:45.501 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.496+0000 7fe873c76700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe86c15e7a0 con 0x7fe86c0b1f50 2023-10-14T10:36:45.502 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.496+0000 7fe873c76700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe86c15f730 con 0x7fe86c0cfe30 2023-10-14T10:36:45.502 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.496+0000 7fe873c76700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe86c0b29a0 con 0x7fe86c15ab10 2023-10-14T10:36:45.502 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.496+0000 7fe871211700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fe86c0b1f50 0x7fe86c0cf720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:53086/0 (socket says 172.21.15.67:53086) 2023-10-14T10:36:45.502 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.496+0000 7fe871211700 1 -- 172.21.15.67:0/541447374 learned_addr learned my addr 172.21.15.67:0/541447374 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:36:45.502 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.496+0000 7fe872213700 1 --2- 172.21.15.67:0/541447374 >> v2:172.21.15.72:3300/0 conn(0x7fe86c0cfe30 0x7fe86c0cbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:45.503 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.496+0000 7fe870a10700 1 -- 172.21.15.67:0/541447374 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2662565543 0 0) 0x7fe86c15e7a0 con 0x7fe86c0b1f50 2023-10-14T10:36:45.503 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.496+0000 7fe871a12700 1 --2- 172.21.15.67:0/541447374 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fe86c15ab10 0x7fe86c15f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:45.503 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.496+0000 7fe870a10700 1 -- 172.21.15.67:0/541447374 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8580035b0 con 0x7fe86c0b1f50 2023-10-14T10:36:45.503 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.497+0000 7fe870a10700 1 -- 172.21.15.67:0/541447374 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3966801615 0 0) 0x7fe8580035b0 con 0x7fe86c0b1f50 2023-10-14T10:36:45.503 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.497+0000 7fe870a10700 1 -- 172.21.15.67:0/541447374 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fe86c15ab10 msgr2=0x7fe86c15f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:45.504 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.497+0000 7fe870a10700 1 --2- 172.21.15.67:0/541447374 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fe86c15ab10 0x7fe86c15f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:45.504 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.497+0000 7fe870a10700 1 -- 172.21.15.67:0/541447374 >> v2:172.21.15.72:3300/0 conn(0x7fe86c0cfe30 msgr2=0x7fe86c0cbf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:45.504 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.497+0000 7fe870a10700 1 --2- 172.21.15.67:0/541447374 >> v2:172.21.15.72:3300/0 conn(0x7fe86c0cfe30 0x7fe86c0cbf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:45.504 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.497+0000 7fe870a10700 1 -- 172.21.15.67:0/541447374 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe86c0b2c80 con 0x7fe86c0b1f50 2023-10-14T10:36:45.504 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.497+0000 7fe870a10700 1 -- 172.21.15.67:0/541447374 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3180043755 0 0) 0x7fe8680017f0 con 0x7fe86c0b1f50 2023-10-14T10:36:45.505 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.497+0000 7fe872213700 1 --2- 172.21.15.67:0/541447374 >> v2:172.21.15.72:3300/0 conn(0x7fe86c0cfe30 0x7fe86c0cbf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-14T10:36:45.505 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.497+0000 7fe870a10700 1 -- 172.21.15.67:0/541447374 >> v1:172.21.15.67:6789/0 conn(0x7fe86c0b1f50 legacy=0x7fe86c0cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:45.505 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.497+0000 7fe870a10700 1 --2- 172.21.15.67:0/541447374 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe858003a00 0x7fe858003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:45.505 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.497+0000 7fe870a10700 1 --2- 172.21.15.67:0/541447374 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe858004850 0x7fe858004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:45.505 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.497+0000 7fe870a10700 1 --2- 172.21.15.67:0/541447374 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fe86c15ab10 0x7fe858005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:45.506 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.497+0000 7fe870a10700 1 -- 172.21.15.67:0/541447374 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe86c15f730 con 0x7fe858003a00 2023-10-14T10:36:45.506 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.497+0000 7fe870a10700 1 -- 172.21.15.67:0/541447374 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe86c0b29a0 con 0x7fe858004850 2023-10-14T10:36:45.506 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.497+0000 7fe870a10700 1 -- 172.21.15.67:0/541447374 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe85800ba10 con 0x7fe86c15ab10 2023-10-14T10:36:45.506 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.497+0000 7fe871a12700 1 --2- 172.21.15.67:0/541447374 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe858003a00 0x7fe858003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:45.506 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.497+0000 7fe871211700 1 --2- 172.21.15.67:0/541447374 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe858004850 0x7fe858004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:45.507 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.497+0000 7fe872213700 1 --2- 172.21.15.67:0/541447374 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fe86c15ab10 0x7fe858005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:45.507 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.497+0000 7fe871a12700 1 -- 172.21.15.67:0/541447374 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fe86c15ab10 msgr2=0x7fe858005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:45.507 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.497+0000 7fe871a12700 1 --2- 172.21.15.67:0/541447374 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fe86c15ab10 0x7fe858005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:45.507 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.497+0000 7fe871a12700 1 -- 172.21.15.67:0/541447374 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe858004850 msgr2=0x7fe858004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:45.507 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.497+0000 7fe871a12700 1 --2- 172.21.15.67:0/541447374 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe858004850 0x7fe858004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:45.508 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.497+0000 7fe871a12700 1 -- 172.21.15.67:0/541447374 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8580035b0 con 0x7fe858003a00 2023-10-14T10:36:45.508 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.497+0000 7fe871a12700 1 --2- 172.21.15.67:0/541447374 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe858003a00 0x7fe858003e20 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe85c009b60 tx=0x7fe858005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2770648fe9f36d61 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:45.508 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.497+0000 7fe870a10700 1 -- 172.21.15.67:0/541447374 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe85c00e6c0 con 0x7fe858003a00 2023-10-14T10:36:45.508 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.497+0000 7fe870a10700 1 -- 172.21.15.67:0/541447374 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe85c00e820 con 0x7fe858003a00 2023-10-14T10:36:45.508 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.497+0000 7fe870a10700 1 -- 172.21.15.67:0/541447374 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe85c00eb90 con 0x7fe858003a00 2023-10-14T10:36:45.508 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.498+0000 7fe873c76700 1 -- 172.21.15.67:0/541447374 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe858003a00 msgr2=0x7fe858003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:45.509 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.498+0000 7fe873c76700 1 --2- 172.21.15.67:0/541447374 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe858003a00 0x7fe858003e20 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe85c009b60 tx=0x7fe858005880 comp rx=0 tx=0).stop 2023-10-14T10:36:45.509 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.498+0000 7fe872213700 1 -- 172.21.15.67:0/541447374 reap_dead start 2023-10-14T10:36:45.509 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.498+0000 7fe873c76700 1 -- 172.21.15.67:0/541447374 shutdown_connections 2023-10-14T10:36:45.509 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.498+0000 7fe873c76700 1 -- 172.21.15.67:0/541447374 >> 172.21.15.67:0/541447374 conn(0x7fe86c06dae0 msgr2=0x7fe86c06def0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:45.509 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.498+0000 7fe873c76700 1 -- 172.21.15.67:0/541447374 shutdown_connections 2023-10-14T10:36:45.510 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.498+0000 7fe873c76700 1 -- 172.21.15.67:0/541447374 wait complete. 2023-10-14T10:36:45.510 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.499+0000 7fe873c76700 1 Processor -- start 2023-10-14T10:36:45.510 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.499+0000 7fe873c76700 1 -- start start 2023-10-14T10:36:45.512 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.500+0000 7fe873c76700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe86c14a3b0 0x7fe86c0aeed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:45.513 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.500+0000 7fe873c76700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fe86c1574b0 0x7fe86c14a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:45.513 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.500+0000 7fe873c76700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe86c157930 0x7fe86c148100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:45.513 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.500+0000 7fe873c76700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe86c14b130 con 0x7fe86c14a3b0 2023-10-14T10:36:45.513 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.500+0000 7fe873c76700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe86c157020 con 0x7fe86c157930 2023-10-14T10:36:45.513 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.500+0000 7fe873c76700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe86c15b090 con 0x7fe86c1574b0 2023-10-14T10:36:45.514 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.500+0000 7fe871a12700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe86c14a3b0 0x7fe86c0aeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:45.514 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.500+0000 7fe871a12700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe86c14a3b0 0x7fe86c0aeed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45330/0 (socket says 172.21.15.67:45330) 2023-10-14T10:36:45.514 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.500+0000 7fe871a12700 1 -- 172.21.15.67:0/1513757028 learned_addr learned my addr 172.21.15.67:0/1513757028 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:45.514 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.500+0000 7fe871211700 1 --2- 172.21.15.67:0/1513757028 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fe86c1574b0 0x7fe86c14a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:45.515 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.500+0000 7fe872213700 1 --2- 172.21.15.67:0/1513757028 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe86c157930 0x7fe86c148100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:45.515 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.500+0000 7fe871a12700 1 -- 172.21.15.67:0/1513757028 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fe86c1574b0 msgr2=0x7fe86c14a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:45.515 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.500+0000 7fe871a12700 1 --2- 172.21.15.67:0/1513757028 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fe86c1574b0 0x7fe86c14a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:45.515 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.500+0000 7fe871a12700 1 -- 172.21.15.67:0/1513757028 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe86c157930 msgr2=0x7fe86c148100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:45.515 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.500+0000 7fe871a12700 1 --2- 172.21.15.67:0/1513757028 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe86c157930 0x7fe86c148100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:45.515 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.500+0000 7fe871a12700 1 -- 172.21.15.67:0/1513757028 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe86c156cb0 con 0x7fe86c14a3b0 2023-10-14T10:36:45.516 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.501+0000 7fe871a12700 1 --2- 172.21.15.67:0/1513757028 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe86c14a3b0 0x7fe86c0aeed0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fe8580046a0 tx=0x7fe85c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:45.516 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.501+0000 7fe8627fc700 1 -- 172.21.15.67:0/1513757028 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe85c00e6c0 con 0x7fe86c14a3b0 2023-10-14T10:36:45.516 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.501+0000 7fe8627fc700 1 -- 172.21.15.67:0/1513757028 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe85c003ad0 con 0x7fe86c14a3b0 2023-10-14T10:36:45.516 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.501+0000 7fe873c76700 1 -- 172.21.15.67:0/1513757028 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe86c15b370 con 0x7fe86c14a3b0 2023-10-14T10:36:45.516 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.501+0000 7fe8627fc700 1 -- 172.21.15.67:0/1513757028 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe85c003e80 con 0x7fe86c14a3b0 2023-10-14T10:36:45.517 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.501+0000 7fe873c76700 1 -- 172.21.15.67:0/1513757028 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe86c14acd0 con 0x7fe86c14a3b0 2023-10-14T10:36:45.517 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.502+0000 7fe873c76700 1 -- 172.21.15.67:0/1513757028 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe86c157300 con 0x7fe86c14a3b0 2023-10-14T10:36:45.517 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.502+0000 7fe8627fc700 1 -- 172.21.15.67:0/1513757028 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe85c013470 con 0x7fe86c14a3b0 2023-10-14T10:36:45.517 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.503+0000 7fe8627fc700 1 --2- 172.21.15.67:0/1513757028 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fe85006af00 0x7fe85006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:45.517 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.503+0000 7fe8627fc700 1 -- 172.21.15.67:0/1513757028 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe85c08af20 con 0x7fe86c14a3b0 2023-10-14T10:36:45.518 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.503+0000 7fe871211700 1 --2- 172.21.15.67:0/1513757028 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fe85006af00 0x7fe85006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:45.518 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.503+0000 7fe871211700 1 --2- 172.21.15.67:0/1513757028 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fe85006af00 0x7fe85006d3c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe8680011d0 tx=0x7fe868009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:45.518 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.507+0000 7fe8627fc700 1 -- 172.21.15.67:0/1513757028 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe85c056180 con 0x7fe86c14a3b0 2023-10-14T10:36:45.708 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.706+0000 7fe873c76700 1 -- 172.21.15.67:0/1513757028 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe86c14aef0 con 0x7fe86c14a3b0 2023-10-14T10:36:45.708 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.706+0000 7fe8627fc700 1 -- 172.21.15.67:0/1513757028 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe85c017020 con 0x7fe86c14a3b0 2023-10-14T10:36:45.709 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2023-10-14T10:36:45.710 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.708+0000 7fe873c76700 1 -- 172.21.15.67:0/1513757028 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fe85006af00 msgr2=0x7fe85006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:45.711 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.708+0000 7fe873c76700 1 --2- 172.21.15.67:0/1513757028 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fe85006af00 0x7fe85006d3c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe8680011d0 tx=0x7fe868009040 comp rx=0 tx=0).stop 2023-10-14T10:36:45.711 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.708+0000 7fe873c76700 1 -- 172.21.15.67:0/1513757028 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe86c14a3b0 msgr2=0x7fe86c0aeed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:45.711 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.708+0000 7fe873c76700 1 --2- 172.21.15.67:0/1513757028 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe86c14a3b0 0x7fe86c0aeed0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fe8580046a0 tx=0x7fe85c0047c0 comp rx=0 tx=0).stop 2023-10-14T10:36:45.711 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.708+0000 7fe873c76700 1 -- 172.21.15.67:0/1513757028 shutdown_connections 2023-10-14T10:36:45.711 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.708+0000 7fe873c76700 1 --2- 172.21.15.67:0/1513757028 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fe85006af00 0x7fe85006d3c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:45.712 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.708+0000 7fe873c76700 1 --2- 172.21.15.67:0/1513757028 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe86c157930 0x7fe86c148100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:45.712 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.708+0000 7fe873c76700 1 --2- 172.21.15.67:0/1513757028 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe86c14a3b0 0x7fe86c0aeed0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:45.712 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.708+0000 7fe873c76700 1 --2- 172.21.15.67:0/1513757028 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fe86c1574b0 0x7fe86c14a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:45.712 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.708+0000 7fe873c76700 1 -- 172.21.15.67:0/1513757028 >> 172.21.15.67:0/1513757028 conn(0x7fe86c06dae0 msgr2=0x7fe86c0aff30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:45.712 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.708+0000 7fe873c76700 1 -- 172.21.15.67:0/1513757028 shutdown_connections 2023-10-14T10:36:45.713 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.708+0000 7fe873c76700 1 -- 172.21.15.67:0/1513757028 wait complete. 2023-10-14T10:36:45.722 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-14T10:36:45.850 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.848+0000 7f8a5cb31700 1 Processor -- start 2023-10-14T10:36:45.851 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.848+0000 7f8a5cb31700 1 -- start start 2023-10-14T10:36:45.853 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.849+0000 7f8a5cb31700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f8a580cd950 0x7f8a5815e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:45.853 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.849+0000 7f8a5cb31700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a580cf290 con 0x7f8a580cede0 2023-10-14T10:36:45.853 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.849+0000 7f8a5cb31700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a580b2d60 con 0x7f8a5806a840 2023-10-14T10:36:45.854 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.849+0000 7f8a5cb31700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a581619a0 con 0x7f8a580cd950 2023-10-14T10:36:45.854 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.849+0000 7f8a5659c700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f8a580cd950 0x7f8a5815e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:45.854 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.849+0000 7f8a55d9b700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f8a580cede0 0x7f8a5815f570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:53104/0 (socket says 172.21.15.67:53104) 2023-10-14T10:36:45.854 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.849+0000 7f8a55d9b700 1 -- 172.21.15.67:0/2860127586 learned_addr learned my addr 172.21.15.67:0/2860127586 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:36:45.854 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.849+0000 7f8a5559a700 1 -- 172.21.15.67:0/2860127586 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 623185549 0 0) 0x7f8a580cf290 con 0x7f8a580cede0 2023-10-14T10:36:45.855 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.850+0000 7f8a5559a700 1 -- 172.21.15.67:0/2860127586 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a400035b0 con 0x7f8a580cede0 2023-10-14T10:36:45.855 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.850+0000 7f8a5659c700 1 -- 172.21.15.67:0/2860127586 >> v1:172.21.15.72:6789/0 conn(0x7f8a5806a840 legacy=0x7f8a58161df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:45.855 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.850+0000 7f8a5659c700 1 -- 172.21.15.67:0/2860127586 >> v1:172.21.15.67:6789/0 conn(0x7f8a580cede0 legacy=0x7f8a5815f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:45.855 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.850+0000 7f8a5659c700 1 -- 172.21.15.67:0/2860127586 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a580cdd30 con 0x7f8a580cd950 2023-10-14T10:36:45.855 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.850+0000 7f8a5659c700 1 --2- 172.21.15.67:0/2860127586 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a580cd950 0x7f8a5815e2f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c009840 tx=0x7f8a4c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fd60087b6242ecf server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:45.856 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.850+0000 7f8a5559a700 1 -- 172.21.15.67:0/2860127586 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a4c011830 con 0x7f8a580cd950 2023-10-14T10:36:45.856 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.850+0000 7f8a5559a700 1 -- 172.21.15.67:0/2860127586 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a4c011990 con 0x7f8a580cd950 2023-10-14T10:36:45.856 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.850+0000 7f8a5559a700 1 -- 172.21.15.67:0/2860127586 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a4c011d20 con 0x7f8a580cd950 2023-10-14T10:36:45.856 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.850+0000 7f8a5cb31700 1 -- 172.21.15.67:0/2860127586 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a580cd950 msgr2=0x7f8a5815e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:45.856 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.850+0000 7f8a5cb31700 1 --2- 172.21.15.67:0/2860127586 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a580cd950 0x7f8a5815e2f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c009840 tx=0x7f8a4c009b50 comp rx=0 tx=0).stop 2023-10-14T10:36:45.857 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.851+0000 7f8a5cb31700 1 -- 172.21.15.67:0/2860127586 shutdown_connections 2023-10-14T10:36:45.857 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.851+0000 7f8a5cb31700 1 --2- 172.21.15.67:0/2860127586 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a580cd950 0x7f8a5815e2f0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:45.857 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.851+0000 7f8a5cb31700 1 -- 172.21.15.67:0/2860127586 >> 172.21.15.67:0/2860127586 conn(0x7f8a5806dae0 msgr2=0x7f8a5806def0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:45.857 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.851+0000 7f8a5cb31700 1 -- 172.21.15.67:0/2860127586 shutdown_connections 2023-10-14T10:36:45.857 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.851+0000 7f8a5cb31700 1 -- 172.21.15.67:0/2860127586 wait complete. 2023-10-14T10:36:45.860 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.852+0000 7f8a5cb31700 1 Processor -- start 2023-10-14T10:36:45.860 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.852+0000 7f8a5cb31700 1 -- start start 2023-10-14T10:36:45.860 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.853+0000 7f8a5cb31700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8a5806a840 0x7f8a580b5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:45.860 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.853+0000 7f8a5cb31700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f8a580cd950 0x7f8a580c8440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:45.861 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.853+0000 7f8a5cb31700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a580cede0 0x7f8a580c9440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:45.861 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.853+0000 7f8a5cb31700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a580c8980 con 0x7f8a580cede0 2023-10-14T10:36:45.861 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.853+0000 7f8a5cb31700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a581f6e30 con 0x7f8a5806a840 2023-10-14T10:36:45.861 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.853+0000 7f8a5cb31700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a58160ca0 con 0x7f8a580cd950 2023-10-14T10:36:45.862 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.853+0000 7f8a56d9d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a580cede0 0x7f8a580c9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:45.862 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.853+0000 7f8a56d9d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a580cede0 0x7f8a580c9440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45346/0 (socket says 172.21.15.67:45346) 2023-10-14T10:36:45.862 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.853+0000 7f8a56d9d700 1 -- 172.21.15.67:0/1727334435 learned_addr learned my addr 172.21.15.67:0/1727334435 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:45.862 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.853+0000 7f8a5659c700 1 --2- 172.21.15.67:0/1727334435 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8a5806a840 0x7f8a580b5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:45.862 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.853+0000 7f8a55d9b700 1 --2- 172.21.15.67:0/1727334435 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f8a580cd950 0x7f8a580c8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:45.866 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.853+0000 7f8a56d9d700 1 -- 172.21.15.67:0/1727334435 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f8a580cd950 msgr2=0x7f8a580c8440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:45.866 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.853+0000 7f8a56d9d700 1 --2- 172.21.15.67:0/1727334435 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f8a580cd950 0x7f8a580c8440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:45.867 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.853+0000 7f8a56d9d700 1 -- 172.21.15.67:0/1727334435 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8a5806a840 msgr2=0x7f8a580b5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:45.867 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.853+0000 7f8a56d9d700 1 --2- 172.21.15.67:0/1727334435 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8a5806a840 0x7f8a580b5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:45.867 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.853+0000 7f8a56d9d700 1 -- 172.21.15.67:0/1727334435 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a4c0094a0 con 0x7f8a580cede0 2023-10-14T10:36:45.867 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.853+0000 7f8a56d9d700 1 --2- 172.21.15.67:0/1727334435 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a580cede0 0x7f8a580c9440 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8a5000cc50 tx=0x7f8a5000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:45.867 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.853+0000 7f8a46ffd700 1 -- 172.21.15.67:0/1727334435 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a50003c00 con 0x7f8a580cede0 2023-10-14T10:36:45.868 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.854+0000 7f8a46ffd700 1 -- 172.21.15.67:0/1727334435 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a50003d60 con 0x7f8a580cede0 2023-10-14T10:36:45.868 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.854+0000 7f8a5cb31700 1 -- 172.21.15.67:0/1727334435 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a581f3db0 con 0x7f8a580cede0 2023-10-14T10:36:45.868 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.854+0000 7f8a46ffd700 1 -- 172.21.15.67:0/1727334435 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a5000aab0 con 0x7f8a580cede0 2023-10-14T10:36:45.868 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.854+0000 7f8a5cb31700 1 -- 172.21.15.67:0/1727334435 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a581f4370 con 0x7f8a580cede0 2023-10-14T10:36:45.868 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.855+0000 7f8a5cb31700 1 -- 172.21.15.67:0/1727334435 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a581f7050 con 0x7f8a580cede0 2023-10-14T10:36:45.868 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.855+0000 7f8a46ffd700 1 -- 172.21.15.67:0/1727334435 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f8a5000ac10 con 0x7f8a580cede0 2023-10-14T10:36:45.869 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.856+0000 7f8a46ffd700 1 --2- 172.21.15.67:0/1727334435 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f8a3806af00 0x7f8a3806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:45.869 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.856+0000 7f8a46ffd700 1 -- 172.21.15.67:0/1727334435 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8a50089560 con 0x7f8a580cede0 2023-10-14T10:36:45.869 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.856+0000 7f8a5659c700 1 --2- 172.21.15.67:0/1727334435 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f8a3806af00 0x7f8a3806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:45.869 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.856+0000 7f8a5659c700 1 --2- 172.21.15.67:0/1727334435 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f8a3806af00 0x7f8a3806d3c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c0090b0 tx=0x7f8a4c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:45.870 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:45.860+0000 7f8a46ffd700 1 -- 172.21.15.67:0/1727334435 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8a581f7050 con 0x7f8a580cede0 2023-10-14T10:36:46.065 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.063+0000 7f8a5cb31700 1 -- 172.21.15.67:0/1727334435 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8a581f6ff0 con 0x7f8a580cede0 2023-10-14T10:36:46.066 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.064+0000 7f8a46ffd700 1 -- 172.21.15.67:0/1727334435 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8a50054610 con 0x7f8a580cede0 2023-10-14T10:36:46.066 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2023-10-14T10:36:46.068 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.066+0000 7f8a5cb31700 1 -- 172.21.15.67:0/1727334435 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f8a3806af00 msgr2=0x7f8a3806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:46.068 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.066+0000 7f8a5cb31700 1 --2- 172.21.15.67:0/1727334435 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f8a3806af00 0x7f8a3806d3c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c0090b0 tx=0x7f8a4c008030 comp rx=0 tx=0).stop 2023-10-14T10:36:46.069 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.066+0000 7f8a5cb31700 1 -- 172.21.15.67:0/1727334435 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a580cede0 msgr2=0x7f8a580c9440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:46.069 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.066+0000 7f8a5cb31700 1 --2- 172.21.15.67:0/1727334435 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a580cede0 0x7f8a580c9440 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8a5000cc50 tx=0x7f8a5000cf60 comp rx=0 tx=0).stop 2023-10-14T10:36:46.069 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.066+0000 7f8a5cb31700 1 -- 172.21.15.67:0/1727334435 shutdown_connections 2023-10-14T10:36:46.069 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.066+0000 7f8a5cb31700 1 --2- 172.21.15.67:0/1727334435 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f8a3806af00 0x7f8a3806d3c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:46.070 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.066+0000 7f8a5cb31700 1 --2- 172.21.15.67:0/1727334435 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a580cede0 0x7f8a580c9440 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:46.070 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.066+0000 7f8a5cb31700 1 --2- 172.21.15.67:0/1727334435 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8a5806a840 0x7f8a580b5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:46.070 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.066+0000 7f8a5cb31700 1 --2- 172.21.15.67:0/1727334435 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f8a580cd950 0x7f8a580c8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:46.070 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.066+0000 7f8a5cb31700 1 -- 172.21.15.67:0/1727334435 >> 172.21.15.67:0/1727334435 conn(0x7f8a5806dae0 msgr2=0x7f8a580b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:46.070 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.066+0000 7f8a5cb31700 1 -- 172.21.15.67:0/1727334435 shutdown_connections 2023-10-14T10:36:46.071 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.066+0000 7f8a5cb31700 1 -- 172.21.15.67:0/1727334435 wait complete. 2023-10-14T10:36:46.080 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-14T10:36:46.208 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.206+0000 7ff3360d5700 1 Processor -- start 2023-10-14T10:36:46.208 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.206+0000 7ff3360d5700 1 -- start start 2023-10-14T10:36:46.211 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.207+0000 7ff3360d5700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7ff330160e30 0x7ff3300d4200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:46.211 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.207+0000 7ff3360d5700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7ff3300d08a0 0x7ff3300ce3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:46.215 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.207+0000 7ff3360d5700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3300b6610 con 0x7ff330161210 2023-10-14T10:36:46.215 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.207+0000 7ff3360d5700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3300d1da0 con 0x7ff330160e30 2023-10-14T10:36:46.215 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.207+0000 7ff3360d5700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3300d2080 con 0x7ff3300d08a0 2023-10-14T10:36:46.215 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.207+0000 7ff32effd700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7ff330161210 0x7ff3300d0190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:53120/0 (socket says 172.21.15.67:53120) 2023-10-14T10:36:46.215 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.207+0000 7ff32effd700 1 -- 172.21.15.67:0/3766420120 learned_addr learned my addr 172.21.15.67:0/3766420120 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:36:46.216 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.207+0000 7ff32f7fe700 1 --2- 172.21.15.67:0/3766420120 >> v2:172.21.15.72:3300/0 conn(0x7ff330160e30 0x7ff3300d4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:46.216 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.207+0000 7ff32ffff700 1 --2- 172.21.15.67:0/3766420120 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7ff3300d08a0 0x7ff3300ce3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:46.216 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.207+0000 7ff32e7fc700 1 -- 172.21.15.67:0/3766420120 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1892193743 0 0) 0x7ff3300b6610 con 0x7ff330161210 2023-10-14T10:36:46.216 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.207+0000 7ff32e7fc700 1 -- 172.21.15.67:0/3766420120 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3180035b0 con 0x7ff330161210 2023-10-14T10:36:46.216 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.207+0000 7ff32f7fe700 1 -- 172.21.15.67:0/3766420120 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7ff3300d08a0 msgr2=0x7ff3300ce3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:46.217 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.207+0000 7ff32f7fe700 1 --2- 172.21.15.67:0/3766420120 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7ff3300d08a0 0x7ff3300ce3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:46.217 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.207+0000 7ff32f7fe700 1 -- 172.21.15.67:0/3766420120 >> v1:172.21.15.67:6789/0 conn(0x7ff330161210 legacy=0x7ff3300d0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:46.217 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.207+0000 7ff32f7fe700 1 -- 172.21.15.67:0/3766420120 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3300d2360 con 0x7ff330160e30 2023-10-14T10:36:46.217 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.207+0000 7ff32ffff700 1 --2- 172.21.15.67:0/3766420120 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7ff3300d08a0 0x7ff3300ce3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-14T10:36:46.217 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.207+0000 7ff32f7fe700 1 --2- 172.21.15.67:0/3766420120 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff330160e30 0x7ff3300d4200 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff320009820 tx=0x7ff320009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6fde27f004a9165e server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:46.218 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.208+0000 7ff32e7fc700 1 -- 172.21.15.67:0/3766420120 <== mon.1 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff320005e30 con 0x7ff330160e30 2023-10-14T10:36:46.218 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.208+0000 7ff32e7fc700 1 -- 172.21.15.67:0/3766420120 <== mon.1 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff320006030 con 0x7ff330160e30 2023-10-14T10:36:46.218 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.208+0000 7ff32e7fc700 1 -- 172.21.15.67:0/3766420120 <== mon.1 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3200063c0 con 0x7ff330160e30 2023-10-14T10:36:46.218 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.208+0000 7ff3360d5700 1 -- 172.21.15.67:0/3766420120 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff330160e30 msgr2=0x7ff3300d4200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:46.218 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.208+0000 7ff3360d5700 1 --2- 172.21.15.67:0/3766420120 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff330160e30 0x7ff3300d4200 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff320009820 tx=0x7ff320009b30 comp rx=0 tx=0).stop 2023-10-14T10:36:46.218 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.208+0000 7ff3360d5700 1 -- 172.21.15.67:0/3766420120 shutdown_connections 2023-10-14T10:36:46.219 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.208+0000 7ff3360d5700 1 --2- 172.21.15.67:0/3766420120 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7ff3300d08a0 0x7ff3300ce3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:46.219 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.208+0000 7ff3360d5700 1 --2- 172.21.15.67:0/3766420120 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff330160e30 0x7ff3300d4200 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:46.219 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.208+0000 7ff3360d5700 1 -- 172.21.15.67:0/3766420120 >> 172.21.15.67:0/3766420120 conn(0x7ff33009cb40 msgr2=0x7ff3300d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:46.219 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.208+0000 7ff3360d5700 1 -- 172.21.15.67:0/3766420120 shutdown_connections 2023-10-14T10:36:46.219 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.208+0000 7ff3360d5700 1 -- 172.21.15.67:0/3766420120 wait complete. 2023-10-14T10:36:46.222 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.210+0000 7ff3360d5700 1 Processor -- start 2023-10-14T10:36:46.222 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.210+0000 7ff3360d5700 1 -- start start 2023-10-14T10:36:46.223 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.211+0000 7ff3360d5700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7ff3300d08a0 0x7ff330155d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:46.223 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.211+0000 7ff3360d5700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff330160e30 0x7ff330159e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:46.223 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.211+0000 7ff3360d5700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff330161210 0x7ff3301f33e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:46.223 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.211+0000 7ff3360d5700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3301562c0 con 0x7ff330161210 2023-10-14T10:36:46.223 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.211+0000 7ff3360d5700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff33015c750 con 0x7ff330160e30 2023-10-14T10:36:46.224 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.211+0000 7ff3360d5700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff33015ca30 con 0x7ff3300d08a0 2023-10-14T10:36:46.224 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.211+0000 7ff32ffff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff330161210 0x7ff3301f33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:46.224 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.211+0000 7ff32ffff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff330161210 0x7ff3301f33e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45360/0 (socket says 172.21.15.67:45360) 2023-10-14T10:36:46.224 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.211+0000 7ff32ffff700 1 -- 172.21.15.67:0/1585707275 learned_addr learned my addr 172.21.15.67:0/1585707275 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:46.224 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.211+0000 7ff32f7fe700 1 --2- 172.21.15.67:0/1585707275 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7ff3300d08a0 0x7ff330155d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:46.225 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.211+0000 7ff32effd700 1 --2- 172.21.15.67:0/1585707275 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff330160e30 0x7ff330159e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:46.225 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.211+0000 7ff32ffff700 1 -- 172.21.15.67:0/1585707275 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7ff3300d08a0 msgr2=0x7ff330155d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:46.225 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.211+0000 7ff32ffff700 1 --2- 172.21.15.67:0/1585707275 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7ff3300d08a0 0x7ff330155d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:46.225 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.211+0000 7ff32ffff700 1 -- 172.21.15.67:0/1585707275 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff330160e30 msgr2=0x7ff330159e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:46.225 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.211+0000 7ff32ffff700 1 --2- 172.21.15.67:0/1585707275 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff330160e30 0x7ff330159e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:46.226 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.212+0000 7ff32ffff700 1 -- 172.21.15.67:0/1585707275 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff328009a70 con 0x7ff330161210 2023-10-14T10:36:46.226 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.212+0000 7ff32ffff700 1 --2- 172.21.15.67:0/1585707275 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff330161210 0x7ff3301f33e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff32800efc0 tx=0x7ff32800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:46.226 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.212+0000 7ff317fff700 1 -- 172.21.15.67:0/1585707275 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32800ca90 con 0x7ff330161210 2023-10-14T10:36:46.226 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.212+0000 7ff317fff700 1 -- 172.21.15.67:0/1585707275 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff32800f040 con 0x7ff330161210 2023-10-14T10:36:46.226 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.212+0000 7ff3360d5700 1 -- 172.21.15.67:0/1585707275 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff320009430 con 0x7ff330161210 2023-10-14T10:36:46.227 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.212+0000 7ff317fff700 1 -- 172.21.15.67:0/1585707275 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32800ce30 con 0x7ff330161210 2023-10-14T10:36:46.227 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.212+0000 7ff3360d5700 1 -- 172.21.15.67:0/1585707275 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff33015cff0 con 0x7ff330161210 2023-10-14T10:36:46.227 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.213+0000 7ff3360d5700 1 -- 172.21.15.67:0/1585707275 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff330067d20 con 0x7ff330161210 2023-10-14T10:36:46.227 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.213+0000 7ff317fff700 1 -- 172.21.15.67:0/1585707275 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff328014480 con 0x7ff330161210 2023-10-14T10:36:46.227 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.214+0000 7ff317fff700 1 --2- 172.21.15.67:0/1585707275 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7ff31c06af00 0x7ff31c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:46.228 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.214+0000 7ff317fff700 1 -- 172.21.15.67:0/1585707275 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff328089c30 con 0x7ff330161210 2023-10-14T10:36:46.228 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.214+0000 7ff32f7fe700 1 --2- 172.21.15.67:0/1585707275 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7ff31c06af00 0x7ff31c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:46.228 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.215+0000 7ff32f7fe700 1 --2- 172.21.15.67:0/1585707275 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7ff31c06af00 0x7ff31c06d3c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff320000c00 tx=0x7ff320008070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:46.228 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.218+0000 7ff317fff700 1 -- 172.21.15.67:0/1585707275 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff328054d70 con 0x7ff330161210 2023-10-14T10:36:46.423 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.420+0000 7ff3360d5700 1 -- 172.21.15.67:0/1585707275 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff33006c230 con 0x7ff330161210 2023-10-14T10:36:46.423 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.421+0000 7ff317fff700 1 -- 172.21.15.67:0/1585707275 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff328009cd0 con 0x7ff330161210 2023-10-14T10:36:46.423 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2023-10-14T10:36:46.425 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.423+0000 7ff3360d5700 1 -- 172.21.15.67:0/1585707275 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7ff31c06af00 msgr2=0x7ff31c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:46.426 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.423+0000 7ff3360d5700 1 --2- 172.21.15.67:0/1585707275 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7ff31c06af00 0x7ff31c06d3c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff320000c00 tx=0x7ff320008070 comp rx=0 tx=0).stop 2023-10-14T10:36:46.426 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.423+0000 7ff3360d5700 1 -- 172.21.15.67:0/1585707275 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff330161210 msgr2=0x7ff3301f33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:46.426 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.423+0000 7ff3360d5700 1 --2- 172.21.15.67:0/1585707275 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff330161210 0x7ff3301f33e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff32800efc0 tx=0x7ff32800c3a0 comp rx=0 tx=0).stop 2023-10-14T10:36:46.426 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.423+0000 7ff3360d5700 1 -- 172.21.15.67:0/1585707275 shutdown_connections 2023-10-14T10:36:46.427 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.423+0000 7ff3360d5700 1 --2- 172.21.15.67:0/1585707275 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7ff31c06af00 0x7ff31c06d3c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:46.427 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.423+0000 7ff3360d5700 1 --2- 172.21.15.67:0/1585707275 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff330161210 0x7ff3301f33e0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:46.427 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.423+0000 7ff3360d5700 1 --2- 172.21.15.67:0/1585707275 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7ff3300d08a0 0x7ff330155d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:46.427 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.423+0000 7ff3360d5700 1 --2- 172.21.15.67:0/1585707275 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff330160e30 0x7ff330159e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:46.427 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.423+0000 7ff3360d5700 1 -- 172.21.15.67:0/1585707275 >> 172.21.15.67:0/1585707275 conn(0x7ff33009cb40 msgr2=0x7ff3300b5730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:46.427 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.423+0000 7ff3360d5700 1 -- 172.21.15.67:0/1585707275 shutdown_connections 2023-10-14T10:36:46.428 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.423+0000 7ff3360d5700 1 -- 172.21.15.67:0/1585707275 wait complete. 2023-10-14T10:36:46.437 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-14T10:36:46.562 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.560+0000 7f52ee23a700 1 Processor -- start 2023-10-14T10:36:46.563 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.560+0000 7f52ee23a700 1 -- start start 2023-10-14T10:36:46.566 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.561+0000 7f52ee23a700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f52e815b740 0x7f52e80d1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:46.566 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.561+0000 7f52ee23a700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f52e80ce6e0 0x7f52e80cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:46.566 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.561+0000 7f52ee23a700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52e80cf190 con 0x7f52e80d0c70 2023-10-14T10:36:46.567 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.561+0000 7f52ee23a700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52e815bb20 con 0x7f52e815b740 2023-10-14T10:36:46.567 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.561+0000 7f52ee23a700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52e80cbd40 con 0x7f52e80ce6e0 2023-10-14T10:36:46.567 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.561+0000 7f52e77fe700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f52e815b740 0x7f52e80d1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:46.567 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.561+0000 7f52dffff700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f52e80d0c70 0x7f52e80cdfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:53132/0 (socket says 172.21.15.67:53132) 2023-10-14T10:36:46.567 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.561+0000 7f52dffff700 1 -- 172.21.15.67:0/1730054203 learned_addr learned my addr 172.21.15.67:0/1730054203 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:36:46.568 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.561+0000 7f52e7fff700 1 --2- 172.21.15.67:0/1730054203 >> v2:172.21.15.72:3300/0 conn(0x7f52e80ce6e0 0x7f52e80cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:46.568 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.562+0000 7f52e6ffd700 1 -- 172.21.15.67:0/1730054203 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1698486510 0 0) 0x7f52e80cf190 con 0x7f52e80d0c70 2023-10-14T10:36:46.568 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.562+0000 7f52e6ffd700 1 -- 172.21.15.67:0/1730054203 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52d40035b0 con 0x7f52e80d0c70 2023-10-14T10:36:46.568 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.562+0000 7f52e77fe700 1 -- 172.21.15.67:0/1730054203 >> v2:172.21.15.72:3300/0 conn(0x7f52e80ce6e0 msgr2=0x7f52e80cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:46.568 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.562+0000 7f52e77fe700 1 --2- 172.21.15.67:0/1730054203 >> v2:172.21.15.72:3300/0 conn(0x7f52e80ce6e0 0x7f52e80cf690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:46.569 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.562+0000 7f52e77fe700 1 -- 172.21.15.67:0/1730054203 >> v1:172.21.15.67:6789/0 conn(0x7f52e80d0c70 legacy=0x7f52e80cdfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:46.569 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.562+0000 7f52e77fe700 1 -- 172.21.15.67:0/1730054203 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52e80cc020 con 0x7f52e815b740 2023-10-14T10:36:46.569 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.562+0000 7f52e77fe700 1 --2- 172.21.15.67:0/1730054203 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52e815b740 0x7f52e80d1cd0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f52e80c8620 tx=0x7f52d8009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=147a5ab207c7a41c server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:46.569 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.562+0000 7f52e6ffd700 1 -- 172.21.15.67:0/1730054203 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d8011830 con 0x7f52e815b740 2023-10-14T10:36:46.569 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.562+0000 7f52e6ffd700 1 -- 172.21.15.67:0/1730054203 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52d8011990 con 0x7f52e815b740 2023-10-14T10:36:46.570 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.562+0000 7f52e6ffd700 1 -- 172.21.15.67:0/1730054203 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d8011d20 con 0x7f52e815b740 2023-10-14T10:36:46.570 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.563+0000 7f52ee23a700 1 -- 172.21.15.67:0/1730054203 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52e815b740 msgr2=0x7f52e80d1cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:46.570 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.563+0000 7f52ee23a700 1 --2- 172.21.15.67:0/1730054203 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52e815b740 0x7f52e80d1cd0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f52e80c8620 tx=0x7f52d8009b20 comp rx=0 tx=0).stop 2023-10-14T10:36:46.570 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.563+0000 7f52ee23a700 1 -- 172.21.15.67:0/1730054203 shutdown_connections 2023-10-14T10:36:46.571 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.563+0000 7f52ee23a700 1 --2- 172.21.15.67:0/1730054203 >> v2:172.21.15.72:3300/0 conn(0x7f52e80ce6e0 0x7f52e80cf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:46.571 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.563+0000 7f52ee23a700 1 --2- 172.21.15.67:0/1730054203 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52e815b740 0x7f52e80d1cd0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:46.571 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.563+0000 7f52ee23a700 1 -- 172.21.15.67:0/1730054203 >> 172.21.15.67:0/1730054203 conn(0x7f52e80665a0 msgr2=0x7f52e8066980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:46.571 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.563+0000 7f52ee23a700 1 -- 172.21.15.67:0/1730054203 shutdown_connections 2023-10-14T10:36:46.571 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.563+0000 7f52ee23a700 1 -- 172.21.15.67:0/1730054203 wait complete. 2023-10-14T10:36:46.572 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.564+0000 7f52ee23a700 1 Processor -- start 2023-10-14T10:36:46.572 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.564+0000 7f52ee23a700 1 -- start start 2023-10-14T10:36:46.574 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.565+0000 7f52ee23a700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52e80ce6e0 0x7f52e8159ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:46.574 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.565+0000 7f52ee23a700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f52e80d0c70 0x7f52e81f27a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:46.575 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.565+0000 7f52ee23a700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f52e815b740 0x7f52e815bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:46.575 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.565+0000 7f52ee23a700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52e8158ae0 con 0x7f52e80ce6e0 2023-10-14T10:36:46.575 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.565+0000 7f52ee23a700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52e8159620 con 0x7f52e815b740 2023-10-14T10:36:46.575 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.565+0000 7f52ee23a700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52e8158cf0 con 0x7f52e80d0c70 2023-10-14T10:36:46.575 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.565+0000 7f52e77fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52e80ce6e0 0x7f52e8159ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:46.576 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.565+0000 7f52e77fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52e80ce6e0 0x7f52e8159ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45370/0 (socket says 172.21.15.67:45370) 2023-10-14T10:36:46.576 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.565+0000 7f52e77fe700 1 -- 172.21.15.67:0/393954091 learned_addr learned my addr 172.21.15.67:0/393954091 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:46.576 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.565+0000 7f52e77fe700 1 -- 172.21.15.67:0/393954091 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f52e80d0c70 msgr2=0x7f52e81f27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:46.576 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.565+0000 7f52e77fe700 1 --2- 172.21.15.67:0/393954091 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f52e80d0c70 0x7f52e81f27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:46.576 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.565+0000 7f52e77fe700 1 -- 172.21.15.67:0/393954091 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f52e815b740 msgr2=0x7f52e815bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:46.576 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.565+0000 7f52e77fe700 1 --2- 172.21.15.67:0/393954091 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f52e815b740 0x7f52e815bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:46.577 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.565+0000 7f52e77fe700 1 -- 172.21.15.67:0/393954091 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52d80094a0 con 0x7f52e80ce6e0 2023-10-14T10:36:46.577 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.565+0000 7f52e77fe700 1 --2- 172.21.15.67:0/393954091 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52e80ce6e0 0x7f52e8159ab0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f52d80090b0 tx=0x7f52d8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:46.577 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.566+0000 7f52e4ff9700 1 -- 172.21.15.67:0/393954091 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d80036a0 con 0x7f52e80ce6e0 2023-10-14T10:36:46.577 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.566+0000 7f52e4ff9700 1 -- 172.21.15.67:0/393954091 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52d801f070 con 0x7f52e80ce6e0 2023-10-14T10:36:46.577 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.566+0000 7f52e4ff9700 1 -- 172.21.15.67:0/393954091 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d8003a20 con 0x7f52e80ce6e0 2023-10-14T10:36:46.578 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.566+0000 7f52ee23a700 1 -- 172.21.15.67:0/393954091 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52e8158fd0 con 0x7f52e80ce6e0 2023-10-14T10:36:46.578 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.566+0000 7f52ee23a700 1 -- 172.21.15.67:0/393954091 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52e815d600 con 0x7f52e80ce6e0 2023-10-14T10:36:46.578 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.567+0000 7f52ee23a700 1 -- 172.21.15.67:0/393954091 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52e815d790 con 0x7f52e80ce6e0 2023-10-14T10:36:46.578 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.567+0000 7f52e4ff9700 1 -- 172.21.15.67:0/393954091 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f52d8003b80 con 0x7f52e80ce6e0 2023-10-14T10:36:46.578 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.567+0000 7f52e4ff9700 1 --2- 172.21.15.67:0/393954091 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f52c806af00 0x7f52c806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:46.579 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.568+0000 7f52e4ff9700 1 -- 172.21.15.67:0/393954091 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f52d8021070 con 0x7f52e80ce6e0 2023-10-14T10:36:46.579 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.568+0000 7f52dffff700 1 --2- 172.21.15.67:0/393954091 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f52c806af00 0x7f52c806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:46.579 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.568+0000 7f52dffff700 1 --2- 172.21.15.67:0/393954091 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f52c806af00 0x7f52c806d3c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f52e815f7b0 tx=0x7f52d00079c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:46.579 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.572+0000 7f52e4ff9700 1 -- 172.21.15.67:0/393954091 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f52d8055ec0 con 0x7f52e80ce6e0 2023-10-14T10:36:46.820 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.818+0000 7f52ee23a700 1 -- 172.21.15.67:0/393954091 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f52e8067d20 con 0x7f52e80ce6e0 2023-10-14T10:36:46.821 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.819+0000 7f52e4ff9700 1 -- 172.21.15.67:0/393954091 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4901 (secure 0 0 0) 0x7f52d802b090 con 0x7f52e80ce6e0 2023-10-14T10:36:46.821 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-14T10:36:46.821 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4217,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6836/838298018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":838298018},{"type":"v1","addr":"172.21.15.72:6836","nonce":838298018}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4255,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/1687192726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6833","nonce":1687192726},{"type":"v1","addr":"172.21.15.67:6835","nonce":1687192726}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6834/3936231765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":3936231765},{"type":"v1","addr":"172.21.15.67:6834","nonce":3936231765}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-14T10:36:03.247976+0000","modified":"2023-10-14T10:36:06.283381+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4267},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4267":{"gid":4267,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.72:6837/757500960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6835","nonce":757500960},{"type":"v1","addr":"172.21.15.72:6837","nonce":757500960}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-14T10:36:46.826 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.821+0000 7f52ee23a700 1 -- 172.21.15.67:0/393954091 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f52c806af00 msgr2=0x7f52c806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:46.826 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.821+0000 7f52ee23a700 1 --2- 172.21.15.67:0/393954091 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f52c806af00 0x7f52c806d3c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f52e815f7b0 tx=0x7f52d00079c0 comp rx=0 tx=0).stop 2023-10-14T10:36:46.827 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.821+0000 7f52ee23a700 1 -- 172.21.15.67:0/393954091 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52e80ce6e0 msgr2=0x7f52e8159ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:46.827 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.821+0000 7f52ee23a700 1 --2- 172.21.15.67:0/393954091 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52e80ce6e0 0x7f52e8159ab0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f52d80090b0 tx=0x7f52d8004750 comp rx=0 tx=0).stop 2023-10-14T10:36:46.827 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.821+0000 7f52ee23a700 1 -- 172.21.15.67:0/393954091 shutdown_connections 2023-10-14T10:36:46.827 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.821+0000 7f52ee23a700 1 --2- 172.21.15.67:0/393954091 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f52c806af00 0x7f52c806d3c0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:46.827 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.821+0000 7f52ee23a700 1 --2- 172.21.15.67:0/393954091 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f52e815b740 0x7f52e815bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:46.828 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.821+0000 7f52ee23a700 1 --2- 172.21.15.67:0/393954091 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52e80ce6e0 0x7f52e8159ab0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:46.828 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.821+0000 7f52ee23a700 1 --2- 172.21.15.67:0/393954091 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f52e80d0c70 0x7f52e81f27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:46.828 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.821+0000 7f52ee23a700 1 -- 172.21.15.67:0/393954091 >> 172.21.15.67:0/393954091 conn(0x7f52e80665a0 msgr2=0x7f52e815ab30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:46.828 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.821+0000 7f52ee23a700 1 -- 172.21.15.67:0/393954091 shutdown_connections 2023-10-14T10:36:46.828 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.821+0000 7f52ee23a700 1 -- 172.21.15.67:0/393954091 wait complete. 2023-10-14T10:36:46.829 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 8 2023-10-14T10:36:46.835 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-14T10:36:46.960 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.958+0000 7fafca31c700 1 Processor -- start 2023-10-14T10:36:46.961 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.958+0000 7fafca31c700 1 -- start start 2023-10-14T10:36:46.963 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.959+0000 7fafca31c700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fafc415b740 0x7fafc40d1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:46.964 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.959+0000 7fafca31c700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fafc40d0c70 0x7fafc40cdfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:46.964 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.959+0000 7fafca31c700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fafc40c5110 0x7fafc40ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:46.964 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.959+0000 7fafca31c700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafc415bb20 con 0x7fafc40c5110 2023-10-14T10:36:46.964 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.959+0000 7fafca31c700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafc40cf190 con 0x7fafc40d0c70 2023-10-14T10:36:46.964 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.959+0000 7fafca31c700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafc40cbd40 con 0x7fafc415b740 2023-10-14T10:36:46.965 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.959+0000 7fafc88b9700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fafc40c5110 0x7fafc40ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:46.965 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.959+0000 7fafc88b9700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fafc40c5110 0x7fafc40ce510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45380/0 (socket says 172.21.15.67:45380) 2023-10-14T10:36:46.965 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.959+0000 7fafc88b9700 1 -- 172.21.15.67:0/2005230424 learned_addr learned my addr 172.21.15.67:0/2005230424 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:46.965 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.959+0000 7fafc3fff700 1 --2- 172.21.15.67:0/2005230424 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fafc415b740 0x7fafc40d1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:46.965 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.959+0000 7fafc37fe700 1 --2- 172.21.15.67:0/2005230424 >> v2:172.21.15.72:3300/0 conn(0x7fafc40d0c70 0x7fafc40cdfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:46.966 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.959+0000 7fafc88b9700 1 -- 172.21.15.67:0/2005230424 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fafc415b740 msgr2=0x7fafc40d1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:46.966 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.959+0000 7fafc88b9700 1 --2- 172.21.15.67:0/2005230424 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fafc415b740 0x7fafc40d1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:46.966 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.959+0000 7fafc88b9700 1 -- 172.21.15.67:0/2005230424 >> v2:172.21.15.72:3300/0 conn(0x7fafc40d0c70 msgr2=0x7fafc40cdfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:46.966 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.959+0000 7fafc88b9700 1 --2- 172.21.15.67:0/2005230424 >> v2:172.21.15.72:3300/0 conn(0x7fafc40d0c70 0x7fafc40cdfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:46.966 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.959+0000 7fafc88b9700 1 -- 172.21.15.67:0/2005230424 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafc40cc020 con 0x7fafc40c5110 2023-10-14T10:36:46.967 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.959+0000 7fafc3fff700 1 --2- 172.21.15.67:0/2005230424 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fafc415b740 0x7fafc40d1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-14T10:36:46.967 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.959+0000 7fafc37fe700 1 --2- 172.21.15.67:0/2005230424 >> v2:172.21.15.72:3300/0 conn(0x7fafc40d0c70 0x7fafc40cdfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-14T10:36:46.967 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.959+0000 7fafc88b9700 1 --2- 172.21.15.67:0/2005230424 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fafc40c5110 0x7fafc40ce510 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fafc406fd00 tx=0x7fafbc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f10d9e86a6126fc6 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:46.967 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.960+0000 7fafc2ffd700 1 -- 172.21.15.67:0/2005230424 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafbc00e070 con 0x7fafc40c5110 2023-10-14T10:36:46.967 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.960+0000 7fafc2ffd700 1 -- 172.21.15.67:0/2005230424 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafbc00b9c0 con 0x7fafc40c5110 2023-10-14T10:36:46.968 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.960+0000 7fafc2ffd700 1 -- 172.21.15.67:0/2005230424 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafbc00bd70 con 0x7fafc40c5110 2023-10-14T10:36:46.968 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.960+0000 7fafca31c700 1 -- 172.21.15.67:0/2005230424 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fafc40c5110 msgr2=0x7fafc40ce510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:46.968 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.960+0000 7fafca31c700 1 --2- 172.21.15.67:0/2005230424 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fafc40c5110 0x7fafc40ce510 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fafc406fd00 tx=0x7fafbc00b330 comp rx=0 tx=0).stop 2023-10-14T10:36:46.968 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.960+0000 7fafca31c700 1 -- 172.21.15.67:0/2005230424 shutdown_connections 2023-10-14T10:36:46.968 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.960+0000 7fafca31c700 1 --2- 172.21.15.67:0/2005230424 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fafc40c5110 0x7fafc40ce510 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:46.969 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.960+0000 7fafca31c700 1 --2- 172.21.15.67:0/2005230424 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fafc415b740 0x7fafc40d1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:46.969 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.960+0000 7fafca31c700 1 --2- 172.21.15.67:0/2005230424 >> v2:172.21.15.72:3300/0 conn(0x7fafc40d0c70 0x7fafc40cdfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:46.969 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.960+0000 7fafca31c700 1 -- 172.21.15.67:0/2005230424 >> 172.21.15.67:0/2005230424 conn(0x7fafc40665a0 msgr2=0x7fafc4066980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:46.969 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.960+0000 7fafca31c700 1 -- 172.21.15.67:0/2005230424 shutdown_connections 2023-10-14T10:36:46.969 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.960+0000 7fafca31c700 1 -- 172.21.15.67:0/2005230424 wait complete. 2023-10-14T10:36:46.969 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.962+0000 7fafca31c700 1 Processor -- start 2023-10-14T10:36:46.972 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.962+0000 7fafca31c700 1 -- start start 2023-10-14T10:36:46.972 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.963+0000 7fafca31c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fafc40c5110 0x7fafc41f3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:46.973 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.963+0000 7fafca31c700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fafc40d0c70 0x7fafc406b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:46.973 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.963+0000 7fafca31c700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fafc415b740 0x7fafc40b5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:46.973 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.963+0000 7fafca31c700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafc41f4080 con 0x7fafc40c5110 2023-10-14T10:36:46.973 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.963+0000 7fafca31c700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafc406be90 con 0x7fafc415b740 2023-10-14T10:36:46.974 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.963+0000 7fafca31c700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafc406c0a0 con 0x7fafc40d0c70 2023-10-14T10:36:46.974 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.963+0000 7fafc3fff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fafc40c5110 0x7fafc41f3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:46.974 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.963+0000 7fafc3fff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fafc40c5110 0x7fafc41f3b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45382/0 (socket says 172.21.15.67:45382) 2023-10-14T10:36:46.974 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.963+0000 7fafc3fff700 1 -- 172.21.15.67:0/194545808 learned_addr learned my addr 172.21.15.67:0/194545808 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:46.975 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.963+0000 7fafc3fff700 1 -- 172.21.15.67:0/194545808 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fafc40d0c70 msgr2=0x7fafc406b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:46.975 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.963+0000 7fafc3fff700 1 --2- 172.21.15.67:0/194545808 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fafc40d0c70 0x7fafc406b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:46.975 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.963+0000 7fafc3fff700 1 -- 172.21.15.67:0/194545808 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fafc415b740 msgr2=0x7fafc40b5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:46.975 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.963+0000 7fafc3fff700 1 --2- 172.21.15.67:0/194545808 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fafc415b740 0x7fafc40b5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:46.975 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.963+0000 7fafc3fff700 1 -- 172.21.15.67:0/194545808 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafbc009b30 con 0x7fafc40c5110 2023-10-14T10:36:46.976 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.964+0000 7fafc3fff700 1 --2- 172.21.15.67:0/194545808 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fafc40c5110 0x7fafc41f3b40 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fafb400e810 tx=0x7fafb400ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:46.976 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.964+0000 7fafc0ff9700 1 -- 172.21.15.67:0/194545808 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb400c020 con 0x7fafc40c5110 2023-10-14T10:36:46.976 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.964+0000 7fafc0ff9700 1 -- 172.21.15.67:0/194545808 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafb4004250 con 0x7fafc40c5110 2023-10-14T10:36:46.976 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.964+0000 7fafca31c700 1 -- 172.21.15.67:0/194545808 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafc406c380 con 0x7fafc40c5110 2023-10-14T10:36:46.976 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.964+0000 7fafc0ff9700 1 -- 172.21.15.67:0/194545808 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb40036a0 con 0x7fafc40c5110 2023-10-14T10:36:46.976 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.964+0000 7fafca31c700 1 -- 172.21.15.67:0/194545808 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafc406cc20 con 0x7fafc40c5110 2023-10-14T10:36:46.977 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.965+0000 7fafca31c700 1 -- 172.21.15.67:0/194545808 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafc406c7a0 con 0x7fafc40c5110 2023-10-14T10:36:46.977 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.965+0000 7fafc0ff9700 1 -- 172.21.15.67:0/194545808 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fafb4003800 con 0x7fafc40c5110 2023-10-14T10:36:46.977 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.966+0000 7fafc0ff9700 1 --2- 172.21.15.67:0/194545808 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fafa406af00 0x7fafa406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:46.977 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.966+0000 7fafc0ff9700 1 -- 172.21.15.67:0/194545808 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fafb4089050 con 0x7fafc40c5110 2023-10-14T10:36:46.977 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.966+0000 7fafc37fe700 1 --2- 172.21.15.67:0/194545808 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fafa406af00 0x7fafa406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:46.978 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.967+0000 7fafc37fe700 1 --2- 172.21.15.67:0/194545808 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fafa406af00 0x7fafa406d3c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fafac006fd0 tx=0x7fafac008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:46.978 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:46.970+0000 7fafc0ff9700 1 -- 172.21.15.67:0/194545808 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fafc406c7a0 con 0x7fafc40c5110 2023-10-14T10:36:47.172 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.170+0000 7fafca31c700 1 -- 172.21.15.67:0/194545808 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fafc406c7a0 con 0x7fafc40c5110 2023-10-14T10:36:47.173 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.171+0000 7fafc0ff9700 1 -- 172.21.15.67:0/194545808 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16457 (secure 0 0 0) 0x7fafc406c7a0 con 0x7fafc40c5110 2023-10-14T10:36:47.174 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-14T10:36:47.174 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":22,"fsid":"37f9ab46-3082-4b71-8b24-08d39897b008","created":"2023-10-14T10:35:46.067554+0000","modified":"2023-10-14T10:36:04.837296+0000","last_up_change":"2023-10-14T10:35:56.668142+0000","last_in_change":"2023-10-14T10:35:48.551490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-14T10:35:57.809566+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-14T10:36:01.667146+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-14T10:36:02.219700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2b59a07e-11ef-440c-b766-9f64ac885838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6801","nonce":1287255499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6803","nonce":1287255499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6807","nonce":1287255499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6805","nonce":1287255499}]},"public_addr":"172.21.15.67:6801/1287255499","cluster_addr":"172.21.15.67:6803/1287255499","heartbeat_back_addr":"172.21.15.67:6807/1287255499","heartbeat_front_addr":"172.21.15.67:6805/1287255499","state":["exists","up"]},{"osd":1,"uuid":"6f813bd6-fa0b-487d-a204-431eb1276a60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6818","nonce":2435538787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6822","nonce":2435538787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6830","nonce":2435538787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6826","nonce":2435538787}]},"public_addr":"172.21.15.67:6818/2435538787","cluster_addr":"172.21.15.67:6822/2435538787","heartbeat_back_addr":"172.21.15.67:6830/2435538787","heartbeat_front_addr":"172.21.15.67:6826/2435538787","state":["exists","up"]},{"osd":2,"uuid":"6af42c6a-1b59-4669-9c04-b6876788446d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6809","nonce":2827491791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6811","nonce":2827491791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6815","nonce":2827491791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6813","nonce":2827491791}]},"public_addr":"172.21.15.67:6809/2827491791","cluster_addr":"172.21.15.67:6811/2827491791","heartbeat_back_addr":"172.21.15.67:6815/2827491791","heartbeat_front_addr":"172.21.15.67:6813/2827491791","state":["exists","up"]},{"osd":3,"uuid":"1d889d76-a646-499d-bfff-ed5bae5434ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6819","nonce":1848001176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6821","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6823","nonce":1848001176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6831","nonce":1848001176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6825","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6827","nonce":1848001176}]},"public_addr":"172.21.15.67:6819/1848001176","cluster_addr":"172.21.15.67:6823/1848001176","heartbeat_back_addr":"172.21.15.67:6831/1848001176","heartbeat_front_addr":"172.21.15.67:6827/1848001176","state":["exists","up"]},{"osd":4,"uuid":"c3271dff-b437-4fdd-b13b-6e49a35473af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6804","nonce":1891230785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6808","nonce":1891230785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6815","nonce":1891230785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6811","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6813","nonce":1891230785}]},"public_addr":"172.21.15.72:6804/1891230785","cluster_addr":"172.21.15.72:6808/1891230785","heartbeat_back_addr":"172.21.15.72:6815/1891230785","heartbeat_front_addr":"172.21.15.72:6813/1891230785","state":["exists","up"]},{"osd":5,"uuid":"0337bb18-4b68-4eff-b528-a667275e6926","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6801","nonce":569686991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6803","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6805","nonce":569686991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6812","nonce":569686991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6807","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6809","nonce":569686991}]},"public_addr":"172.21.15.72:6801/569686991","cluster_addr":"172.21.15.72:6805/569686991","heartbeat_back_addr":"172.21.15.72:6812/569686991","heartbeat_front_addr":"172.21.15.72:6809/569686991","state":["exists","up"]},{"osd":6,"uuid":"8e92d9cf-0c8c-46eb-89a7-db2023ef1509","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6817","nonce":3767685934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6819","nonce":3767685934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6823","nonce":3767685934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6821","nonce":3767685934}]},"public_addr":"172.21.15.72:6817/3767685934","cluster_addr":"172.21.15.72:6819/3767685934","heartbeat_back_addr":"172.21.15.72:6823/3767685934","heartbeat_front_addr":"172.21.15.72:6821/3767685934","state":["exists","up"]},{"osd":7,"uuid":"3fd6aa7d-50a9-45c9-9561-5902a044a119","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6825","nonce":3567465926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6827","nonce":3567465926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6831","nonce":3567465926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6829","nonce":3567465926}]},"public_addr":"172.21.15.72:6825/3567465926","cluster_addr":"172.21.15.72:6827/3567465926","heartbeat_back_addr":"172.21.15.72:6831/3567465926","heartbeat_front_addr":"172.21.15.72:6829/3567465926","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.511466+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.795107+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.858674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.728895+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.794842+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.750064+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.732122+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.765185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-14T10:36:47.176 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.173+0000 7fafca31c700 1 -- 172.21.15.67:0/194545808 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fafa406af00 msgr2=0x7fafa406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:47.176 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.173+0000 7fafca31c700 1 --2- 172.21.15.67:0/194545808 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fafa406af00 0x7fafa406d3c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fafac006fd0 tx=0x7fafac008040 comp rx=0 tx=0).stop 2023-10-14T10:36:47.176 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.173+0000 7fafca31c700 1 -- 172.21.15.67:0/194545808 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fafc40c5110 msgr2=0x7fafc41f3b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:47.176 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.173+0000 7fafca31c700 1 --2- 172.21.15.67:0/194545808 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fafc40c5110 0x7fafc41f3b40 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fafb400e810 tx=0x7fafb400ebd0 comp rx=0 tx=0).stop 2023-10-14T10:36:47.176 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.173+0000 7fafca31c700 1 -- 172.21.15.67:0/194545808 shutdown_connections 2023-10-14T10:36:47.177 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.173+0000 7fafca31c700 1 --2- 172.21.15.67:0/194545808 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fafa406af00 0x7fafa406d3c0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:47.177 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.173+0000 7fafca31c700 1 --2- 172.21.15.67:0/194545808 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fafc415b740 0x7fafc40b5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:47.177 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.173+0000 7fafca31c700 1 --2- 172.21.15.67:0/194545808 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fafc40c5110 0x7fafc41f3b40 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:47.177 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.173+0000 7fafca31c700 1 --2- 172.21.15.67:0/194545808 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fafc40d0c70 0x7fafc406b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:47.177 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.173+0000 7fafca31c700 1 -- 172.21.15.67:0/194545808 >> 172.21.15.67:0/194545808 conn(0x7fafc40665a0 msgr2=0x7fafc415aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:47.178 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.173+0000 7fafca31c700 1 -- 172.21.15.67:0/194545808 shutdown_connections 2023-10-14T10:36:47.178 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.174+0000 7fafca31c700 1 -- 172.21.15.67:0/194545808 wait complete. 2023-10-14T10:36:47.188 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-14T10:36:47.189 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-14T10:36:47.314 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.312+0000 7f351f005700 1 Processor -- start 2023-10-14T10:36:47.314 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.312+0000 7f351f005700 1 -- start start 2023-10-14T10:36:47.317 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.312+0000 7f351f005700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f351815e2f0 0x7f35180cede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:47.318 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.312+0000 7f351f005700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f351806a840 0x7f351806ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:47.318 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.312+0000 7f351f005700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f351806b1a0 con 0x7f35180cd950 2023-10-14T10:36:47.318 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.312+0000 7f351f005700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35181619a0 con 0x7f351815e2f0 2023-10-14T10:36:47.318 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.312+0000 7f351f005700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35180b2d60 con 0x7f351806a840 2023-10-14T10:36:47.319 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.313+0000 7f3517fff700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f351815e2f0 0x7f35180cede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:47.319 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.313+0000 7f351cda1700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f35180cd950 0x7f351815f570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:53158/0 (socket says 172.21.15.67:53158) 2023-10-14T10:36:47.319 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.313+0000 7f351cda1700 1 -- 172.21.15.67:0/2434808380 learned_addr learned my addr 172.21.15.67:0/2434808380 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:36:47.319 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.313+0000 7f351d5a2700 1 --2- 172.21.15.67:0/2434808380 >> v2:172.21.15.72:3300/0 conn(0x7f351806a840 0x7f351806ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:47.319 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.313+0000 7f3517fff700 1 -- 172.21.15.67:0/2434808380 >> v2:172.21.15.72:3300/0 conn(0x7f351806a840 msgr2=0x7f351806ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:47.320 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.313+0000 7f3517fff700 1 --2- 172.21.15.67:0/2434808380 >> v2:172.21.15.72:3300/0 conn(0x7f351806a840 0x7f351806ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:47.320 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.313+0000 7f3517fff700 1 -- 172.21.15.67:0/2434808380 >> v1:172.21.15.67:6789/0 conn(0x7f35180cd950 legacy=0x7f351815f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:47.320 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.313+0000 7f3517fff700 1 -- 172.21.15.67:0/2434808380 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35180cdd30 con 0x7f351815e2f0 2023-10-14T10:36:47.320 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.313+0000 7f35177fe700 1 -- 172.21.15.67:0/2434808380 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3879233421 0 0) 0x7f351806b1a0 con 0x7f35180cd950 2023-10-14T10:36:47.321 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.313+0000 7f351d5a2700 1 --2- 172.21.15.67:0/2434808380 >> v2:172.21.15.72:3300/0 conn(0x7f351806a840 0x7f351806ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-14T10:36:47.321 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.313+0000 7f3517fff700 1 --2- 172.21.15.67:0/2434808380 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f351815e2f0 0x7f35180cede0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f35180afed0 tx=0x7f3504009b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48f530988ebf6704 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:47.321 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.313+0000 7f35177fe700 1 -- 172.21.15.67:0/2434808380 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3504005e30 con 0x7f351815e2f0 2023-10-14T10:36:47.321 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.313+0000 7f35177fe700 1 -- 172.21.15.67:0/2434808380 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3504006030 con 0x7f351815e2f0 2023-10-14T10:36:47.322 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.313+0000 7f35177fe700 1 -- 172.21.15.67:0/2434808380 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35040063c0 con 0x7f351815e2f0 2023-10-14T10:36:47.322 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.314+0000 7f351f005700 1 -- 172.21.15.67:0/2434808380 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f351815e2f0 msgr2=0x7f35180cede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:47.322 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.314+0000 7f351f005700 1 --2- 172.21.15.67:0/2434808380 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f351815e2f0 0x7f35180cede0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f35180afed0 tx=0x7f3504009b00 comp rx=0 tx=0).stop 2023-10-14T10:36:47.322 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.314+0000 7f351f005700 1 -- 172.21.15.67:0/2434808380 shutdown_connections 2023-10-14T10:36:47.322 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.314+0000 7f351f005700 1 --2- 172.21.15.67:0/2434808380 >> v2:172.21.15.72:3300/0 conn(0x7f351806a840 0x7f351806ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:47.323 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.314+0000 7f351f005700 1 --2- 172.21.15.67:0/2434808380 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f351815e2f0 0x7f35180cede0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:47.323 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.314+0000 7f351f005700 1 -- 172.21.15.67:0/2434808380 >> 172.21.15.67:0/2434808380 conn(0x7f351806dae0 msgr2=0x7f351806def0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:47.323 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.314+0000 7f351f005700 1 -- 172.21.15.67:0/2434808380 shutdown_connections 2023-10-14T10:36:47.323 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.314+0000 7f351f005700 1 -- 172.21.15.67:0/2434808380 wait complete. 2023-10-14T10:36:47.323 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.315+0000 7f351f005700 1 Processor -- start 2023-10-14T10:36:47.324 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.315+0000 7f351f005700 1 -- start start 2023-10-14T10:36:47.326 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.316+0000 7f351f005700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f351806a840 0x7f35181f3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:47.326 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.316+0000 7f351f005700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35180cd950 0x7f351815a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:47.327 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.316+0000 7f351f005700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f351815e2f0 0x7f35180b0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:47.327 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.316+0000 7f351f005700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35180ca200 con 0x7f35180cd950 2023-10-14T10:36:47.327 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.316+0000 7f351f005700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f351815ab00 con 0x7f351815e2f0 2023-10-14T10:36:47.327 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.316+0000 7f351f005700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35180b10c0 con 0x7f351806a840 2023-10-14T10:36:47.327 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.316+0000 7f3517fff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35180cd950 0x7f351815a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:47.328 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.316+0000 7f3517fff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35180cd950 0x7f351815a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45398/0 (socket says 172.21.15.67:45398) 2023-10-14T10:36:47.328 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.316+0000 7f3517fff700 1 -- 172.21.15.67:0/3547172367 learned_addr learned my addr 172.21.15.67:0/3547172367 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:47.328 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.316+0000 7f351d5a2700 1 --2- 172.21.15.67:0/3547172367 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f351815e2f0 0x7f35180b0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:47.328 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.316+0000 7f351cda1700 1 --2- 172.21.15.67:0/3547172367 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f351806a840 0x7f35181f3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:47.328 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.316+0000 7f3517fff700 1 -- 172.21.15.67:0/3547172367 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f351806a840 msgr2=0x7f35181f3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:47.328 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.316+0000 7f3517fff700 1 --2- 172.21.15.67:0/3547172367 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f351806a840 0x7f35181f3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:47.329 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.316+0000 7f3517fff700 1 -- 172.21.15.67:0/3547172367 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f351815e2f0 msgr2=0x7f35180b0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:47.329 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.316+0000 7f3517fff700 1 --2- 172.21.15.67:0/3547172367 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f351815e2f0 0x7f35180b0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:47.329 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.316+0000 7f3517fff700 1 -- 172.21.15.67:0/3547172367 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3504009430 con 0x7f35180cd950 2023-10-14T10:36:47.329 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.316+0000 7f351cda1700 1 --2- 172.21.15.67:0/3547172367 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f351806a840 0x7f35181f3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-14T10:36:47.329 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.317+0000 7f3517fff700 1 --2- 172.21.15.67:0/3547172367 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35180cd950 0x7f351815a5c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f3504009fd0 tx=0x7f3504004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:47.330 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.317+0000 7f35157fa700 1 -- 172.21.15.67:0/3547172367 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3504006960 con 0x7f35180cd950 2023-10-14T10:36:47.330 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.317+0000 7f35157fa700 1 -- 172.21.15.67:0/3547172367 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3504020070 con 0x7f35180cd950 2023-10-14T10:36:47.330 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.317+0000 7f351f005700 1 -- 172.21.15.67:0/3547172367 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35181f4120 con 0x7f35180cd950 2023-10-14T10:36:47.330 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.317+0000 7f35157fa700 1 -- 172.21.15.67:0/3547172367 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3504004280 con 0x7f35180cd950 2023-10-14T10:36:47.330 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.317+0000 7f351f005700 1 -- 172.21.15.67:0/3547172367 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35181f46e0 con 0x7f35180cd950 2023-10-14T10:36:47.331 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.318+0000 7f351f005700 1 -- 172.21.15.67:0/3547172367 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35180ca800 con 0x7f35180cd950 2023-10-14T10:36:47.331 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.318+0000 7f35157fa700 1 -- 172.21.15.67:0/3547172367 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f3504022030 con 0x7f35180cd950 2023-10-14T10:36:47.331 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.319+0000 7f35157fa700 1 --2- 172.21.15.67:0/3547172367 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f34fc06af00 0x7f34fc06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:47.331 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.319+0000 7f35157fa700 1 -- 172.21.15.67:0/3547172367 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f350408ab50 con 0x7f35180cd950 2023-10-14T10:36:47.331 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.319+0000 7f351cda1700 1 --2- 172.21.15.67:0/3547172367 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f34fc06af00 0x7f34fc06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:47.332 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.319+0000 7f351cda1700 1 --2- 172.21.15.67:0/3547172367 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f34fc06af00 0x7f34fc06d3c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f350c000f60 tx=0x7f350c007ca0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:47.332 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.323+0000 7f35157fa700 1 -- 172.21.15.67:0/3547172367 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3504055c90 con 0x7f35180cd950 2023-10-14T10:36:47.567 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.565+0000 7f351f005700 1 -- 172.21.15.67:0/3547172367 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3518067d20 con 0x7f35180cd950 2023-10-14T10:36:47.568 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.565+0000 7f35157fa700 1 -- 172.21.15.67:0/3547172367 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4901 (secure 0 0 0) 0x7f35040592b0 con 0x7f35180cd950 2023-10-14T10:36:47.568 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-14T10:36:47.568 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4217,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6836/838298018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":838298018},{"type":"v1","addr":"172.21.15.72:6836","nonce":838298018}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4255,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/1687192726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6833","nonce":1687192726},{"type":"v1","addr":"172.21.15.67:6835","nonce":1687192726}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6834/3936231765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":3936231765},{"type":"v1","addr":"172.21.15.67:6834","nonce":3936231765}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-14T10:36:03.247976+0000","modified":"2023-10-14T10:36:06.283381+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4267},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4267":{"gid":4267,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.72:6837/757500960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6835","nonce":757500960},{"type":"v1","addr":"172.21.15.72:6837","nonce":757500960}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-14T10:36:47.570 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.567+0000 7f351f005700 1 -- 172.21.15.67:0/3547172367 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f34fc06af00 msgr2=0x7f34fc06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:47.570 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.567+0000 7f351f005700 1 --2- 172.21.15.67:0/3547172367 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f34fc06af00 0x7f34fc06d3c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f350c000f60 tx=0x7f350c007ca0 comp rx=0 tx=0).stop 2023-10-14T10:36:47.570 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.567+0000 7f351f005700 1 -- 172.21.15.67:0/3547172367 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35180cd950 msgr2=0x7f351815a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:47.571 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.567+0000 7f351f005700 1 --2- 172.21.15.67:0/3547172367 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35180cd950 0x7f351815a5c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f3504009fd0 tx=0x7f3504004750 comp rx=0 tx=0).stop 2023-10-14T10:36:47.571 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.567+0000 7f351f005700 1 -- 172.21.15.67:0/3547172367 shutdown_connections 2023-10-14T10:36:47.571 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.567+0000 7f351f005700 1 --2- 172.21.15.67:0/3547172367 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f34fc06af00 0x7f34fc06d3c0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:47.571 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.567+0000 7f351f005700 1 --2- 172.21.15.67:0/3547172367 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f351815e2f0 0x7f35180b0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:47.571 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.567+0000 7f351f005700 1 --2- 172.21.15.67:0/3547172367 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f351806a840 0x7f35181f3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:47.572 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.567+0000 7f351f005700 1 --2- 172.21.15.67:0/3547172367 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35180cd950 0x7f351815a5c0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:47.572 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.568+0000 7f351f005700 1 -- 172.21.15.67:0/3547172367 >> 172.21.15.67:0/3547172367 conn(0x7f351806dae0 msgr2=0x7f351806ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:47.572 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.568+0000 7f351f005700 1 -- 172.21.15.67:0/3547172367 shutdown_connections 2023-10-14T10:36:47.572 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.568+0000 7f351f005700 1 -- 172.21.15.67:0/3547172367 wait complete. 2023-10-14T10:36:47.573 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 8 2023-10-14T10:36:47.582 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-14T10:36:47.707 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.705+0000 7f581759c700 1 Processor -- start 2023-10-14T10:36:47.707 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.705+0000 7f581759c700 1 -- start start 2023-10-14T10:36:47.709 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.706+0000 7f581759c700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f581015e030 0x7f58100cb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:47.710 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.706+0000 7f581759c700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f581006adb0 con 0x7f58100ce280 2023-10-14T10:36:47.710 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.706+0000 7f581759c700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f581006b090 con 0x7f581006a840 2023-10-14T10:36:47.710 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.706+0000 7f581759c700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f581006b370 con 0x7f581015e030 2023-10-14T10:36:47.710 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.706+0000 7f5815338700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f581015e030 0x7f58100cb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:47.711 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.706+0000 7f5815338700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f581015e030 0x7f58100cb3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45404/0 (socket says 172.21.15.67:45404) 2023-10-14T10:36:47.711 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.706+0000 7f5815b39700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f58100ce280 0x7f581006d3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:53176/0 (socket says 172.21.15.67:53176) 2023-10-14T10:36:47.711 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.706+0000 7f5815338700 1 -- 172.21.15.67:0/3254118934 learned_addr learned my addr 172.21.15.67:0/3254118934 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:47.711 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.706+0000 7f5815338700 1 -- 172.21.15.67:0/3254118934 >> v1:172.21.15.72:6789/0 conn(0x7f581006a840 legacy=0x7f58100cdb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:47.711 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.706+0000 7f5815338700 1 -- 172.21.15.67:0/3254118934 >> v1:172.21.15.67:6789/0 conn(0x7f58100ce280 legacy=0x7f581006d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:47.712 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.706+0000 7f5807fff700 1 -- 172.21.15.67:0/3254118934 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 152871940 0 0) 0x7f581006adb0 con 0x7f58100ce280 2023-10-14T10:36:47.712 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.706+0000 7f5815338700 1 -- 172.21.15.67:0/3254118934 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f581006b650 con 0x7f581015e030 2023-10-14T10:36:47.712 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.706+0000 7f5815338700 1 --2- 172.21.15.67:0/3254118934 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f581015e030 0x7f58100cb3a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f580c0098f0 tx=0x7f580c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9197e99dd277c28f server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:47.712 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.706+0000 7f5807fff700 1 -- 172.21.15.67:0/3254118934 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f580c00b8a0 con 0x7f581015e030 2023-10-14T10:36:47.712 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.707+0000 7f5807fff700 1 -- 172.21.15.67:0/3254118934 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f580c00ba00 con 0x7f581015e030 2023-10-14T10:36:47.713 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.707+0000 7f5807fff700 1 -- 172.21.15.67:0/3254118934 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f580c00bd90 con 0x7f581015e030 2023-10-14T10:36:47.713 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.707+0000 7f581759c700 1 -- 172.21.15.67:0/3254118934 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f581015e030 msgr2=0x7f58100cb3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:47.713 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.707+0000 7f581759c700 1 --2- 172.21.15.67:0/3254118934 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f581015e030 0x7f58100cb3a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f580c0098f0 tx=0x7f580c009c00 comp rx=0 tx=0).stop 2023-10-14T10:36:47.713 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.707+0000 7f581759c700 1 -- 172.21.15.67:0/3254118934 shutdown_connections 2023-10-14T10:36:47.713 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.707+0000 7f581759c700 1 --2- 172.21.15.67:0/3254118934 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f581015e030 0x7f58100cb3a0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:47.714 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.707+0000 7f581759c700 1 -- 172.21.15.67:0/3254118934 >> 172.21.15.67:0/3254118934 conn(0x7f58100cc6f0 msgr2=0x7f58100ccb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:47.714 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.707+0000 7f581759c700 1 -- 172.21.15.67:0/3254118934 shutdown_connections 2023-10-14T10:36:47.714 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.707+0000 7f581759c700 1 -- 172.21.15.67:0/3254118934 wait complete. 2023-10-14T10:36:47.717 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.708+0000 7f581759c700 1 Processor -- start 2023-10-14T10:36:47.717 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.708+0000 7f581759c700 1 -- start start 2023-10-14T10:36:47.717 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.709+0000 7f581759c700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f581006a840 0x7f58101f2de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:47.718 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.709+0000 7f581759c700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f58100ce280 0x7f58100b3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:47.718 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.709+0000 7f581759c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f581015e030 0x7f58100b44c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:47.718 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.709+0000 7f581759c700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5810150ab0 con 0x7f581015e030 2023-10-14T10:36:47.718 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.709+0000 7f581759c700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58100b4d30 con 0x7f58100ce280 2023-10-14T10:36:47.718 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.709+0000 7f581759c700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58100b5010 con 0x7f581006a840 2023-10-14T10:36:47.719 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.709+0000 7f5815b39700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f581015e030 0x7f58100b44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:47.719 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.709+0000 7f5815b39700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f581015e030 0x7f58100b44c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45414/0 (socket says 172.21.15.67:45414) 2023-10-14T10:36:47.719 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.709+0000 7f5815b39700 1 -- 172.21.15.67:0/2736687718 learned_addr learned my addr 172.21.15.67:0/2736687718 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:47.719 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.709+0000 7f5814b37700 1 --2- 172.21.15.67:0/2736687718 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f58100ce280 0x7f58100b3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:47.719 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.710+0000 7f5815338700 1 --2- 172.21.15.67:0/2736687718 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f581006a840 0x7f58101f2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:47.720 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.710+0000 7f5815b39700 1 -- 172.21.15.67:0/2736687718 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f581006a840 msgr2=0x7f58101f2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:47.720 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.710+0000 7f5815b39700 1 --2- 172.21.15.67:0/2736687718 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f581006a840 0x7f58101f2de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:47.720 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.710+0000 7f5815b39700 1 -- 172.21.15.67:0/2736687718 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f58100ce280 msgr2=0x7f58100b3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:47.720 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.710+0000 7f5815b39700 1 --2- 172.21.15.67:0/2736687718 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f58100ce280 0x7f58100b3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:47.720 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.710+0000 7f5815b39700 1 -- 172.21.15.67:0/2736687718 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f580c009550 con 0x7f581015e030 2023-10-14T10:36:47.721 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.710+0000 7f5814b37700 1 --2- 172.21.15.67:0/2736687718 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f58100ce280 0x7f58100b3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-14T10:36:47.721 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.710+0000 7f5815b39700 1 --2- 172.21.15.67:0/2736687718 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f581015e030 0x7f58100b44c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f580800cc70 tx=0x7f580800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:47.721 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.710+0000 7f5805ffb700 1 -- 172.21.15.67:0/2736687718 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f580800ad80 con 0x7f581015e030 2023-10-14T10:36:47.721 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.710+0000 7f5805ffb700 1 -- 172.21.15.67:0/2736687718 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f580800ce70 con 0x7f581015e030 2023-10-14T10:36:47.721 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.710+0000 7f581759c700 1 -- 172.21.15.67:0/2736687718 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58100b52f0 con 0x7f581015e030 2023-10-14T10:36:47.722 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.710+0000 7f5805ffb700 1 -- 172.21.15.67:0/2736687718 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58080072f0 con 0x7f581015e030 2023-10-14T10:36:47.722 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.710+0000 7f581759c700 1 -- 172.21.15.67:0/2736687718 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58100b58b0 con 0x7f581015e030 2023-10-14T10:36:47.722 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.711+0000 7f581759c700 1 -- 172.21.15.67:0/2736687718 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58100c9d70 con 0x7f581015e030 2023-10-14T10:36:47.722 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.711+0000 7f5805ffb700 1 -- 172.21.15.67:0/2736687718 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f5808007450 con 0x7f581015e030 2023-10-14T10:36:47.723 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.712+0000 7f5805ffb700 1 --2- 172.21.15.67:0/2736687718 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f57f406af30 0x7f57f406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:47.723 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.712+0000 7f5805ffb700 1 -- 172.21.15.67:0/2736687718 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5808089170 con 0x7f581015e030 2023-10-14T10:36:47.723 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.712+0000 7f5815338700 1 --2- 172.21.15.67:0/2736687718 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f57f406af30 0x7f57f406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:47.723 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.712+0000 7f5815338700 1 --2- 172.21.15.67:0/2736687718 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f57f406af30 0x7f57f406d3f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f580c009c40 tx=0x7f580c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:47.724 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.716+0000 7f5805ffb700 1 -- 172.21.15.67:0/2736687718 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5808054190 con 0x7f581015e030 2023-10-14T10:36:47.965 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.963+0000 7f581759c700 1 -- 172.21.15.67:0/2736687718 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5810066a30 con 0x7f581015e030 2023-10-14T10:36:47.966 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.964+0000 7f5805ffb700 1 -- 172.21.15.67:0/2736687718 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4901 (secure 0 0 0) 0x7f5808018090 con 0x7f581015e030 2023-10-14T10:36:47.966 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-14T10:36:47.967 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4217,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6836/838298018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":838298018},{"type":"v1","addr":"172.21.15.72:6836","nonce":838298018}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4255,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/1687192726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6833","nonce":1687192726},{"type":"v1","addr":"172.21.15.67:6835","nonce":1687192726}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6834/3936231765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":3936231765},{"type":"v1","addr":"172.21.15.67:6834","nonce":3936231765}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-14T10:36:03.247976+0000","modified":"2023-10-14T10:36:06.283381+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4267},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4267":{"gid":4267,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.72:6837/757500960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6835","nonce":757500960},{"type":"v1","addr":"172.21.15.72:6837","nonce":757500960}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-14T10:36:47.968 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.966+0000 7f581759c700 1 -- 172.21.15.67:0/2736687718 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f57f406af30 msgr2=0x7f57f406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:47.968 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.966+0000 7f581759c700 1 --2- 172.21.15.67:0/2736687718 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f57f406af30 0x7f57f406d3f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f580c009c40 tx=0x7f580c008030 comp rx=0 tx=0).stop 2023-10-14T10:36:47.969 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.966+0000 7f581759c700 1 -- 172.21.15.67:0/2736687718 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f581015e030 msgr2=0x7f58100b44c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:47.969 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.966+0000 7f581759c700 1 --2- 172.21.15.67:0/2736687718 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f581015e030 0x7f58100b44c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f580800cc70 tx=0x7f580800a340 comp rx=0 tx=0).stop 2023-10-14T10:36:47.969 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.966+0000 7f581759c700 1 -- 172.21.15.67:0/2736687718 shutdown_connections 2023-10-14T10:36:47.969 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.966+0000 7f581759c700 1 --2- 172.21.15.67:0/2736687718 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f57f406af30 0x7f57f406d3f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:47.969 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.966+0000 7f581759c700 1 --2- 172.21.15.67:0/2736687718 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f581015e030 0x7f58100b44c0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:47.970 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.966+0000 7f581759c700 1 --2- 172.21.15.67:0/2736687718 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f581006a840 0x7f58101f2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:47.970 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.966+0000 7f581759c700 1 --2- 172.21.15.67:0/2736687718 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f58100ce280 0x7f58100b3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:47.970 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.966+0000 7f581759c700 1 -- 172.21.15.67:0/2736687718 >> 172.21.15.67:0/2736687718 conn(0x7f58100cc6f0 msgr2=0x7f581015bb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:47.970 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.966+0000 7f581759c700 1 -- 172.21.15.67:0/2736687718 shutdown_connections 2023-10-14T10:36:47.970 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:47.966+0000 7f581759c700 1 -- 172.21.15.67:0/2736687718 wait complete. 2023-10-14T10:36:47.971 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 8 2023-10-14T10:36:47.980 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-14T10:36:48.106 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.104+0000 7fe99ea61700 1 Processor -- start 2023-10-14T10:36:48.107 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.104+0000 7fe99ea61700 1 -- start start 2023-10-14T10:36:48.117 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.105+0000 7fe99ea61700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe9980ceb10 0x7fe9980cef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:48.117 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.105+0000 7fe99ea61700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fe9980b0cc0 0x7fe9980b1110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:48.117 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.105+0000 7fe99ea61700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9980af9d0 con 0x7fe99815a940 2023-10-14T10:36:48.118 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.105+0000 7fe99ea61700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9980ad140 con 0x7fe9980ceb10 2023-10-14T10:36:48.118 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.105+0000 7fe99ea61700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9980cf4c0 con 0x7fe9980b0cc0 2023-10-14T10:36:48.118 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.105+0000 7fe9977fe700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe9980ceb10 0x7fe9980cef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:48.119 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.105+0000 7fe997fff700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fe99815a940 0x7fe9980cbbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:53186/0 (socket says 172.21.15.67:53186) 2023-10-14T10:36:48.119 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.105+0000 7fe9977fe700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe9980ceb10 0x7fe9980cef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45420/0 (socket says 172.21.15.67:45420) 2023-10-14T10:36:48.120 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.105+0000 7fe997fff700 1 -- 172.21.15.67:0/4082629050 learned_addr learned my addr 172.21.15.67:0/4082629050 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:36:48.120 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.105+0000 7fe99cffe700 1 --2- 172.21.15.67:0/4082629050 >> v2:172.21.15.72:3300/0 conn(0x7fe9980b0cc0 0x7fe9980b1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:48.120 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.105+0000 7fe9977fe700 1 -- 172.21.15.67:0/4082629050 >> v2:172.21.15.72:3300/0 conn(0x7fe9980b0cc0 msgr2=0x7fe9980b1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:48.121 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.105+0000 7fe9977fe700 1 --2- 172.21.15.67:0/4082629050 >> v2:172.21.15.72:3300/0 conn(0x7fe9980b0cc0 0x7fe9980b1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:48.121 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.105+0000 7fe9977fe700 1 -- 172.21.15.67:0/4082629050 >> v1:172.21.15.67:6789/0 conn(0x7fe99815a940 legacy=0x7fe9980cbbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:48.121 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.105+0000 7fe996ffd700 1 -- 172.21.15.67:0/4082629050 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3751062500 0 0) 0x7fe9980af9d0 con 0x7fe99815a940 2023-10-14T10:36:48.122 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.105+0000 7fe9977fe700 1 -- 172.21.15.67:0/4082629050 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9980afc20 con 0x7fe9980ceb10 2023-10-14T10:36:48.122 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.106+0000 7fe99cffe700 1 --2- 172.21.15.67:0/4082629050 >> v2:172.21.15.72:3300/0 conn(0x7fe9980b0cc0 0x7fe9980b1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-14T10:36:48.123 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.106+0000 7fe9977fe700 1 --2- 172.21.15.67:0/4082629050 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9980ceb10 0x7fe9980cef80 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe99806d600 tx=0x7fe98c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b89c017f31f336e server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:48.123 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.106+0000 7fe996ffd700 1 -- 172.21.15.67:0/4082629050 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe98c011830 con 0x7fe9980ceb10 2023-10-14T10:36:48.123 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.106+0000 7fe996ffd700 1 -- 172.21.15.67:0/4082629050 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe98c011990 con 0x7fe9980ceb10 2023-10-14T10:36:48.124 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.106+0000 7fe996ffd700 1 -- 172.21.15.67:0/4082629050 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe98c011d20 con 0x7fe9980ceb10 2023-10-14T10:36:48.124 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.106+0000 7fe99ea61700 1 -- 172.21.15.67:0/4082629050 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9980ceb10 msgr2=0x7fe9980cef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:48.124 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.106+0000 7fe99ea61700 1 --2- 172.21.15.67:0/4082629050 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9980ceb10 0x7fe9980cef80 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe99806d600 tx=0x7fe98c009b50 comp rx=0 tx=0).stop 2023-10-14T10:36:48.125 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.106+0000 7fe99ea61700 1 -- 172.21.15.67:0/4082629050 shutdown_connections 2023-10-14T10:36:48.125 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.106+0000 7fe99ea61700 1 --2- 172.21.15.67:0/4082629050 >> v2:172.21.15.72:3300/0 conn(0x7fe9980b0cc0 0x7fe9980b1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:48.126 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.106+0000 7fe99ea61700 1 --2- 172.21.15.67:0/4082629050 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9980ceb10 0x7fe9980cef80 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:48.126 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.106+0000 7fe99ea61700 1 -- 172.21.15.67:0/4082629050 >> 172.21.15.67:0/4082629050 conn(0x7fe998099a30 msgr2=0x7fe9980d0530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:48.126 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.107+0000 7fe99ea61700 1 -- 172.21.15.67:0/4082629050 shutdown_connections 2023-10-14T10:36:48.127 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.107+0000 7fe99ea61700 1 -- 172.21.15.67:0/4082629050 wait complete. 2023-10-14T10:36:48.127 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.108+0000 7fe99ea61700 1 Processor -- start 2023-10-14T10:36:48.128 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.108+0000 7fe99ea61700 1 -- start start 2023-10-14T10:36:48.128 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.109+0000 7fe99ea61700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe9980b0cc0 0x7fe9981575f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:48.128 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.109+0000 7fe99ea61700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9980ceb10 0x7fe9980c65c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:48.129 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.109+0000 7fe99ea61700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fe99815a940 0x7fe9980cd910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:48.129 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.109+0000 7fe99ea61700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9980c5fd0 con 0x7fe9980ceb10 2023-10-14T10:36:48.129 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.109+0000 7fe99ea61700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9980c5b40 con 0x7fe9980b0cc0 2023-10-14T10:36:48.130 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.109+0000 7fe99ea61700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9980c4f50 con 0x7fe99815a940 2023-10-14T10:36:48.130 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.109+0000 7fe9977fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9980ceb10 0x7fe9980c65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:48.130 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.109+0000 7fe9977fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9980ceb10 0x7fe9980c65c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45426/0 (socket says 172.21.15.67:45426) 2023-10-14T10:36:48.131 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.109+0000 7fe9977fe700 1 -- 172.21.15.67:0/3518366914 learned_addr learned my addr 172.21.15.67:0/3518366914 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:48.131 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.109+0000 7fe997fff700 1 --2- 172.21.15.67:0/3518366914 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe9980b0cc0 0x7fe9981575f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:48.132 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.109+0000 7fe99cffe700 1 --2- 172.21.15.67:0/3518366914 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fe99815a940 0x7fe9980cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:48.132 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.109+0000 7fe9977fe700 1 -- 172.21.15.67:0/3518366914 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fe99815a940 msgr2=0x7fe9980cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:48.132 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.109+0000 7fe9977fe700 1 --2- 172.21.15.67:0/3518366914 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fe99815a940 0x7fe9980cd910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:48.133 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.109+0000 7fe9977fe700 1 -- 172.21.15.67:0/3518366914 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe9980b0cc0 msgr2=0x7fe9981575f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:48.133 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.109+0000 7fe9977fe700 1 --2- 172.21.15.67:0/3518366914 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe9980b0cc0 0x7fe9981575f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:48.133 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.109+0000 7fe9977fe700 1 -- 172.21.15.67:0/3518366914 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe98c0094a0 con 0x7fe9980ceb10 2023-10-14T10:36:48.134 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.109+0000 7fe997fff700 1 --2- 172.21.15.67:0/3518366914 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe9980b0cc0 0x7fe9981575f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-14T10:36:48.134 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.109+0000 7fe99cffe700 1 --2- 172.21.15.67:0/3518366914 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fe99815a940 0x7fe9980cd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-14T10:36:48.135 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.109+0000 7fe9977fe700 1 --2- 172.21.15.67:0/3518366914 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9980ceb10 0x7fe9980c65c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fe98c0090b0 tx=0x7fe98c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:48.135 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.109+0000 7fe994ff9700 1 -- 172.21.15.67:0/3518366914 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe98c0038a0 con 0x7fe9980ceb10 2023-10-14T10:36:48.135 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.109+0000 7fe994ff9700 1 -- 172.21.15.67:0/3518366914 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe98c01f070 con 0x7fe9980ceb10 2023-10-14T10:36:48.136 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.109+0000 7fe99ea61700 1 -- 172.21.15.67:0/3518366914 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9980c5210 con 0x7fe9980ceb10 2023-10-14T10:36:48.136 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.109+0000 7fe994ff9700 1 -- 172.21.15.67:0/3518366914 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe98c003c40 con 0x7fe9980ceb10 2023-10-14T10:36:48.136 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.109+0000 7fe99ea61700 1 -- 172.21.15.67:0/3518366914 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9980c57d0 con 0x7fe9980ceb10 2023-10-14T10:36:48.137 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.110+0000 7fe99ea61700 1 -- 172.21.15.67:0/3518366914 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9980c5960 con 0x7fe9980ceb10 2023-10-14T10:36:48.137 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.111+0000 7fe994ff9700 1 -- 172.21.15.67:0/3518366914 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe98c026480 con 0x7fe9980ceb10 2023-10-14T10:36:48.138 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.111+0000 7fe994ff9700 1 --2- 172.21.15.67:0/3518366914 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fe98406af00 0x7fe98406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:48.138 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.111+0000 7fe994ff9700 1 -- 172.21.15.67:0/3518366914 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe98c021070 con 0x7fe9980ceb10 2023-10-14T10:36:48.138 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.111+0000 7fe997fff700 1 --2- 172.21.15.67:0/3518366914 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fe98406af00 0x7fe98406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:48.139 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.112+0000 7fe997fff700 1 --2- 172.21.15.67:0/3518366914 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fe98406af00 0x7fe98406d3c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe988001730 tx=0x7fe988009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:48.140 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.116+0000 7fe994ff9700 1 -- 172.21.15.67:0/3518366914 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe98c0bdca0 con 0x7fe9980ceb10 2023-10-14T10:36:48.318 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.316+0000 7fe99ea61700 1 -- 172.21.15.67:0/3518366914 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe9980b01b0 con 0x7fe9980ceb10 2023-10-14T10:36:48.319 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.317+0000 7fe994ff9700 1 -- 172.21.15.67:0/3518366914 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16457 (secure 0 0 0) 0x7fe98c0561e0 con 0x7fe9980ceb10 2023-10-14T10:36:48.320 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-14T10:36:48.320 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":22,"fsid":"37f9ab46-3082-4b71-8b24-08d39897b008","created":"2023-10-14T10:35:46.067554+0000","modified":"2023-10-14T10:36:04.837296+0000","last_up_change":"2023-10-14T10:35:56.668142+0000","last_in_change":"2023-10-14T10:35:48.551490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-14T10:35:57.809566+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-14T10:36:01.667146+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-14T10:36:02.219700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2b59a07e-11ef-440c-b766-9f64ac885838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6801","nonce":1287255499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6803","nonce":1287255499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6807","nonce":1287255499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6805","nonce":1287255499}]},"public_addr":"172.21.15.67:6801/1287255499","cluster_addr":"172.21.15.67:6803/1287255499","heartbeat_back_addr":"172.21.15.67:6807/1287255499","heartbeat_front_addr":"172.21.15.67:6805/1287255499","state":["exists","up"]},{"osd":1,"uuid":"6f813bd6-fa0b-487d-a204-431eb1276a60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6818","nonce":2435538787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6822","nonce":2435538787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6830","nonce":2435538787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6826","nonce":2435538787}]},"public_addr":"172.21.15.67:6818/2435538787","cluster_addr":"172.21.15.67:6822/2435538787","heartbeat_back_addr":"172.21.15.67:6830/2435538787","heartbeat_front_addr":"172.21.15.67:6826/2435538787","state":["exists","up"]},{"osd":2,"uuid":"6af42c6a-1b59-4669-9c04-b6876788446d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6809","nonce":2827491791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6811","nonce":2827491791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6815","nonce":2827491791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6813","nonce":2827491791}]},"public_addr":"172.21.15.67:6809/2827491791","cluster_addr":"172.21.15.67:6811/2827491791","heartbeat_back_addr":"172.21.15.67:6815/2827491791","heartbeat_front_addr":"172.21.15.67:6813/2827491791","state":["exists","up"]},{"osd":3,"uuid":"1d889d76-a646-499d-bfff-ed5bae5434ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6819","nonce":1848001176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6821","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6823","nonce":1848001176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6831","nonce":1848001176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6825","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6827","nonce":1848001176}]},"public_addr":"172.21.15.67:6819/1848001176","cluster_addr":"172.21.15.67:6823/1848001176","heartbeat_back_addr":"172.21.15.67:6831/1848001176","heartbeat_front_addr":"172.21.15.67:6827/1848001176","state":["exists","up"]},{"osd":4,"uuid":"c3271dff-b437-4fdd-b13b-6e49a35473af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6804","nonce":1891230785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6808","nonce":1891230785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6815","nonce":1891230785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6811","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6813","nonce":1891230785}]},"public_addr":"172.21.15.72:6804/1891230785","cluster_addr":"172.21.15.72:6808/1891230785","heartbeat_back_addr":"172.21.15.72:6815/1891230785","heartbeat_front_addr":"172.21.15.72:6813/1891230785","state":["exists","up"]},{"osd":5,"uuid":"0337bb18-4b68-4eff-b528-a667275e6926","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6801","nonce":569686991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6803","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6805","nonce":569686991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6812","nonce":569686991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6807","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6809","nonce":569686991}]},"public_addr":"172.21.15.72:6801/569686991","cluster_addr":"172.21.15.72:6805/569686991","heartbeat_back_addr":"172.21.15.72:6812/569686991","heartbeat_front_addr":"172.21.15.72:6809/569686991","state":["exists","up"]},{"osd":6,"uuid":"8e92d9cf-0c8c-46eb-89a7-db2023ef1509","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6817","nonce":3767685934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6819","nonce":3767685934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6823","nonce":3767685934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6821","nonce":3767685934}]},"public_addr":"172.21.15.72:6817/3767685934","cluster_addr":"172.21.15.72:6819/3767685934","heartbeat_back_addr":"172.21.15.72:6823/3767685934","heartbeat_front_addr":"172.21.15.72:6821/3767685934","state":["exists","up"]},{"osd":7,"uuid":"3fd6aa7d-50a9-45c9-9561-5902a044a119","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6825","nonce":3567465926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6827","nonce":3567465926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6831","nonce":3567465926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6829","nonce":3567465926}]},"public_addr":"172.21.15.72:6825/3567465926","cluster_addr":"172.21.15.72:6827/3567465926","heartbeat_back_addr":"172.21.15.72:6831/3567465926","heartbeat_front_addr":"172.21.15.72:6829/3567465926","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.511466+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.795107+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.858674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.728895+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.794842+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.750064+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.732122+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.765185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-14T10:36:48.322 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.319+0000 7fe99ea61700 1 -- 172.21.15.67:0/3518366914 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fe98406af00 msgr2=0x7fe98406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:48.322 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.319+0000 7fe99ea61700 1 --2- 172.21.15.67:0/3518366914 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fe98406af00 0x7fe98406d3c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe988001730 tx=0x7fe988009040 comp rx=0 tx=0).stop 2023-10-14T10:36:48.322 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.319+0000 7fe99ea61700 1 -- 172.21.15.67:0/3518366914 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9980ceb10 msgr2=0x7fe9980c65c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:48.322 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.319+0000 7fe99ea61700 1 --2- 172.21.15.67:0/3518366914 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9980ceb10 0x7fe9980c65c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fe98c0090b0 tx=0x7fe98c004750 comp rx=0 tx=0).stop 2023-10-14T10:36:48.323 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.319+0000 7fe99ea61700 1 -- 172.21.15.67:0/3518366914 shutdown_connections 2023-10-14T10:36:48.323 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.319+0000 7fe99ea61700 1 --2- 172.21.15.67:0/3518366914 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fe98406af00 0x7fe98406d3c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:48.323 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.319+0000 7fe99ea61700 1 --2- 172.21.15.67:0/3518366914 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fe99815a940 0x7fe9980cd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:48.323 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.319+0000 7fe99ea61700 1 --2- 172.21.15.67:0/3518366914 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe9980b0cc0 0x7fe9981575f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:48.324 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.319+0000 7fe99ea61700 1 --2- 172.21.15.67:0/3518366914 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9980ceb10 0x7fe9980c65c0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:48.324 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.319+0000 7fe99ea61700 1 -- 172.21.15.67:0/3518366914 >> 172.21.15.67:0/3518366914 conn(0x7fe998099a30 msgr2=0x7fe99806f040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:48.324 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.320+0000 7fe99ea61700 1 -- 172.21.15.67:0/3518366914 shutdown_connections 2023-10-14T10:36:48.324 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.320+0000 7fe99ea61700 1 -- 172.21.15.67:0/3518366914 wait complete. 2023-10-14T10:36:48.334 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-14T10:36:48.460 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.458+0000 7fea69c31700 1 Processor -- start 2023-10-14T10:36:48.460 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.458+0000 7fea69c31700 1 -- start start 2023-10-14T10:36:48.473 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.458+0000 7fea69c31700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fea640cc3f0 0x7fea640cc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:48.473 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.458+0000 7fea69c31700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fea64161670 0x7fea6406a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:48.473 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.458+0000 7fea69c31700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fea640ccd10 0x7fea6406ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:48.474 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.458+0000 7fea69c31700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea64071ab0 con 0x7fea640cc3f0 2023-10-14T10:36:48.474 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.458+0000 7fea69c31700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea64161bc0 con 0x7fea640ccd10 2023-10-14T10:36:48.475 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.458+0000 7fea69c31700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea64161ea0 con 0x7fea64161670 2023-10-14T10:36:48.475 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.459+0000 7fea637fe700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fea640cc3f0 0x7fea640cc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:48.475 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.459+0000 7fea637fe700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fea640cc3f0 0x7fea640cc7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45432/0 (socket says 172.21.15.67:45432) 2023-10-14T10:36:48.476 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.459+0000 7fea637fe700 1 -- 172.21.15.67:0/1395188518 learned_addr learned my addr 172.21.15.67:0/1395188518 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:48.476 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.459+0000 7fea63fff700 1 --2- 172.21.15.67:0/1395188518 >> v2:172.21.15.72:3300/0 conn(0x7fea640ccd10 0x7fea6406ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:48.476 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.459+0000 7fea62ffd700 1 --2- 172.21.15.67:0/1395188518 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fea64161670 0x7fea6406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:48.477 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.459+0000 7fea637fe700 1 -- 172.21.15.67:0/1395188518 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fea64161670 msgr2=0x7fea6406a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:48.477 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.459+0000 7fea637fe700 1 --2- 172.21.15.67:0/1395188518 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fea64161670 0x7fea6406a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:48.478 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.459+0000 7fea637fe700 1 -- 172.21.15.67:0/1395188518 >> v2:172.21.15.72:3300/0 conn(0x7fea640ccd10 msgr2=0x7fea6406ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:48.478 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.459+0000 7fea637fe700 1 --2- 172.21.15.67:0/1395188518 >> v2:172.21.15.72:3300/0 conn(0x7fea640ccd10 0x7fea6406ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:48.478 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.459+0000 7fea637fe700 1 -- 172.21.15.67:0/1395188518 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea64162180 con 0x7fea640cc3f0 2023-10-14T10:36:48.479 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.459+0000 7fea637fe700 1 --2- 172.21.15.67:0/1395188518 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea640cc3f0 0x7fea640cc7d0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fea640c7480 tx=0x7fea54009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7221bae04065344 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:48.479 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.459+0000 7fea627fc700 1 -- 172.21.15.67:0/1395188518 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea5400b870 con 0x7fea640cc3f0 2023-10-14T10:36:48.479 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.459+0000 7fea627fc700 1 -- 172.21.15.67:0/1395188518 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea5400b9d0 con 0x7fea640cc3f0 2023-10-14T10:36:48.480 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.459+0000 7fea627fc700 1 -- 172.21.15.67:0/1395188518 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea5400bd60 con 0x7fea640cc3f0 2023-10-14T10:36:48.480 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.460+0000 7fea69c31700 1 -- 172.21.15.67:0/1395188518 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea640cc3f0 msgr2=0x7fea640cc7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:48.481 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.460+0000 7fea69c31700 1 --2- 172.21.15.67:0/1395188518 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea640cc3f0 0x7fea640cc7d0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fea640c7480 tx=0x7fea54009bd0 comp rx=0 tx=0).stop 2023-10-14T10:36:48.481 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.460+0000 7fea69c31700 1 -- 172.21.15.67:0/1395188518 shutdown_connections 2023-10-14T10:36:48.481 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.460+0000 7fea69c31700 1 --2- 172.21.15.67:0/1395188518 >> v2:172.21.15.72:3300/0 conn(0x7fea640ccd10 0x7fea6406ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:48.482 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.460+0000 7fea69c31700 1 --2- 172.21.15.67:0/1395188518 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea640cc3f0 0x7fea640cc7d0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:48.482 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.460+0000 7fea69c31700 1 --2- 172.21.15.67:0/1395188518 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fea64161670 0x7fea6406a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:48.483 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.460+0000 7fea69c31700 1 -- 172.21.15.67:0/1395188518 >> 172.21.15.67:0/1395188518 conn(0x7fea6406dac0 msgr2=0x7fea6406dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:48.483 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.460+0000 7fea69c31700 1 -- 172.21.15.67:0/1395188518 shutdown_connections 2023-10-14T10:36:48.483 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.460+0000 7fea69c31700 1 -- 172.21.15.67:0/1395188518 wait complete. 2023-10-14T10:36:48.484 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.461+0000 7fea69c31700 1 Processor -- start 2023-10-14T10:36:48.484 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.461+0000 7fea69c31700 1 -- start start 2023-10-14T10:36:48.484 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.462+0000 7fea69c31700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea640cc3f0 0x7fea640b30d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:48.485 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.462+0000 7fea69c31700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fea640ccd10 0x7fea640b4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:48.485 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.462+0000 7fea69c31700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fea64161670 0x7fea6406dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:48.485 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.462+0000 7fea69c31700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea640b43a0 con 0x7fea640cc3f0 2023-10-14T10:36:48.486 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.462+0000 7fea69c31700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea640b3790 con 0x7fea640ccd10 2023-10-14T10:36:48.486 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.462+0000 7fea69c31700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea640b3a70 con 0x7fea64161670 2023-10-14T10:36:48.487 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.462+0000 7fea637fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea640cc3f0 0x7fea640b30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:48.487 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.462+0000 7fea637fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea640cc3f0 0x7fea640b30d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45436/0 (socket says 172.21.15.67:45436) 2023-10-14T10:36:48.488 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.462+0000 7fea637fe700 1 -- 172.21.15.67:0/2908857601 learned_addr learned my addr 172.21.15.67:0/2908857601 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:48.488 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.462+0000 7fea62ffd700 1 --2- 172.21.15.67:0/2908857601 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fea640ccd10 0x7fea640b4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:48.488 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.462+0000 7fea63fff700 1 --2- 172.21.15.67:0/2908857601 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fea64161670 0x7fea6406dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:48.489 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.462+0000 7fea637fe700 1 -- 172.21.15.67:0/2908857601 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fea64161670 msgr2=0x7fea6406dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:48.489 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.462+0000 7fea637fe700 1 --2- 172.21.15.67:0/2908857601 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fea64161670 0x7fea6406dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:48.489 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.462+0000 7fea637fe700 1 -- 172.21.15.67:0/2908857601 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fea640ccd10 msgr2=0x7fea640b4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:48.490 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.462+0000 7fea637fe700 1 --2- 172.21.15.67:0/2908857601 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fea640ccd10 0x7fea640b4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:48.490 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.462+0000 7fea637fe700 1 -- 172.21.15.67:0/2908857601 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea54009550 con 0x7fea640cc3f0 2023-10-14T10:36:48.491 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.462+0000 7fea637fe700 1 --2- 172.21.15.67:0/2908857601 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea640cc3f0 0x7fea640b30d0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fea54014040 tx=0x7fea54004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:48.491 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.463+0000 7fea4bfff700 1 -- 172.21.15.67:0/2908857601 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea54004520 con 0x7fea640cc3f0 2023-10-14T10:36:48.491 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.463+0000 7fea4bfff700 1 -- 172.21.15.67:0/2908857601 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea5401e070 con 0x7fea640cc3f0 2023-10-14T10:36:48.492 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.463+0000 7fea69c31700 1 -- 172.21.15.67:0/2908857601 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea640b3d50 con 0x7fea640cc3f0 2023-10-14T10:36:48.492 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.463+0000 7fea4bfff700 1 -- 172.21.15.67:0/2908857601 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea540038e0 con 0x7fea640cc3f0 2023-10-14T10:36:48.492 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.463+0000 7fea69c31700 1 -- 172.21.15.67:0/2908857601 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea640b4d70 con 0x7fea640cc3f0 2023-10-14T10:36:48.493 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.464+0000 7fea69c31700 1 -- 172.21.15.67:0/2908857601 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea64067d20 con 0x7fea640cc3f0 2023-10-14T10:36:48.493 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.464+0000 7fea4bfff700 1 -- 172.21.15.67:0/2908857601 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fea54003a40 con 0x7fea640cc3f0 2023-10-14T10:36:48.494 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.465+0000 7fea4bfff700 1 --2- 172.21.15.67:0/2908857601 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fea5006af30 0x7fea5006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:48.494 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.465+0000 7fea4bfff700 1 -- 172.21.15.67:0/2908857601 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fea54020070 con 0x7fea640cc3f0 2023-10-14T10:36:48.494 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.465+0000 7fea62ffd700 1 --2- 172.21.15.67:0/2908857601 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fea5006af30 0x7fea5006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:48.495 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.465+0000 7fea62ffd700 1 --2- 172.21.15.67:0/2908857601 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fea5006af30 0x7fea5006d3f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fea64153a00 tx=0x7fea58008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:48.495 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.469+0000 7fea4bfff700 1 -- 172.21.15.67:0/2908857601 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fea54055d00 con 0x7fea640cc3f0 2023-10-14T10:36:48.713 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.711+0000 7fea69c31700 1 -- 172.21.15.67:0/2908857601 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fea640b6430 con 0x7fea640cc3f0 2023-10-14T10:36:48.713 INFO:tasks.ceph.mon.a.smithi067.stderr:2023-10-14T10:36:48.711+0000 7fcd62d9b700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-14T10:36:48.714 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.713+0000 7fea4bfff700 1 -- 172.21.15.67:0/2908857601 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) v1 ==== 161+0+0 (secure 0 0 0) 0x7fea54015020 con 0x7fea640cc3f0 2023-10-14T10:36:48.718 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.716+0000 7fea69c31700 1 -- 172.21.15.67:0/2908857601 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fea5006af30 msgr2=0x7fea5006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:48.719 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.716+0000 7fea69c31700 1 --2- 172.21.15.67:0/2908857601 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fea5006af30 0x7fea5006d3f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fea64153a00 tx=0x7fea58008040 comp rx=0 tx=0).stop 2023-10-14T10:36:48.719 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.716+0000 7fea69c31700 1 -- 172.21.15.67:0/2908857601 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea640cc3f0 msgr2=0x7fea640b30d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:48.719 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.716+0000 7fea69c31700 1 --2- 172.21.15.67:0/2908857601 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea640cc3f0 0x7fea640b30d0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fea54014040 tx=0x7fea54004750 comp rx=0 tx=0).stop 2023-10-14T10:36:48.719 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.716+0000 7fea69c31700 1 -- 172.21.15.67:0/2908857601 shutdown_connections 2023-10-14T10:36:48.719 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.716+0000 7fea69c31700 1 --2- 172.21.15.67:0/2908857601 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fea5006af30 0x7fea5006d3f0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:48.720 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.716+0000 7fea69c31700 1 --2- 172.21.15.67:0/2908857601 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fea64161670 0x7fea6406dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:48.720 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.716+0000 7fea69c31700 1 --2- 172.21.15.67:0/2908857601 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea640cc3f0 0x7fea640b30d0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:48.720 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.716+0000 7fea69c31700 1 --2- 172.21.15.67:0/2908857601 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fea640ccd10 0x7fea640b4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:48.720 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.716+0000 7fea69c31700 1 -- 172.21.15.67:0/2908857601 >> 172.21.15.67:0/2908857601 conn(0x7fea6406dac0 msgr2=0x7fea640ae8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:48.720 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.716+0000 7fea69c31700 1 -- 172.21.15.67:0/2908857601 shutdown_connections 2023-10-14T10:36:48.721 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.716+0000 7fea69c31700 1 -- 172.21.15.67:0/2908857601 wait complete. 2023-10-14T10:36:48.721 INFO:teuthology.orchestra.run.smithi067.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-14T10:36:48.730 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-14T10:36:48.757 INFO:tasks.ceph.mds.b.smithi072.stdout:starting mds.b at 2023-10-14T10:36:48.858 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.856+0000 7fbf96546700 1 Processor -- start 2023-10-14T10:36:48.858 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.856+0000 7fbf96546700 1 -- start start 2023-10-14T10:36:48.861 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.856+0000 7fbf96546700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fbf9015b740 0x7fbf900d1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:48.861 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.856+0000 7fbf96546700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fbf900d0c70 0x7fbf900cdfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:48.862 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.856+0000 7fbf96546700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fbf900c5110 0x7fbf900ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:48.862 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.856+0000 7fbf96546700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf9015bb20 con 0x7fbf9015b740 2023-10-14T10:36:48.862 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.856+0000 7fbf96546700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf900cf190 con 0x7fbf900d0c70 2023-10-14T10:36:48.862 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.856+0000 7fbf96546700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf900cbd40 con 0x7fbf900c5110 2023-10-14T10:36:48.863 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.857+0000 7fbf8ffff700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fbf9015b740 0x7fbf900d1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:48.863 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.857+0000 7fbf8ffff700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fbf9015b740 0x7fbf900d1cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45444/0 (socket says 172.21.15.67:45444) 2023-10-14T10:36:48.863 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.857+0000 7fbf8ffff700 1 -- 172.21.15.67:0/1659402414 learned_addr learned my addr 172.21.15.67:0/1659402414 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:48.863 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.857+0000 7fbf94ae3700 1 --2- 172.21.15.67:0/1659402414 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fbf900c5110 0x7fbf900ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:48.863 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.857+0000 7fbf8f7fe700 1 --2- 172.21.15.67:0/1659402414 >> v2:172.21.15.72:3300/0 conn(0x7fbf900d0c70 0x7fbf900cdfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:48.864 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.857+0000 7fbf8ffff700 1 -- 172.21.15.67:0/1659402414 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fbf900c5110 msgr2=0x7fbf900ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:48.864 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.857+0000 7fbf8ffff700 1 --2- 172.21.15.67:0/1659402414 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fbf900c5110 0x7fbf900ce510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:48.864 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.857+0000 7fbf8ffff700 1 -- 172.21.15.67:0/1659402414 >> v2:172.21.15.72:3300/0 conn(0x7fbf900d0c70 msgr2=0x7fbf900cdfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:48.864 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.857+0000 7fbf8ffff700 1 --2- 172.21.15.67:0/1659402414 >> v2:172.21.15.72:3300/0 conn(0x7fbf900d0c70 0x7fbf900cdfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:48.864 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.857+0000 7fbf8ffff700 1 -- 172.21.15.67:0/1659402414 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf900cc020 con 0x7fbf9015b740 2023-10-14T10:36:48.865 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.857+0000 7fbf8f7fe700 1 --2- 172.21.15.67:0/1659402414 >> v2:172.21.15.72:3300/0 conn(0x7fbf900d0c70 0x7fbf900cdfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-14T10:36:48.865 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.857+0000 7fbf8ffff700 1 --2- 172.21.15.67:0/1659402414 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbf9015b740 0x7fbf900d1cd0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fbf900cec30 tx=0x7fbf80009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0946c5cde5e60bc server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:48.865 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.857+0000 7fbf8effd700 1 -- 172.21.15.67:0/1659402414 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf80011830 con 0x7fbf9015b740 2023-10-14T10:36:48.865 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.857+0000 7fbf8effd700 1 -- 172.21.15.67:0/1659402414 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf80011990 con 0x7fbf9015b740 2023-10-14T10:36:48.866 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.857+0000 7fbf8effd700 1 -- 172.21.15.67:0/1659402414 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf80011d20 con 0x7fbf9015b740 2023-10-14T10:36:48.866 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.858+0000 7fbf96546700 1 -- 172.21.15.67:0/1659402414 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbf9015b740 msgr2=0x7fbf900d1cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:48.866 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.858+0000 7fbf96546700 1 --2- 172.21.15.67:0/1659402414 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbf9015b740 0x7fbf900d1cd0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fbf900cec30 tx=0x7fbf80009bd0 comp rx=0 tx=0).stop 2023-10-14T10:36:48.866 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.858+0000 7fbf96546700 1 -- 172.21.15.67:0/1659402414 shutdown_connections 2023-10-14T10:36:48.866 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.858+0000 7fbf96546700 1 --2- 172.21.15.67:0/1659402414 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fbf900c5110 0x7fbf900ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:48.867 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.858+0000 7fbf96546700 1 --2- 172.21.15.67:0/1659402414 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbf9015b740 0x7fbf900d1cd0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:48.867 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.858+0000 7fbf96546700 1 --2- 172.21.15.67:0/1659402414 >> v2:172.21.15.72:3300/0 conn(0x7fbf900d0c70 0x7fbf900cdfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:48.867 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.858+0000 7fbf96546700 1 -- 172.21.15.67:0/1659402414 >> 172.21.15.67:0/1659402414 conn(0x7fbf900665a0 msgr2=0x7fbf90066980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:48.867 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.858+0000 7fbf96546700 1 -- 172.21.15.67:0/1659402414 shutdown_connections 2023-10-14T10:36:48.868 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.858+0000 7fbf96546700 1 -- 172.21.15.67:0/1659402414 wait complete. 2023-10-14T10:36:48.868 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.859+0000 7fbf96546700 1 Processor -- start 2023-10-14T10:36:48.868 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.859+0000 7fbf96546700 1 -- start start 2023-10-14T10:36:48.871 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.860+0000 7fbf96546700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbf900c5110 0x7fbf9014b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:48.872 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.860+0000 7fbf96546700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbf900d0c70 0x7fbf900b5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:48.872 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.860+0000 7fbf96546700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fbf9015b740 0x7fbf900b6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:48.872 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.860+0000 7fbf96546700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf900b5000 con 0x7fbf900c5110 2023-10-14T10:36:48.872 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.860+0000 7fbf96546700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf900c8c00 con 0x7fbf900d0c70 2023-10-14T10:36:48.873 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.860+0000 7fbf96546700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf900b5250 con 0x7fbf9015b740 2023-10-14T10:36:48.873 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.860+0000 7fbf8ffff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbf900c5110 0x7fbf9014b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:48.873 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.860+0000 7fbf8ffff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbf900c5110 0x7fbf9014b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45450/0 (socket says 172.21.15.67:45450) 2023-10-14T10:36:48.873 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.860+0000 7fbf8ffff700 1 -- 172.21.15.67:0/269794162 learned_addr learned my addr 172.21.15.67:0/269794162 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:48.873 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.860+0000 7fbf94ae3700 1 --2- 172.21.15.67:0/269794162 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fbf9015b740 0x7fbf900b6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:48.874 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.860+0000 7fbf8f7fe700 1 --2- 172.21.15.67:0/269794162 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbf900d0c70 0x7fbf900b5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:48.874 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.860+0000 7fbf8ffff700 1 -- 172.21.15.67:0/269794162 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fbf9015b740 msgr2=0x7fbf900b6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:48.874 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.860+0000 7fbf8ffff700 1 --2- 172.21.15.67:0/269794162 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fbf9015b740 0x7fbf900b6270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:48.874 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.860+0000 7fbf8ffff700 1 -- 172.21.15.67:0/269794162 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbf900d0c70 msgr2=0x7fbf900b5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:48.875 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.860+0000 7fbf8ffff700 1 --2- 172.21.15.67:0/269794162 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbf900d0c70 0x7fbf900b5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:48.875 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.861+0000 7fbf8ffff700 1 -- 172.21.15.67:0/269794162 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf80009550 con 0x7fbf900c5110 2023-10-14T10:36:48.875 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.861+0000 7fbf8f7fe700 1 --2- 172.21.15.67:0/269794162 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbf900d0c70 0x7fbf900b5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-14T10:36:48.875 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.861+0000 7fbf8ffff700 1 --2- 172.21.15.67:0/269794162 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbf900c5110 0x7fbf9014b110 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fbf80009c10 tx=0x7fbf80004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:48.876 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.861+0000 7fbf8cff9700 1 -- 172.21.15.67:0/269794162 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf80004510 con 0x7fbf900c5110 2023-10-14T10:36:48.876 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.861+0000 7fbf8cff9700 1 -- 172.21.15.67:0/269794162 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf8001e070 con 0x7fbf900c5110 2023-10-14T10:36:48.876 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.861+0000 7fbf8cff9700 1 -- 172.21.15.67:0/269794162 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf800038e0 con 0x7fbf900c5110 2023-10-14T10:36:48.876 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.862+0000 7fbf96546700 1 -- 172.21.15.67:0/269794162 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf900b5530 con 0x7fbf900c5110 2023-10-14T10:36:48.877 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.862+0000 7fbf96546700 1 -- 172.21.15.67:0/269794162 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf90159e90 con 0x7fbf900c5110 2023-10-14T10:36:48.877 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.862+0000 7fbf96546700 1 -- 172.21.15.67:0/269794162 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf900b5a50 con 0x7fbf900c5110 2023-10-14T10:36:48.877 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.863+0000 7fbf8cff9700 1 -- 172.21.15.67:0/269794162 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fbf80003a40 con 0x7fbf900c5110 2023-10-14T10:36:48.877 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.863+0000 7fbf8cff9700 1 --2- 172.21.15.67:0/269794162 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fbf7c06af00 0x7fbf7c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:48.878 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.863+0000 7fbf8cff9700 1 -- 172.21.15.67:0/269794162 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fbf8008a4b0 con 0x7fbf900c5110 2023-10-14T10:36:48.878 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.864+0000 7fbf8f7fe700 1 --2- 172.21.15.67:0/269794162 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fbf7c06af00 0x7fbf7c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:48.878 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.864+0000 7fbf8f7fe700 1 --2- 172.21.15.67:0/269794162 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fbf7c06af00 0x7fbf7c06d3c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fbf840099f0 tx=0x7fbf84008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:48.878 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:48.868+0000 7fbf8cff9700 1 -- 172.21.15.67:0/269794162 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbf900b5a50 con 0x7fbf900c5110 2023-10-14T10:36:49.116 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:49.114+0000 7fbf96546700 1 -- 172.21.15.67:0/269794162 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fbf900b5a50 con 0x7fbf900c5110 2023-10-14T10:36:49.716 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:49.714+0000 7fbf8cff9700 1 -- 172.21.15.67:0/269794162 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) v1 ==== 104+0+0 (secure 0 0 0) 0x7fbf900b5a50 con 0x7fbf900c5110 2023-10-14T10:36:49.719 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:49.717+0000 7fbf96546700 1 -- 172.21.15.67:0/269794162 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fbf7c06af00 msgr2=0x7fbf7c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:49.719 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:49.717+0000 7fbf96546700 1 --2- 172.21.15.67:0/269794162 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fbf7c06af00 0x7fbf7c06d3c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fbf840099f0 tx=0x7fbf84008040 comp rx=0 tx=0).stop 2023-10-14T10:36:49.720 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:49.717+0000 7fbf96546700 1 -- 172.21.15.67:0/269794162 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbf900c5110 msgr2=0x7fbf9014b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:49.720 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:49.717+0000 7fbf96546700 1 --2- 172.21.15.67:0/269794162 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbf900c5110 0x7fbf9014b110 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fbf80009c10 tx=0x7fbf80004750 comp rx=0 tx=0).stop 2023-10-14T10:36:49.720 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:49.717+0000 7fbf96546700 1 -- 172.21.15.67:0/269794162 shutdown_connections 2023-10-14T10:36:49.720 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:49.717+0000 7fbf96546700 1 --2- 172.21.15.67:0/269794162 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fbf7c06af00 0x7fbf7c06d3c0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:49.720 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:49.717+0000 7fbf96546700 1 --2- 172.21.15.67:0/269794162 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fbf9015b740 0x7fbf900b6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:49.721 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:49.717+0000 7fbf96546700 1 --2- 172.21.15.67:0/269794162 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbf900c5110 0x7fbf9014b110 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:49.721 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:49.717+0000 7fbf96546700 1 --2- 172.21.15.67:0/269794162 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbf900d0c70 0x7fbf900b5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:49.721 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:49.717+0000 7fbf96546700 1 -- 172.21.15.67:0/269794162 >> 172.21.15.67:0/269794162 conn(0x7fbf900665a0 msgr2=0x7fbf900c63c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:49.721 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:49.717+0000 7fbf96546700 1 -- 172.21.15.67:0/269794162 shutdown_connections 2023-10-14T10:36:49.722 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:49.717+0000 7fbf96546700 1 -- 172.21.15.67:0/269794162 wait complete. 2023-10-14T10:36:49.733 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-14T10:36:49.868 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:49.865+0000 7ff2fa1eb700 1 Processor -- start 2023-10-14T10:36:49.868 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:49.866+0000 7ff2fa1eb700 1 -- start start 2023-10-14T10:36:49.870 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:49.866+0000 7ff2fa1eb700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7ff2f40b0730 0x7ff2f40d04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:49.871 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:49.866+0000 7ff2fa1eb700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff2f40cc240 0x7ff2f40cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:49.871 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:49.866+0000 7ff2fa1eb700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2f40cbdb0 con 0x7ff2f40cdf10 2023-10-14T10:36:49.871 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:49.866+0000 7ff2fa1eb700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2f40d0b40 con 0x7ff2f40cc240 2023-10-14T10:36:49.871 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:49.866+0000 7ff2fa1eb700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2f40d0e20 con 0x7ff2f40b0730 2023-10-14T10:36:49.871 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:49.867+0000 7ff2f3fff700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff2f40cc240 0x7ff2f40cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:49.872 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:49.867+0000 7ff2f37fe700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7ff2f40cdf10 0x7ff2f40ce2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:53226/0 (socket says 172.21.15.67:53226) 2023-10-14T10:36:49.872 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:49.867+0000 7ff2f37fe700 1 -- 172.21.15.67:0/3309249362 learned_addr learned my addr 172.21.15.67:0/3309249362 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:36:49.872 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:49.867+0000 7ff2f2ffd700 1 --2- 172.21.15.67:0/3309249362 >> v2:172.21.15.72:3300/0 conn(0x7ff2f40b0730 0x7ff2f40d04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:49.872 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:49.867+0000 7ff2f27fc700 1 -- 172.21.15.67:0/3309249362 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2866184576 0 0) 0x7ff2f40cbdb0 con 0x7ff2f40cdf10 2023-10-14T10:36:49.872 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:49.867+0000 7ff2f27fc700 1 -- 172.21.15.67:0/3309249362 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2e00035b0 con 0x7ff2f40cdf10 2023-10-14T10:36:49.873 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:49.867+0000 7ff2f3fff700 1 -- 172.21.15.67:0/3309249362 >> v2:172.21.15.72:3300/0 conn(0x7ff2f40b0730 msgr2=0x7ff2f40d04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:49.873 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:49.867+0000 7ff2f3fff700 1 --2- 172.21.15.67:0/3309249362 >> v2:172.21.15.72:3300/0 conn(0x7ff2f40b0730 0x7ff2f40d04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:49.873 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:49.867+0000 7ff2f3fff700 1 -- 172.21.15.67:0/3309249362 >> v1:172.21.15.67:6789/0 conn(0x7ff2f40cdf10 legacy=0x7ff2f40ce2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:49.873 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:49.867+0000 7ff2f3fff700 1 -- 172.21.15.67:0/3309249362 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2f40d1100 con 0x7ff2f40cc240 2023-10-14T10:36:49.873 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:49.867+0000 7ff2f3fff700 1 --2- 172.21.15.67:0/3309249362 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2f40cc240 0x7ff2f40cc6d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4067c40 tx=0x7ff2ec00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80d6df5a3269edb7 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:49.874 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:49.867+0000 7ff2f27fc700 1 -- 172.21.15.67:0/3309249362 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2ec00e070 con 0x7ff2f40cc240 2023-10-14T10:36:49.874 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:49.867+0000 7ff2f27fc700 1 -- 172.21.15.67:0/3309249362 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2ec00bb20 con 0x7ff2f40cc240 2023-10-14T10:36:49.874 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:49.867+0000 7ff2f27fc700 1 -- 172.21.15.67:0/3309249362 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2ec012430 con 0x7ff2f40cc240 2023-10-14T10:36:49.874 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:49.868+0000 7ff2fa1eb700 1 -- 172.21.15.67:0/3309249362 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2f40cc240 msgr2=0x7ff2f40cc6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:49.874 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:49.868+0000 7ff2fa1eb700 1 --2- 172.21.15.67:0/3309249362 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2f40cc240 0x7ff2f40cc6d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4067c40 tx=0x7ff2ec00b330 comp rx=0 tx=0).stop 2023-10-14T10:36:49.875 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:49.868+0000 7ff2fa1eb700 1 -- 172.21.15.67:0/3309249362 shutdown_connections 2023-10-14T10:36:49.875 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:49.868+0000 7ff2fa1eb700 1 --2- 172.21.15.67:0/3309249362 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2f40cc240 0x7ff2f40cc6d0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:49.875 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:49.868+0000 7ff2fa1eb700 1 --2- 172.21.15.67:0/3309249362 >> v2:172.21.15.72:3300/0 conn(0x7ff2f40b0730 0x7ff2f40d04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:49.875 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:49.868+0000 7ff2fa1eb700 1 -- 172.21.15.67:0/3309249362 >> 172.21.15.67:0/3309249362 conn(0x7ff2f40d1530 msgr2=0x7ff2f40d1940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:49.875 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:49.868+0000 7ff2fa1eb700 1 -- 172.21.15.67:0/3309249362 shutdown_connections 2023-10-14T10:36:49.876 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:49.868+0000 7ff2fa1eb700 1 -- 172.21.15.67:0/3309249362 wait complete. 2023-10-14T10:36:49.878 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:49.869+0000 7ff2fa1eb700 1 Processor -- start 2023-10-14T10:36:49.878 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:49.869+0000 7ff2fa1eb700 1 -- start start 2023-10-14T10:36:49.879 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:49.870+0000 7ff2fa1eb700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7ff2f40b0730 0x7ff2f40b4a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:49.879 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:49.870+0000 7ff2fa1eb700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff2f40cc240 0x7ff2f415c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:49.879 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:49.870+0000 7ff2fa1eb700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2f40cdf10 0x7ff2f406be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:49.879 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:49.870+0000 7ff2fa1eb700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f415c270 con 0x7ff2f40cdf10 2023-10-14T10:36:49.879 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:49.870+0000 7ff2fa1eb700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f415b5f0 con 0x7ff2f40cc240 2023-10-14T10:36:49.880 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:49.870+0000 7ff2fa1eb700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f415b8d0 con 0x7ff2f40b0730 2023-10-14T10:36:49.880 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:49.870+0000 7ff2f3fff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2f40cdf10 0x7ff2f406be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:49.880 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:49.870+0000 7ff2f3fff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2f40cdf10 0x7ff2f406be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45468/0 (socket says 172.21.15.67:45468) 2023-10-14T10:36:49.880 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:49.870+0000 7ff2f3fff700 1 -- 172.21.15.67:0/2428593950 learned_addr learned my addr 172.21.15.67:0/2428593950 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:49.880 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:49.870+0000 7ff2f2ffd700 1 --2- 172.21.15.67:0/2428593950 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff2f40cc240 0x7ff2f415c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:49.881 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:49.870+0000 7ff2f37fe700 1 --2- 172.21.15.67:0/2428593950 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7ff2f40b0730 0x7ff2f40b4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:49.881 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:49.870+0000 7ff2f3fff700 1 -- 172.21.15.67:0/2428593950 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7ff2f40b0730 msgr2=0x7ff2f40b4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:49.881 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:49.870+0000 7ff2f3fff700 1 --2- 172.21.15.67:0/2428593950 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7ff2f40b0730 0x7ff2f40b4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:49.881 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:49.870+0000 7ff2f3fff700 1 -- 172.21.15.67:0/2428593950 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff2f40cc240 msgr2=0x7ff2f415c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:49.881 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:49.870+0000 7ff2f3fff700 1 --2- 172.21.15.67:0/2428593950 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff2f40cc240 0x7ff2f415c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:49.882 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:49.870+0000 7ff2f3fff700 1 -- 172.21.15.67:0/2428593950 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2ec009a70 con 0x7ff2f40cdf10 2023-10-14T10:36:49.882 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:49.870+0000 7ff2f2ffd700 1 --2- 172.21.15.67:0/2428593950 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff2f40cc240 0x7ff2f415c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-14T10:36:49.882 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:49.870+0000 7ff2f37fe700 1 --2- 172.21.15.67:0/2428593950 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7ff2f40b0730 0x7ff2f40b4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-14T10:36:49.882 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:49.871+0000 7ff2f3fff700 1 --2- 172.21.15.67:0/2428593950 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2f40cdf10 0x7ff2f406be10 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec00b370 tx=0x7ff2ec004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:49.882 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:49.871+0000 7ff2ebfff700 1 -- 172.21.15.67:0/2428593950 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2ec012430 con 0x7ff2f40cdf10 2023-10-14T10:36:49.883 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:49.871+0000 7ff2ebfff700 1 -- 172.21.15.67:0/2428593950 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2ec020070 con 0x7ff2f40cdf10 2023-10-14T10:36:49.883 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:49.871+0000 7ff2fa1eb700 1 -- 172.21.15.67:0/2428593950 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2f415bbb0 con 0x7ff2f40cdf10 2023-10-14T10:36:49.883 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:49.871+0000 7ff2ebfff700 1 -- 172.21.15.67:0/2428593950 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2ec004280 con 0x7ff2f40cdf10 2023-10-14T10:36:49.883 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:49.871+0000 7ff2fa1eb700 1 -- 172.21.15.67:0/2428593950 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2f415d1a0 con 0x7ff2f40cdf10 2023-10-14T10:36:49.883 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:49.872+0000 7ff2fa1eb700 1 -- 172.21.15.67:0/2428593950 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2f415c3b0 con 0x7ff2f40cdf10 2023-10-14T10:36:49.884 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:49.872+0000 7ff2ebfff700 1 -- 172.21.15.67:0/2428593950 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff2ec0043e0 con 0x7ff2f40cdf10 2023-10-14T10:36:49.884 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:49.873+0000 7ff2ebfff700 1 --2- 172.21.15.67:0/2428593950 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7ff2d406af00 0x7ff2d406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:49.884 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:49.873+0000 7ff2ebfff700 1 -- 172.21.15.67:0/2428593950 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5795+0+0 (secure 0 0 0) 0x7ff2ec022070 con 0x7ff2f40cdf10 2023-10-14T10:36:49.884 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:49.873+0000 7ff2f37fe700 1 --2- 172.21.15.67:0/2428593950 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7ff2d406af00 0x7ff2d406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:49.884 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:49.873+0000 7ff2f37fe700 1 --2- 172.21.15.67:0/2428593950 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7ff2d406af00 0x7ff2d406d3c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc0010e0 tx=0x7ff2dc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:49.885 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:49.877+0000 7ff2ebfff700 1 -- 172.21.15.67:0/2428593950 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff2f415c3b0 con 0x7ff2f40cdf10 2023-10-14T10:36:50.082 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:50.080+0000 7ff2fa1eb700 1 -- 172.21.15.67:0/2428593950 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff2f415c3b0 con 0x7ff2f40cdf10 2023-10-14T10:36:51.016 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:51.014+0000 7ff2ebfff700 1 -- 172.21.15.67:0/2428593950 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v24) v1 ==== 181+0+0 (secure 0 0 0) 0x7ff2f415c3b0 con 0x7ff2f40cdf10 2023-10-14T10:36:51.021 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:51.018+0000 7ff2fa1eb700 1 -- 172.21.15.67:0/2428593950 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7ff2d406af00 msgr2=0x7ff2d406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:51.021 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:51.018+0000 7ff2fa1eb700 1 --2- 172.21.15.67:0/2428593950 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7ff2d406af00 0x7ff2d406d3c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc0010e0 tx=0x7ff2dc009040 comp rx=0 tx=0).stop 2023-10-14T10:36:51.021 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:51.018+0000 7ff2fa1eb700 1 -- 172.21.15.67:0/2428593950 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2f40cdf10 msgr2=0x7ff2f406be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:51.021 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:51.018+0000 7ff2fa1eb700 1 --2- 172.21.15.67:0/2428593950 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2f40cdf10 0x7ff2f406be10 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec00b370 tx=0x7ff2ec004ad0 comp rx=0 tx=0).stop 2023-10-14T10:36:51.021 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:51.018+0000 7ff2fa1eb700 1 -- 172.21.15.67:0/2428593950 shutdown_connections 2023-10-14T10:36:51.022 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:51.018+0000 7ff2fa1eb700 1 --2- 172.21.15.67:0/2428593950 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7ff2d406af00 0x7ff2d406d3c0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:51.022 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:51.018+0000 7ff2fa1eb700 1 --2- 172.21.15.67:0/2428593950 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2f40cdf10 0x7ff2f406be10 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:51.022 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:51.018+0000 7ff2fa1eb700 1 --2- 172.21.15.67:0/2428593950 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7ff2f40b0730 0x7ff2f40b4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:51.022 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:51.018+0000 7ff2fa1eb700 1 --2- 172.21.15.67:0/2428593950 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff2f40cc240 0x7ff2f415c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:51.022 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:51.018+0000 7ff2fa1eb700 1 -- 172.21.15.67:0/2428593950 >> 172.21.15.67:0/2428593950 conn(0x7ff2f40d1530 msgr2=0x7ff2f415a970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:51.023 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:51.018+0000 7ff2fa1eb700 1 -- 172.21.15.67:0/2428593950 shutdown_connections 2023-10-14T10:36:51.023 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:51.018+0000 7ff2fa1eb700 1 -- 172.21.15.67:0/2428593950 wait complete. 2023-10-14T10:36:51.023 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs_metadata' removed 2023-10-14T10:36:51.036 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-14T10:36:51.173 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:51.171+0000 7fcd39574700 1 Processor -- start 2023-10-14T10:36:51.173 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:51.171+0000 7fcd39574700 1 -- start start 2023-10-14T10:36:51.175 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:51.171+0000 7fcd39574700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fcd340d1760 0x7fcd340b4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:51.175 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:51.171+0000 7fcd39574700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fcd340d1b40 0x7fcd340d1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:51.176 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:51.171+0000 7fcd39574700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd340b6190 con 0x7fcd3415d940 2023-10-14T10:36:51.176 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:51.171+0000 7fcd39574700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd340d2500 con 0x7fcd340d1760 2023-10-14T10:36:51.176 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:51.171+0000 7fcd39574700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd340cf6c0 con 0x7fcd340d1b40 2023-10-14T10:36:51.176 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:51.172+0000 7fcd32ffd700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fcd340d1760 0x7fcd340b4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:51.176 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:51.172+0000 7fcd337fe700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fcd3415d940 0x7fcd3415ddd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:53240/0 (socket says 172.21.15.67:53240) 2023-10-14T10:36:51.177 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:51.172+0000 7fcd337fe700 1 -- 172.21.15.67:0/3579586006 learned_addr learned my addr 172.21.15.67:0/3579586006 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:36:51.177 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:51.172+0000 7fcd327fc700 1 --2- 172.21.15.67:0/3579586006 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fcd340d1b40 0x7fcd340d1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:51.177 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:51.172+0000 7fcd32ffd700 1 -- 172.21.15.67:0/3579586006 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fcd340d1b40 msgr2=0x7fcd340d1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:51.177 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:51.172+0000 7fcd32ffd700 1 --2- 172.21.15.67:0/3579586006 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fcd340d1b40 0x7fcd340d1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:51.177 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:51.172+0000 7fcd32ffd700 1 -- 172.21.15.67:0/3579586006 >> v1:172.21.15.67:6789/0 conn(0x7fcd3415d940 legacy=0x7fcd3415ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:51.178 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:51.172+0000 7fcd32ffd700 1 -- 172.21.15.67:0/3579586006 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd340b58f0 con 0x7fcd340d1760 2023-10-14T10:36:51.178 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:51.172+0000 7fcd32ffd700 1 --2- 172.21.15.67:0/3579586006 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd340d1760 0x7fcd340b4890 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c009820 tx=0x7fcd1c009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a123fdf97f60c91a server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:51.178 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:51.172+0000 7fcd31ffb700 1 -- 172.21.15.67:0/3579586006 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd1c005e30 con 0x7fcd340d1760 2023-10-14T10:36:51.178 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:51.172+0000 7fcd31ffb700 1 -- 172.21.15.67:0/3579586006 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd1c006030 con 0x7fcd340d1760 2023-10-14T10:36:51.178 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:51.172+0000 7fcd31ffb700 1 -- 172.21.15.67:0/3579586006 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd1c0063e0 con 0x7fcd340d1760 2023-10-14T10:36:51.179 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:51.173+0000 7fcd39574700 1 -- 172.21.15.67:0/3579586006 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd340d1760 msgr2=0x7fcd340b4890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:51.179 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:51.173+0000 7fcd39574700 1 --2- 172.21.15.67:0/3579586006 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd340d1760 0x7fcd340b4890 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c009820 tx=0x7fcd1c009b30 comp rx=0 tx=0).stop 2023-10-14T10:36:51.179 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:51.173+0000 7fcd39574700 1 -- 172.21.15.67:0/3579586006 shutdown_connections 2023-10-14T10:36:51.179 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:51.173+0000 7fcd39574700 1 --2- 172.21.15.67:0/3579586006 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd340d1760 0x7fcd340b4890 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:51.179 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:51.173+0000 7fcd39574700 1 --2- 172.21.15.67:0/3579586006 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fcd340d1b40 0x7fcd340d1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:51.180 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:51.173+0000 7fcd39574700 1 -- 172.21.15.67:0/3579586006 >> 172.21.15.67:0/3579586006 conn(0x7fcd340d37a0 msgr2=0x7fcd340d3bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:51.180 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:51.173+0000 7fcd39574700 1 -- 172.21.15.67:0/3579586006 shutdown_connections 2023-10-14T10:36:51.180 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:51.173+0000 7fcd39574700 1 -- 172.21.15.67:0/3579586006 wait complete. 2023-10-14T10:36:51.183 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:51.174+0000 7fcd39574700 1 Processor -- start 2023-10-14T10:36:51.183 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:51.174+0000 7fcd39574700 1 -- start start 2023-10-14T10:36:51.183 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:51.175+0000 7fcd39574700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcd340d1760 0x7fcd340b4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:51.183 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:51.175+0000 7fcd39574700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd340d1b40 0x7fcd340b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:51.184 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:51.175+0000 7fcd39574700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fcd3415d940 0x7fcd340ca900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:51.184 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:51.175+0000 7fcd39574700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd340c81d0 con 0x7fcd340d1b40 2023-10-14T10:36:51.184 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:51.175+0000 7fcd39574700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd341e6a20 con 0x7fcd340d1760 2023-10-14T10:36:51.184 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:51.175+0000 7fcd39574700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd340c83b0 con 0x7fcd3415d940 2023-10-14T10:36:51.184 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:51.175+0000 7fcd327fc700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd340d1b40 0x7fcd340b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:51.185 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:51.175+0000 7fcd327fc700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd340d1b40 0x7fcd340b4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45478/0 (socket says 172.21.15.67:45478) 2023-10-14T10:36:51.185 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:51.175+0000 7fcd327fc700 1 -- 172.21.15.67:0/1083130698 learned_addr learned my addr 172.21.15.67:0/1083130698 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:51.185 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:51.175+0000 7fcd337fe700 1 --2- 172.21.15.67:0/1083130698 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fcd3415d940 0x7fcd340ca900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:51.185 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:51.175+0000 7fcd32ffd700 1 --2- 172.21.15.67:0/1083130698 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcd340d1760 0x7fcd340b4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:51.185 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:51.175+0000 7fcd327fc700 1 -- 172.21.15.67:0/1083130698 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fcd3415d940 msgr2=0x7fcd340ca900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:51.185 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:51.175+0000 7fcd327fc700 1 --2- 172.21.15.67:0/1083130698 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fcd3415d940 0x7fcd340ca900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:51.186 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:51.175+0000 7fcd327fc700 1 -- 172.21.15.67:0/1083130698 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcd340d1760 msgr2=0x7fcd340b4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:51.186 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:51.175+0000 7fcd327fc700 1 --2- 172.21.15.67:0/1083130698 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcd340d1760 0x7fcd340b4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:51.186 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:51.175+0000 7fcd327fc700 1 -- 172.21.15.67:0/1083130698 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd1c009430 con 0x7fcd340d1b40 2023-10-14T10:36:51.186 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:51.175+0000 7fcd32ffd700 1 --2- 172.21.15.67:0/1083130698 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcd340d1760 0x7fcd340b4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-14T10:36:51.186 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:51.175+0000 7fcd337fe700 1 --2- 172.21.15.67:0/1083130698 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fcd3415d940 0x7fcd340ca900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-14T10:36:51.187 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:51.175+0000 7fcd327fc700 1 --2- 172.21.15.67:0/1083130698 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd340d1b40 0x7fcd340b4d50 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fcd2400e810 tx=0x7fcd2400ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:51.187 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:51.176+0000 7fcd2b7fe700 1 -- 172.21.15.67:0/1083130698 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd24013610 con 0x7fcd340d1b40 2023-10-14T10:36:51.187 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:51.176+0000 7fcd39574700 1 -- 172.21.15.67:0/1083130698 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd340c8690 con 0x7fcd340d1b40 2023-10-14T10:36:51.187 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:51.176+0000 7fcd2b7fe700 1 -- 172.21.15.67:0/1083130698 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd24013770 con 0x7fcd340d1b40 2023-10-14T10:36:51.187 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:51.176+0000 7fcd2b7fe700 1 -- 172.21.15.67:0/1083130698 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd24013b20 con 0x7fcd340d1b40 2023-10-14T10:36:51.188 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:51.176+0000 7fcd39574700 1 -- 172.21.15.67:0/1083130698 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd3415ce50 con 0x7fcd340d1b40 2023-10-14T10:36:51.188 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:51.177+0000 7fcd39574700 1 -- 172.21.15.67:0/1083130698 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd3404ebb0 con 0x7fcd340d1b40 2023-10-14T10:36:51.188 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:51.177+0000 7fcd2b7fe700 1 -- 172.21.15.67:0/1083130698 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fcd24013c80 con 0x7fcd340d1b40 2023-10-14T10:36:51.188 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:51.177+0000 7fcd2b7fe700 1 --2- 172.21.15.67:0/1083130698 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fcd1406af00 0x7fcd1406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:51.188 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:51.178+0000 7fcd2b7fe700 1 -- 172.21.15.67:0/1083130698 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5357+0+0 (secure 0 0 0) 0x7fcd240891a0 con 0x7fcd340d1b40 2023-10-14T10:36:51.189 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:51.178+0000 7fcd32ffd700 1 --2- 172.21.15.67:0/1083130698 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fcd1406af00 0x7fcd1406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:51.189 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:51.178+0000 7fcd32ffd700 1 --2- 172.21.15.67:0/1083130698 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fcd1406af00 0x7fcd1406d3c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c013040 tx=0x7fcd1c008070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:51.189 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:51.182+0000 7fcd2b7fe700 1 -- 172.21.15.67:0/1083130698 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcd240543c0 con 0x7fcd340d1b40 2023-10-14T10:36:51.382 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:51.380+0000 7fcd39574700 1 -- 172.21.15.67:0/1083130698 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fcd340c7750 con 0x7fcd340d1b40 2023-10-14T10:36:52.020 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:52.018+0000 7fcd2b7fe700 1 -- 172.21.15.67:0/1083130698 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v25) v1 ==== 169+0+0 (secure 0 0 0) 0x7fcd240579e0 con 0x7fcd340d1b40 2023-10-14T10:36:52.024 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:52.022+0000 7fcd39574700 1 -- 172.21.15.67:0/1083130698 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fcd1406af00 msgr2=0x7fcd1406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:52.024 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:52.022+0000 7fcd39574700 1 --2- 172.21.15.67:0/1083130698 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fcd1406af00 0x7fcd1406d3c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c013040 tx=0x7fcd1c008070 comp rx=0 tx=0).stop 2023-10-14T10:36:52.024 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:52.022+0000 7fcd39574700 1 -- 172.21.15.67:0/1083130698 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd340d1b40 msgr2=0x7fcd340b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:52.024 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:52.022+0000 7fcd39574700 1 --2- 172.21.15.67:0/1083130698 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd340d1b40 0x7fcd340b4d50 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fcd2400e810 tx=0x7fcd2400ebd0 comp rx=0 tx=0).stop 2023-10-14T10:36:52.025 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:52.022+0000 7fcd39574700 1 -- 172.21.15.67:0/1083130698 shutdown_connections 2023-10-14T10:36:52.025 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:52.022+0000 7fcd39574700 1 --2- 172.21.15.67:0/1083130698 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fcd1406af00 0x7fcd1406d3c0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:52.025 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:52.022+0000 7fcd39574700 1 --2- 172.21.15.67:0/1083130698 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fcd3415d940 0x7fcd340ca900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:52.025 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:52.022+0000 7fcd39574700 1 --2- 172.21.15.67:0/1083130698 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcd340d1760 0x7fcd340b4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:52.025 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:52.022+0000 7fcd39574700 1 --2- 172.21.15.67:0/1083130698 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd340d1b40 0x7fcd340b4d50 secure :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fcd2400e810 tx=0x7fcd2400ebd0 comp rx=0 tx=0).stop 2023-10-14T10:36:52.026 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:52.022+0000 7fcd39574700 1 -- 172.21.15.67:0/1083130698 >> 172.21.15.67:0/1083130698 conn(0x7fcd340d37a0 msgr2=0x7fcd3406b890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:52.026 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:52.023+0000 7fcd39574700 1 -- 172.21.15.67:0/1083130698 shutdown_connections 2023-10-14T10:36:52.027 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:52.023+0000 7fcd39574700 1 -- 172.21.15.67:0/1083130698 wait complete. 2023-10-14T10:36:52.029 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs_data' removed 2023-10-14T10:36:52.041 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-14T10:36:52.041 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-14T10:36:52.042 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-14T10:36:52.042 INFO:teuthology.orchestra.run:waiting for 300 2023-10-14T10:36:52.042 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-14T10:36:52.042 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-14T10:36:52.043 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-14T10:36:52.043 INFO:teuthology.orchestra.run:waiting for 300 2023-10-14T10:36:52.043 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-14T10:36:52.044 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-14T10:36:52.044 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-14T10:36:52.044 INFO:teuthology.orchestra.run:waiting for 300 2023-10-14T10:36:52.044 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-14T10:36:52.045 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-14T10:36:52.045 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-14T10:36:52.047 INFO:teuthology.orchestra.run:waiting for 300 2023-10-14T10:36:52.053 INFO:tasks.ceph.mds.a:Stopped 2023-10-14T10:36:52.054 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-14T10:36:52.055 INFO:tasks.ceph.mds.c:Stopped 2023-10-14T10:36:52.055 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-14T10:36:52.057 INFO:tasks.ceph.mds.d:Stopped 2023-10-14T10:36:52.057 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-14T10:36:52.058 INFO:tasks.ceph.mds.b:Stopped 2023-10-14T10:36:52.059 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-14T10:36:52.094 INFO:tasks.ceph.mds.a:Started 2023-10-14T10:36:52.095 DEBUG:teuthology.parallel:result is None 2023-10-14T10:36:52.096 INFO:tasks.ceph.mds.c:Started 2023-10-14T10:36:52.096 DEBUG:teuthology.parallel:result is None 2023-10-14T10:36:52.099 INFO:tasks.ceph.mds.d:Started 2023-10-14T10:36:52.099 DEBUG:teuthology.parallel:result is None 2023-10-14T10:36:52.100 INFO:tasks.ceph.mds.b:Started 2023-10-14T10:36:52.100 DEBUG:teuthology.parallel:result is None 2023-10-14T10:36:52.101 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-14T10:36:52.180 INFO:tasks.ceph.mds.a.smithi067.stdout:starting mds.a at 2023-10-14T10:36:52.180 INFO:tasks.ceph.mds.c.smithi067.stdout:starting mds.c at 2023-10-14T10:36:52.185 INFO:tasks.ceph.mds.d.smithi072.stdout:starting mds.d at 2023-10-14T10:36:52.186 INFO:tasks.ceph.mds.b.smithi072.stdout:starting mds.b at 2023-10-14T10:36:52.248 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:52.234+0000 7f6f4212e700 1 Processor -- start 2023-10-14T10:36:52.249 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:52.234+0000 7f6f4212e700 1 -- start start 2023-10-14T10:36:52.249 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:52.235+0000 7f6f4212e700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f6f3c15ddc0 0x7f6f3c161800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:52.250 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:52.235+0000 7f6f4212e700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f6f3c161d40 0x7f6f3c167b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:52.250 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:52.235+0000 7f6f4212e700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f3c0ce350 con 0x7f6f3c15d9e0 2023-10-14T10:36:52.250 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:52.235+0000 7f6f4212e700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f3c1621f0 con 0x7f6f3c161d40 2023-10-14T10:36:52.251 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:52.235+0000 7f6f4212e700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f3c0b5140 con 0x7f6f3c15ddc0 2023-10-14T10:36:52.251 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:52.235+0000 7f6f3bfff700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f6f3c161d40 0x7f6f3c167b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:52.252 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:52.235+0000 7f6f3b7fe700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f6f3c15d9e0 0x7f6f3c0d1fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:53278/0 (socket says 172.21.15.67:53278) 2023-10-14T10:36:52.252 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:52.235+0000 7f6f3b7fe700 1 -- 172.21.15.67:0/22608810 learned_addr learned my addr 172.21.15.67:0/22608810 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:36:52.253 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:52.236+0000 7f6f3affd700 1 --2- 172.21.15.67:0/22608810 >> v2:172.21.15.72:3300/0 conn(0x7f6f3c15ddc0 0x7f6f3c161800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:52.253 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:52.236+0000 7f6f3bfff700 1 -- 172.21.15.67:0/22608810 >> v2:172.21.15.72:3300/0 conn(0x7f6f3c15ddc0 msgr2=0x7f6f3c161800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:52.253 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:52.236+0000 7f6f3bfff700 1 --2- 172.21.15.67:0/22608810 >> v2:172.21.15.72:3300/0 conn(0x7f6f3c15ddc0 0x7f6f3c161800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:52.254 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:52.236+0000 7f6f3bfff700 1 -- 172.21.15.67:0/22608810 >> v1:172.21.15.67:6789/0 conn(0x7f6f3c15d9e0 legacy=0x7f6f3c0d1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:52.254 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:52.236+0000 7f6f3a7fc700 1 -- 172.21.15.67:0/22608810 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 183532919 0 0) 0x7f6f3c0ce350 con 0x7f6f3c15d9e0 2023-10-14T10:36:52.254 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:52.236+0000 7f6f3bfff700 1 -- 172.21.15.67:0/22608810 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f3c162630 con 0x7f6f3c161d40 2023-10-14T10:36:52.255 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:52.236+0000 7f6f3bfff700 1 --2- 172.21.15.67:0/22608810 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f3c161d40 0x7f6f3c167b30 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c0b9880 tx=0x7f6f3400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0b064ab3684530a server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:52.255 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:52.236+0000 7f6f3a7fc700 1 -- 172.21.15.67:0/22608810 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f3400e070 con 0x7f6f3c161d40 2023-10-14T10:36:52.255 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:52.236+0000 7f6f3a7fc700 1 -- 172.21.15.67:0/22608810 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f3400bae0 con 0x7f6f3c161d40 2023-10-14T10:36:52.256 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:52.236+0000 7f6f3a7fc700 1 -- 172.21.15.67:0/22608810 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f3400be90 con 0x7f6f3c161d40 2023-10-14T10:36:52.256 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:52.237+0000 7f6f4212e700 1 -- 172.21.15.67:0/22608810 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f3c161d40 msgr2=0x7f6f3c167b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:52.257 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:52.237+0000 7f6f4212e700 1 --2- 172.21.15.67:0/22608810 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f3c161d40 0x7f6f3c167b30 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c0b9880 tx=0x7f6f3400b330 comp rx=0 tx=0).stop 2023-10-14T10:36:52.257 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:52.237+0000 7f6f4212e700 1 -- 172.21.15.67:0/22608810 shutdown_connections 2023-10-14T10:36:52.257 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:52.237+0000 7f6f4212e700 1 --2- 172.21.15.67:0/22608810 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f3c161d40 0x7f6f3c167b30 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:52.258 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:52.237+0000 7f6f4212e700 1 --2- 172.21.15.67:0/22608810 >> v2:172.21.15.72:3300/0 conn(0x7f6f3c15ddc0 0x7f6f3c161800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:52.258 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:52.237+0000 7f6f4212e700 1 -- 172.21.15.67:0/22608810 >> 172.21.15.67:0/22608810 conn(0x7f6f3c09cb40 msgr2=0x7f6f3c0d37d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:52.258 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:52.237+0000 7f6f4212e700 1 -- 172.21.15.67:0/22608810 shutdown_connections 2023-10-14T10:36:52.259 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:52.237+0000 7f6f4212e700 1 -- 172.21.15.67:0/22608810 wait complete. 2023-10-14T10:36:52.259 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:52.238+0000 7f6f4212e700 1 Processor -- start 2023-10-14T10:36:52.259 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:52.238+0000 7f6f4212e700 1 -- start start 2023-10-14T10:36:52.260 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:52.239+0000 7f6f4212e700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f3c15d9e0 0x7f6f3c1f2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:52.260 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:52.239+0000 7f6f4212e700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6f3c15ddc0 0x7f6f3c15f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:52.260 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:52.239+0000 7f6f4212e700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f6f3c161d40 0x7f6f3c15f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:52.261 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:52.239+0000 7f6f4212e700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f3c14a380 con 0x7f6f3c15d9e0 2023-10-14T10:36:52.261 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:52.239+0000 7f6f4212e700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f3c0b91d0 con 0x7f6f3c15ddc0 2023-10-14T10:36:52.261 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:52.239+0000 7f6f4212e700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f3c0b94b0 con 0x7f6f3c161d40 2023-10-14T10:36:52.262 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:52.239+0000 7f6f3b7fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f3c15d9e0 0x7f6f3c1f2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:52.262 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:52.239+0000 7f6f3b7fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f3c15d9e0 0x7f6f3c1f2b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45516/0 (socket says 172.21.15.67:45516) 2023-10-14T10:36:52.263 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:52.239+0000 7f6f3b7fe700 1 -- 172.21.15.67:0/2881122274 learned_addr learned my addr 172.21.15.67:0/2881122274 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:52.263 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:52.239+0000 7f6f3affd700 1 --2- 172.21.15.67:0/2881122274 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6f3c15ddc0 0x7f6f3c15f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:52.263 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:52.239+0000 7f6f3bfff700 1 --2- 172.21.15.67:0/2881122274 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f6f3c161d40 0x7f6f3c15f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:52.264 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:52.239+0000 7f6f3b7fe700 1 -- 172.21.15.67:0/2881122274 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f6f3c161d40 msgr2=0x7f6f3c15f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:52.264 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:52.239+0000 7f6f3b7fe700 1 --2- 172.21.15.67:0/2881122274 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f6f3c161d40 0x7f6f3c15f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:52.264 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:52.239+0000 7f6f3b7fe700 1 -- 172.21.15.67:0/2881122274 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6f3c15ddc0 msgr2=0x7f6f3c15f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:52.265 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:52.239+0000 7f6f3b7fe700 1 --2- 172.21.15.67:0/2881122274 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6f3c15ddc0 0x7f6f3c15f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:52.266 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:52.239+0000 7f6f3b7fe700 1 -- 172.21.15.67:0/2881122274 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f34009a70 con 0x7f6f3c15d9e0 2023-10-14T10:36:52.266 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:52.239+0000 7f6f3affd700 1 --2- 172.21.15.67:0/2881122274 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6f3c15ddc0 0x7f6f3c15f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-14T10:36:52.266 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:52.239+0000 7f6f3b7fe700 1 --2- 172.21.15.67:0/2881122274 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f3c15d9e0 0x7f6f3c1f2b50 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f6f240011e0 tx=0x7f6f2400cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:52.266 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:52.240+0000 7f6f33fff700 1 -- 172.21.15.67:0/2881122274 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f24004c80 con 0x7f6f3c15d9e0 2023-10-14T10:36:52.266 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:52.240+0000 7f6f33fff700 1 -- 172.21.15.67:0/2881122274 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f24004e00 con 0x7f6f3c15d9e0 2023-10-14T10:36:52.267 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:52.240+0000 7f6f33fff700 1 -- 172.21.15.67:0/2881122274 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f24007310 con 0x7f6f3c15d9e0 2023-10-14T10:36:52.267 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:52.240+0000 7f6f4212e700 1 -- 172.21.15.67:0/2881122274 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f3c0cab80 con 0x7f6f3c15d9e0 2023-10-14T10:36:52.267 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:52.240+0000 7f6f4212e700 1 -- 172.21.15.67:0/2881122274 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f3c0cb140 con 0x7f6f3c15d9e0 2023-10-14T10:36:52.267 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:52.241+0000 7f6f4212e700 1 -- 172.21.15.67:0/2881122274 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f3c067d20 con 0x7f6f3c15d9e0 2023-10-14T10:36:52.267 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:52.241+0000 7f6f33fff700 1 -- 172.21.15.67:0/2881122274 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f6f24007470 con 0x7f6f3c15d9e0 2023-10-14T10:36:52.268 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:52.242+0000 7f6f33fff700 1 --2- 172.21.15.67:0/2881122274 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f6f1c06af00 0x7f6f1c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:52.268 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:52.242+0000 7f6f33fff700 1 -- 172.21.15.67:0/2881122274 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f6f24089080 con 0x7f6f3c15d9e0 2023-10-14T10:36:52.268 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:52.242+0000 7f6f3affd700 1 --2- 172.21.15.67:0/2881122274 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f6f1c06af00 0x7f6f1c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:52.268 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:52.242+0000 7f6f3affd700 1 --2- 172.21.15.67:0/2881122274 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f6f1c06af00 0x7f6f1c06d3c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c0a16f0 tx=0x7f6f2c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:52.269 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:52.246+0000 7f6f33fff700 1 -- 172.21.15.67:0/2881122274 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6f24054430 con 0x7f6f3c15d9e0 2023-10-14T10:36:52.467 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:52.465+0000 7f6f4212e700 1 -- 172.21.15.67:0/2881122274 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f6f3c08d8d0 con 0x7f6f3c15d9e0 2023-10-14T10:36:53.034 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.032+0000 7f6f33fff700 1 -- 172.21.15.67:0/2881122274 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v27) v1 ==== 97+0+0 (secure 0 0 0) 0x7f6f24057a50 con 0x7f6f3c15d9e0 2023-10-14T10:36:53.036 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.034+0000 7f6f4212e700 1 -- 172.21.15.67:0/2881122274 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f6f1c06af00 msgr2=0x7f6f1c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:53.037 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.034+0000 7f6f4212e700 1 --2- 172.21.15.67:0/2881122274 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f6f1c06af00 0x7f6f1c06d3c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c0a16f0 tx=0x7f6f2c008040 comp rx=0 tx=0).stop 2023-10-14T10:36:53.037 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.034+0000 7f6f4212e700 1 -- 172.21.15.67:0/2881122274 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f3c15d9e0 msgr2=0x7f6f3c1f2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:53.037 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.034+0000 7f6f4212e700 1 --2- 172.21.15.67:0/2881122274 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f3c15d9e0 0x7f6f3c1f2b50 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f6f240011e0 tx=0x7f6f2400cae0 comp rx=0 tx=0).stop 2023-10-14T10:36:53.037 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.034+0000 7f6f4212e700 1 -- 172.21.15.67:0/2881122274 shutdown_connections 2023-10-14T10:36:53.037 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.034+0000 7f6f4212e700 1 --2- 172.21.15.67:0/2881122274 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f6f1c06af00 0x7f6f1c06d3c0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:53.038 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.034+0000 7f6f4212e700 1 --2- 172.21.15.67:0/2881122274 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f6f3c161d40 0x7f6f3c15f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:53.038 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.034+0000 7f6f4212e700 1 --2- 172.21.15.67:0/2881122274 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f3c15d9e0 0x7f6f3c1f2b50 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:53.038 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.034+0000 7f6f4212e700 1 --2- 172.21.15.67:0/2881122274 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6f3c15ddc0 0x7f6f3c15f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:53.038 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.034+0000 7f6f4212e700 1 -- 172.21.15.67:0/2881122274 >> 172.21.15.67:0/2881122274 conn(0x7f6f3c09cb40 msgr2=0x7f6f3c06d190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:53.038 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.034+0000 7f6f4212e700 1 -- 172.21.15.67:0/2881122274 shutdown_connections 2023-10-14T10:36:53.039 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.034+0000 7f6f4212e700 1 -- 172.21.15.67:0/2881122274 wait complete. 2023-10-14T10:36:53.039 INFO:teuthology.orchestra.run.smithi067.stderr: removed all blocklist entries 2023-10-14T10:36:53.056 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-14T10:36:53.185 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.183+0000 7fd776db4700 1 Processor -- start 2023-10-14T10:36:53.185 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.183+0000 7fd776db4700 1 -- start start 2023-10-14T10:36:53.188 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.183+0000 7fd776db4700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fd7700cd950 0x7fd77015e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:53.188 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.183+0000 7fd776db4700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7700cf260 con 0x7fd7700cede0 2023-10-14T10:36:53.188 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.183+0000 7fd776db4700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7700b2d60 con 0x7fd77006a840 2023-10-14T10:36:53.188 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.183+0000 7fd776db4700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7701619a0 con 0x7fd7700cd950 2023-10-14T10:36:53.188 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.184+0000 7fd76ffff700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fd7700cede0 0x7fd77015f570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:53292/0 (socket says 172.21.15.67:53292) 2023-10-14T10:36:53.189 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.184+0000 7fd76ffff700 1 -- 172.21.15.67:0/1094469536 learned_addr learned my addr 172.21.15.67:0/1094469536 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:36:53.189 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.184+0000 7fd774b50700 1 --2- 172.21.15.67:0/1094469536 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fd7700cd950 0x7fd77015e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:53.189 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.184+0000 7fd76f7fe700 1 -- 172.21.15.67:0/1094469536 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 7429854 0 0) 0x7fd7700cf260 con 0x7fd7700cede0 2023-10-14T10:36:53.189 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.184+0000 7fd76f7fe700 1 -- 172.21.15.67:0/1094469536 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7600035b0 con 0x7fd7700cede0 2023-10-14T10:36:53.189 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.184+0000 7fd76f7fe700 1 -- 172.21.15.67:0/1094469536 <== mon.1 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2857702992 0 0) 0x7fd7700b2d60 con 0x7fd77006a840 2023-10-14T10:36:53.190 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.184+0000 7fd76f7fe700 1 -- 172.21.15.67:0/1094469536 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7700cf260 con 0x7fd77006a840 2023-10-14T10:36:53.190 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.184+0000 7fd76f7fe700 1 -- 172.21.15.67:0/1094469536 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3156243989 0 0) 0x7fd7600035b0 con 0x7fd7700cede0 2023-10-14T10:36:53.190 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.184+0000 7fd774b50700 1 -- 172.21.15.67:0/1094469536 >> v1:172.21.15.72:6789/0 conn(0x7fd77006a840 legacy=0x7fd770161df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:53.190 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.184+0000 7fd774b50700 1 -- 172.21.15.67:0/1094469536 >> v1:172.21.15.67:6789/0 conn(0x7fd7700cede0 legacy=0x7fd77015f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:53.191 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.184+0000 7fd774b50700 1 -- 172.21.15.67:0/1094469536 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7700cdd30 con 0x7fd7700cd950 2023-10-14T10:36:53.191 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.184+0000 7fd774b50700 1 --2- 172.21.15.67:0/1094469536 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fd7700cd950 0x7fd77015e2f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd764009870 tx=0x7fd764009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3a568ae3264a578 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:53.191 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.184+0000 7fd76f7fe700 1 -- 172.21.15.67:0/1094469536 <== mon.2 v2:172.21.15.72:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd75c0017f0 con 0x7fd7700cd950 2023-10-14T10:36:53.191 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.184+0000 7fd76f7fe700 1 -- 172.21.15.67:0/1094469536 <== mon.2 v2:172.21.15.72:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd764004030 con 0x7fd7700cd950 2023-10-14T10:36:53.191 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.185+0000 7fd76f7fe700 1 -- 172.21.15.67:0/1094469536 <== mon.2 v2:172.21.15.72:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7640043c0 con 0x7fd7700cd950 2023-10-14T10:36:53.191 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.185+0000 7fd776db4700 1 -- 172.21.15.67:0/1094469536 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fd7700cd950 msgr2=0x7fd77015e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:53.192 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.185+0000 7fd776db4700 1 --2- 172.21.15.67:0/1094469536 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fd7700cd950 0x7fd77015e2f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd764009870 tx=0x7fd764009b80 comp rx=0 tx=0).stop 2023-10-14T10:36:53.192 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.185+0000 7fd776db4700 1 -- 172.21.15.67:0/1094469536 shutdown_connections 2023-10-14T10:36:53.192 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.185+0000 7fd776db4700 1 --2- 172.21.15.67:0/1094469536 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fd7700cd950 0x7fd77015e2f0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:53.192 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.185+0000 7fd776db4700 1 -- 172.21.15.67:0/1094469536 >> 172.21.15.67:0/1094469536 conn(0x7fd77006dae0 msgr2=0x7fd77006def0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:53.192 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.185+0000 7fd776db4700 1 -- 172.21.15.67:0/1094469536 shutdown_connections 2023-10-14T10:36:53.193 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.185+0000 7fd776db4700 1 -- 172.21.15.67:0/1094469536 wait complete. 2023-10-14T10:36:53.195 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.186+0000 7fd776db4700 1 Processor -- start 2023-10-14T10:36:53.196 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.186+0000 7fd776db4700 1 -- start start 2023-10-14T10:36:53.196 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.187+0000 7fd776db4700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fd77006a840 0x7fd7700b5860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:53.196 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.187+0000 7fd776db4700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd7700cd950 0x7fd7700b0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:53.196 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.187+0000 7fd776db4700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd7700cede0 0x7fd7700b1240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:53.197 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.187+0000 7fd776db4700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7700b5da0 con 0x7fd7700cd950 2023-10-14T10:36:53.197 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.187+0000 7fd776db4700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7700c7a20 con 0x7fd7700cede0 2023-10-14T10:36:53.197 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.187+0000 7fd776db4700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7700c7c30 con 0x7fd77006a840 2023-10-14T10:36:53.197 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.187+0000 7fd76ffff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd7700cd950 0x7fd7700b0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:53.197 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.187+0000 7fd76ffff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd7700cd950 0x7fd7700b0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45530/0 (socket says 172.21.15.67:45530) 2023-10-14T10:36:53.198 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.187+0000 7fd76ffff700 1 -- 172.21.15.67:0/1134294370 learned_addr learned my addr 172.21.15.67:0/1134294370 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:53.198 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.187+0000 7fd775351700 1 --2- 172.21.15.67:0/1134294370 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd7700cede0 0x7fd7700b1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:53.198 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.188+0000 7fd774b50700 1 --2- 172.21.15.67:0/1134294370 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fd77006a840 0x7fd7700b5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:53.198 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.188+0000 7fd76ffff700 1 -- 172.21.15.67:0/1134294370 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fd77006a840 msgr2=0x7fd7700b5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:53.198 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.188+0000 7fd76ffff700 1 --2- 172.21.15.67:0/1134294370 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fd77006a840 0x7fd7700b5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:53.199 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.188+0000 7fd76ffff700 1 -- 172.21.15.67:0/1134294370 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd7700cede0 msgr2=0x7fd7700b1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:53.199 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.188+0000 7fd76ffff700 1 --2- 172.21.15.67:0/1134294370 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd7700cede0 0x7fd7700b1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:53.199 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.188+0000 7fd76ffff700 1 -- 172.21.15.67:0/1134294370 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7640094a0 con 0x7fd7700cd950 2023-10-14T10:36:53.199 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.188+0000 7fd76ffff700 1 --2- 172.21.15.67:0/1134294370 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd7700cd950 0x7fd7700b0d00 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd75c00c750 tx=0x7fd75c00cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:53.199 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.188+0000 7fd76d7fa700 1 -- 172.21.15.67:0/1134294370 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd75c0074e0 con 0x7fd7700cd950 2023-10-14T10:36:53.200 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.188+0000 7fd76d7fa700 1 -- 172.21.15.67:0/1134294370 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd75c007640 con 0x7fd7700cd950 2023-10-14T10:36:53.200 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.188+0000 7fd76d7fa700 1 -- 172.21.15.67:0/1134294370 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd75c005a70 con 0x7fd7700cd950 2023-10-14T10:36:53.200 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.188+0000 7fd776db4700 1 -- 172.21.15.67:0/1134294370 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7700c7f10 con 0x7fd7700cd950 2023-10-14T10:36:53.200 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.188+0000 7fd776db4700 1 -- 172.21.15.67:0/1134294370 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7700c84d0 con 0x7fd7700cd950 2023-10-14T10:36:53.200 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.189+0000 7fd776db4700 1 -- 172.21.15.67:0/1134294370 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7700c86f0 con 0x7fd7700cd950 2023-10-14T10:36:53.200 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.190+0000 7fd76d7fa700 1 -- 172.21.15.67:0/1134294370 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd75c005bd0 con 0x7fd7700cd950 2023-10-14T10:36:53.201 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.190+0000 7fd76d7fa700 1 --2- 172.21.15.67:0/1134294370 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fd75406af00 0x7fd75406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:53.201 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.191+0000 7fd76d7fa700 1 -- 172.21.15.67:0/1134294370 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd75c088c10 con 0x7fd7700cd950 2023-10-14T10:36:53.201 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.191+0000 7fd774b50700 1 --2- 172.21.15.67:0/1134294370 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fd75406af00 0x7fd75406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:53.201 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.191+0000 7fd774b50700 1 --2- 172.21.15.67:0/1134294370 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fd75406af00 0x7fd75406d3c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd7640097c0 tx=0x7fd764008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:53.202 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.195+0000 7fd76d7fa700 1 -- 172.21.15.67:0/1134294370 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd75c0540f0 con 0x7fd7700cd950 2023-10-14T10:36:53.477 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.475+0000 7fd776db4700 1 -- 172.21.15.67:0/1134294370 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fd7700b2100 con 0x7fd7700cd950 2023-10-14T10:36:53.478 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.476+0000 7fd76d7fa700 1 -- 172.21.15.67:0/1134294370 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v6) v1 ==== 80+0+6645 (secure 0 0 0) 0x7fd75c017020 con 0x7fd7700cd950 2023-10-14T10:36:53.478 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-14T10:36:53.479 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2023-10-14T10:36:53.479 INFO:teuthology.orchestra.run.smithi067.stdout: "auth_dump": [ 2023-10-14T10:36:53.479 INFO:teuthology.orchestra.run.smithi067.stdout: { 2023-10-14T10:36:53.479 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "mds.a", 2023-10-14T10:36:53.479 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQDfbiplhHkzNRAAu6V9TUs+PZgWJTL1PslPTQ==", 2023-10-14T10:36:53.480 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2023-10-14T10:36:53.480 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow", 2023-10-14T10:36:53.480 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow *", 2023-10-14T10:36:53.480 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow *", 2023-10-14T10:36:53.480 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2023-10-14T10:36:53.481 INFO:teuthology.orchestra.run.smithi067.stdout: } 2023-10-14T10:36:53.481 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2023-10-14T10:36:53.481 INFO:teuthology.orchestra.run.smithi067.stdout: { 2023-10-14T10:36:53.481 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "mds.b", 2023-10-14T10:36:53.481 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQDgbiplYRkFBhAALpCIvRKJUBBNWk7ulMhsHg==", 2023-10-14T10:36:53.482 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2023-10-14T10:36:53.482 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow", 2023-10-14T10:36:53.482 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow *", 2023-10-14T10:36:53.482 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow *", 2023-10-14T10:36:53.483 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2023-10-14T10:36:53.483 INFO:teuthology.orchestra.run.smithi067.stdout: } 2023-10-14T10:36:53.483 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2023-10-14T10:36:53.483 INFO:teuthology.orchestra.run.smithi067.stdout: { 2023-10-14T10:36:53.483 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "mds.c", 2023-10-14T10:36:53.484 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQDgbiplLMACARAAtLK9Ksyh+PWJMflIG7N0gg==", 2023-10-14T10:36:53.484 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2023-10-14T10:36:53.484 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow", 2023-10-14T10:36:53.484 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow *", 2023-10-14T10:36:53.484 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow *", 2023-10-14T10:36:53.485 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2023-10-14T10:36:53.485 INFO:teuthology.orchestra.run.smithi067.stdout: } 2023-10-14T10:36:53.485 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2023-10-14T10:36:53.485 INFO:teuthology.orchestra.run.smithi067.stdout: { 2023-10-14T10:36:53.485 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "mds.d", 2023-10-14T10:36:53.486 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQDgbipldutQDRAAptZ4A6PV8ngnjdyDLP01eg==", 2023-10-14T10:36:53.486 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2023-10-14T10:36:53.486 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow", 2023-10-14T10:36:53.486 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow *", 2023-10-14T10:36:53.486 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow *", 2023-10-14T10:36:53.487 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2023-10-14T10:36:53.487 INFO:teuthology.orchestra.run.smithi067.stdout: } 2023-10-14T10:36:53.487 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2023-10-14T10:36:53.487 INFO:teuthology.orchestra.run.smithi067.stdout: { 2023-10-14T10:36:53.487 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "osd.0", 2023-10-14T10:36:53.487 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQDjbiplJJhtLBAA68bGP9xw1933h7ZwGRvrTQ==", 2023-10-14T10:36:53.488 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2023-10-14T10:36:53.488 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow profile osd", 2023-10-14T10:36:53.488 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile osd", 2023-10-14T10:36:53.488 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2023-10-14T10:36:53.488 INFO:teuthology.orchestra.run.smithi067.stdout: } 2023-10-14T10:36:53.489 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2023-10-14T10:36:53.489 INFO:teuthology.orchestra.run.smithi067.stdout: { 2023-10-14T10:36:53.489 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "osd.1", 2023-10-14T10:36:53.489 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQDmbiplbkEwLBAASrn+o/58EZvt8jxm/x9Ing==", 2023-10-14T10:36:53.490 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2023-10-14T10:36:53.490 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow profile osd", 2023-10-14T10:36:53.490 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile osd", 2023-10-14T10:36:53.490 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2023-10-14T10:36:53.491 INFO:teuthology.orchestra.run.smithi067.stdout: } 2023-10-14T10:36:53.491 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2023-10-14T10:36:53.491 INFO:teuthology.orchestra.run.smithi067.stdout: { 2023-10-14T10:36:53.491 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "osd.2", 2023-10-14T10:36:53.491 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQDpbipl+X0ULBAAjZfhTIYrX7SGQcl+21d04Q==", 2023-10-14T10:36:53.492 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2023-10-14T10:36:53.492 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow profile osd", 2023-10-14T10:36:53.492 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile osd", 2023-10-14T10:36:53.492 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2023-10-14T10:36:53.493 INFO:teuthology.orchestra.run.smithi067.stdout: } 2023-10-14T10:36:53.493 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2023-10-14T10:36:53.493 INFO:teuthology.orchestra.run.smithi067.stdout: { 2023-10-14T10:36:53.493 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "osd.3", 2023-10-14T10:36:53.493 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQDsbipldEFxKxAAbNwsAPRkpQOAYiQjBsTTDw==", 2023-10-14T10:36:53.494 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2023-10-14T10:36:53.494 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow profile osd", 2023-10-14T10:36:53.494 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile osd", 2023-10-14T10:36:53.494 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2023-10-14T10:36:53.494 INFO:teuthology.orchestra.run.smithi067.stdout: } 2023-10-14T10:36:53.495 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2023-10-14T10:36:53.495 INFO:teuthology.orchestra.run.smithi067.stdout: { 2023-10-14T10:36:53.495 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "osd.4", 2023-10-14T10:36:53.495 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQDzbiplkgfNAxAAwtb4YKBFvu0SPQ81F9KphQ==", 2023-10-14T10:36:53.495 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2023-10-14T10:36:53.496 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow profile osd", 2023-10-14T10:36:53.496 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile osd", 2023-10-14T10:36:53.496 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2023-10-14T10:36:53.496 INFO:teuthology.orchestra.run.smithi067.stdout: } 2023-10-14T10:36:53.496 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2023-10-14T10:36:53.497 INFO:teuthology.orchestra.run.smithi067.stdout: { 2023-10-14T10:36:53.497 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "osd.5", 2023-10-14T10:36:53.497 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQD2bipltwp0AxAAiduwcaM3rVGQa05otP99yA==", 2023-10-14T10:36:53.497 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2023-10-14T10:36:53.497 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow profile osd", 2023-10-14T10:36:53.498 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile osd", 2023-10-14T10:36:53.498 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2023-10-14T10:36:53.498 INFO:teuthology.orchestra.run.smithi067.stdout: } 2023-10-14T10:36:53.498 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2023-10-14T10:36:53.498 INFO:teuthology.orchestra.run.smithi067.stdout: { 2023-10-14T10:36:53.499 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "osd.6", 2023-10-14T10:36:53.499 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQD5biplD9QqAxAA4jezF4GFarOijLCF5XTXMQ==", 2023-10-14T10:36:53.499 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2023-10-14T10:36:53.499 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow profile osd", 2023-10-14T10:36:53.499 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile osd", 2023-10-14T10:36:53.500 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2023-10-14T10:36:53.500 INFO:teuthology.orchestra.run.smithi067.stdout: } 2023-10-14T10:36:53.500 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2023-10-14T10:36:53.500 INFO:teuthology.orchestra.run.smithi067.stdout: { 2023-10-14T10:36:53.500 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "osd.7", 2023-10-14T10:36:53.501 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQD8biplubOvAhAA28OZRR1FtTk0x3sIFk9K7w==", 2023-10-14T10:36:53.501 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2023-10-14T10:36:53.501 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow profile osd", 2023-10-14T10:36:53.501 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile osd", 2023-10-14T10:36:53.501 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2023-10-14T10:36:53.501 INFO:teuthology.orchestra.run.smithi067.stdout: } 2023-10-14T10:36:53.502 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2023-10-14T10:36:53.502 INFO:teuthology.orchestra.run.smithi067.stdout: { 2023-10-14T10:36:53.502 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "client.0", 2023-10-14T10:36:53.502 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQDgbiplgPnBERAAH7t74KYvneMtdrMTWfuThw==", 2023-10-14T10:36:53.502 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2023-10-14T10:36:53.503 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow", 2023-10-14T10:36:53.503 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow r", 2023-10-14T10:36:53.503 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow rw", 2023-10-14T10:36:53.503 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow rwx" 2023-10-14T10:36:53.503 INFO:teuthology.orchestra.run.smithi067.stdout: } 2023-10-14T10:36:53.504 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2023-10-14T10:36:53.504 INFO:teuthology.orchestra.run.smithi067.stdout: { 2023-10-14T10:36:53.504 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "client.1", 2023-10-14T10:36:53.504 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQDgbiplo1CCGBAApLIw2B2Gg0oOfJmwaWeSsA==", 2023-10-14T10:36:53.504 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2023-10-14T10:36:53.504 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow", 2023-10-14T10:36:53.505 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow r", 2023-10-14T10:36:53.505 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow rw", 2023-10-14T10:36:53.505 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow rwx" 2023-10-14T10:36:53.505 INFO:teuthology.orchestra.run.smithi067.stdout: } 2023-10-14T10:36:53.506 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2023-10-14T10:36:53.506 INFO:teuthology.orchestra.run.smithi067.stdout: { 2023-10-14T10:36:53.506 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "client.2", 2023-10-14T10:36:53.506 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQDgbiplHnsmFRAA8jpTGMcY/uLMdRdS9/+QbQ==", 2023-10-14T10:36:53.506 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2023-10-14T10:36:53.506 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow", 2023-10-14T10:36:53.507 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow r", 2023-10-14T10:36:53.507 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow rw", 2023-10-14T10:36:53.507 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow rwx" 2023-10-14T10:36:53.507 INFO:teuthology.orchestra.run.smithi067.stdout: } 2023-10-14T10:36:53.507 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2023-10-14T10:36:53.508 INFO:teuthology.orchestra.run.smithi067.stdout: { 2023-10-14T10:36:53.508 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "client.3", 2023-10-14T10:36:53.508 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQDgbiplFIX8HhAAhID21aVUrm/0+OvEr+LSEw==", 2023-10-14T10:36:53.508 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2023-10-14T10:36:53.508 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow", 2023-10-14T10:36:53.509 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow r", 2023-10-14T10:36:53.509 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow rw", 2023-10-14T10:36:53.509 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow rwx" 2023-10-14T10:36:53.509 INFO:teuthology.orchestra.run.smithi067.stdout: } 2023-10-14T10:36:53.509 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2023-10-14T10:36:53.510 INFO:teuthology.orchestra.run.smithi067.stdout: { 2023-10-14T10:36:53.510 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "client.admin", 2023-10-14T10:36:53.510 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQDfbiplxNd9FhAAU1PDTi70GNzRdRGneHno9w==", 2023-10-14T10:36:53.510 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2023-10-14T10:36:53.510 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow *", 2023-10-14T10:36:53.510 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow *", 2023-10-14T10:36:53.511 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow *", 2023-10-14T10:36:53.511 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2023-10-14T10:36:53.511 INFO:teuthology.orchestra.run.smithi067.stdout: } 2023-10-14T10:36:53.511 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2023-10-14T10:36:53.511 INFO:teuthology.orchestra.run.smithi067.stdout: { 2023-10-14T10:36:53.512 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "client.bootstrap-mds", 2023-10-14T10:36:53.512 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQACbypli90NBBAAi0hOJ5PvxkK3RmjlbhXrDA==", 2023-10-14T10:36:53.512 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2023-10-14T10:36:53.512 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile bootstrap-mds" 2023-10-14T10:36:53.512 INFO:teuthology.orchestra.run.smithi067.stdout: } 2023-10-14T10:36:53.513 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2023-10-14T10:36:53.513 INFO:teuthology.orchestra.run.smithi067.stdout: { 2023-10-14T10:36:53.513 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "client.bootstrap-mgr", 2023-10-14T10:36:53.513 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQACbyplTOcNBBAArBBjpKCuJHZDWrqrIp75xQ==", 2023-10-14T10:36:53.513 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2023-10-14T10:36:53.514 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile bootstrap-mgr" 2023-10-14T10:36:53.514 INFO:teuthology.orchestra.run.smithi067.stdout: } 2023-10-14T10:36:53.514 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2023-10-14T10:36:53.514 INFO:teuthology.orchestra.run.smithi067.stdout: { 2023-10-14T10:36:53.514 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "client.bootstrap-osd", 2023-10-14T10:36:53.515 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQACbypla/ENBBAAo1knwCUEg3i0jArhNxzQzw==", 2023-10-14T10:36:53.515 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2023-10-14T10:36:53.515 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile bootstrap-osd" 2023-10-14T10:36:53.515 INFO:teuthology.orchestra.run.smithi067.stdout: } 2023-10-14T10:36:53.515 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2023-10-14T10:36:53.516 INFO:teuthology.orchestra.run.smithi067.stdout: { 2023-10-14T10:36:53.516 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "client.bootstrap-rbd", 2023-10-14T10:36:53.516 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQACbypli/kNBBAAPRkMfSlsplWYIo+L4kyNYw==", 2023-10-14T10:36:53.516 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2023-10-14T10:36:53.516 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile bootstrap-rbd" 2023-10-14T10:36:53.516 INFO:teuthology.orchestra.run.smithi067.stdout: } 2023-10-14T10:36:53.517 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2023-10-14T10:36:53.517 INFO:teuthology.orchestra.run.smithi067.stdout: { 2023-10-14T10:36:53.517 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-10-14T10:36:53.517 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQACbypl5gEOBBAAKMawUAx0Oy7gyVT4dokNCA==", 2023-10-14T10:36:53.517 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2023-10-14T10:36:53.518 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-10-14T10:36:53.518 INFO:teuthology.orchestra.run.smithi067.stdout: } 2023-10-14T10:36:53.518 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2023-10-14T10:36:53.518 INFO:teuthology.orchestra.run.smithi067.stdout: { 2023-10-14T10:36:53.518 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "client.bootstrap-rgw", 2023-10-14T10:36:53.519 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQACbyplaQoOBBAAjAO2hmk4PLC0PTbDW2wJxQ==", 2023-10-14T10:36:53.519 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2023-10-14T10:36:53.519 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile bootstrap-rgw" 2023-10-14T10:36:53.519 INFO:teuthology.orchestra.run.smithi067.stdout: } 2023-10-14T10:36:53.519 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2023-10-14T10:36:53.520 INFO:teuthology.orchestra.run.smithi067.stdout: { 2023-10-14T10:36:53.520 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "mgr.x", 2023-10-14T10:36:53.520 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQDfbipl5Z+hMRAA5qRfiQzDEjpQ93opgGqVOQ==", 2023-10-14T10:36:53.520 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2023-10-14T10:36:53.520 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow *", 2023-10-14T10:36:53.521 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile mgr", 2023-10-14T10:36:53.521 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2023-10-14T10:36:53.521 INFO:teuthology.orchestra.run.smithi067.stdout: } 2023-10-14T10:36:53.521 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2023-10-14T10:36:53.521 INFO:teuthology.orchestra.run.smithi067.stdout: { 2023-10-14T10:36:53.521 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "mgr.y", 2023-10-14T10:36:53.522 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQDfbiplkwIpLBAAG1UzPJBzmMBeRev2BTrBEA==", 2023-10-14T10:36:53.522 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2023-10-14T10:36:53.522 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow *", 2023-10-14T10:36:53.522 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile mgr", 2023-10-14T10:36:53.522 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2023-10-14T10:36:53.523 INFO:teuthology.orchestra.run.smithi067.stdout: } 2023-10-14T10:36:53.523 INFO:teuthology.orchestra.run.smithi067.stdout: } 2023-10-14T10:36:53.523 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2023-10-14T10:36:53.523 INFO:teuthology.orchestra.run.smithi067.stdout:} 2023-10-14T10:36:53.525 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.478+0000 7fd776db4700 1 -- 172.21.15.67:0/1134294370 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fd75406af00 msgr2=0x7fd75406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:53.525 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.478+0000 7fd776db4700 1 --2- 172.21.15.67:0/1134294370 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fd75406af00 0x7fd75406d3c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd7640097c0 tx=0x7fd764008030 comp rx=0 tx=0).stop 2023-10-14T10:36:53.525 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.478+0000 7fd776db4700 1 -- 172.21.15.67:0/1134294370 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd7700cd950 msgr2=0x7fd7700b0d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:53.525 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.478+0000 7fd776db4700 1 --2- 172.21.15.67:0/1134294370 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd7700cd950 0x7fd7700b0d00 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd75c00c750 tx=0x7fd75c00cb10 comp rx=0 tx=0).stop 2023-10-14T10:36:53.526 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.478+0000 7fd776db4700 1 -- 172.21.15.67:0/1134294370 shutdown_connections 2023-10-14T10:36:53.526 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.478+0000 7fd776db4700 1 --2- 172.21.15.67:0/1134294370 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fd75406af00 0x7fd75406d3c0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:53.526 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.478+0000 7fd776db4700 1 --2- 172.21.15.67:0/1134294370 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd7700cede0 0x7fd7700b1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:53.526 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.478+0000 7fd776db4700 1 --2- 172.21.15.67:0/1134294370 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fd77006a840 0x7fd7700b5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:53.526 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.478+0000 7fd776db4700 1 --2- 172.21.15.67:0/1134294370 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd7700cd950 0x7fd7700b0d00 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:53.527 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.478+0000 7fd776db4700 1 -- 172.21.15.67:0/1134294370 >> 172.21.15.67:0/1134294370 conn(0x7fd77006dae0 msgr2=0x7fd7701e8ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:53.527 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.478+0000 7fd776db4700 1 -- 172.21.15.67:0/1134294370 shutdown_connections 2023-10-14T10:36:53.527 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.478+0000 7fd776db4700 1 -- 172.21.15.67:0/1134294370 wait complete. 2023-10-14T10:36:53.527 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-10-14T10:36:53.654 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.652+0000 7f5eab057700 1 Processor -- start 2023-10-14T10:36:53.654 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.652+0000 7f5eab057700 1 -- start start 2023-10-14T10:36:53.657 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.653+0000 7f5eab057700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f5ea415d9e0 0x7f5ea415ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:53.657 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.653+0000 7f5eab057700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ea40ce350 con 0x7f5ea40d1fc0 2023-10-14T10:36:53.657 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.653+0000 7f5eab057700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ea4162630 con 0x7f5ea4161f10 2023-10-14T10:36:53.657 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.653+0000 7f5eab057700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ea40b5140 con 0x7f5ea415d9e0 2023-10-14T10:36:53.657 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.653+0000 7f5ea3fff700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f5ea40d1fc0 0x7f5ea4161800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:53306/0 (socket says 172.21.15.67:53306) 2023-10-14T10:36:53.658 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.653+0000 7f5ea3fff700 1 -- 172.21.15.67:0/3254026317 learned_addr learned my addr 172.21.15.67:0/3254026317 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:36:53.658 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.653+0000 7f5ea8df3700 1 --2- 172.21.15.67:0/3254026317 >> v2:172.21.15.72:3300/0 conn(0x7f5ea415d9e0 0x7f5ea415ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:53.659 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.653+0000 7f5ea37fe700 1 -- 172.21.15.67:0/3254026317 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4178829027 0 0) 0x7f5ea40ce350 con 0x7f5ea40d1fc0 2023-10-14T10:36:53.659 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.653+0000 7f5ea37fe700 1 -- 172.21.15.67:0/3254026317 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e940035b0 con 0x7f5ea40d1fc0 2023-10-14T10:36:53.659 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.653+0000 7f5ea8df3700 1 -- 172.21.15.67:0/3254026317 >> v1:172.21.15.72:6789/0 conn(0x7f5ea4161f10 legacy=0x7f5ea4167b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:53.659 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.653+0000 7f5ea37fe700 1 -- 172.21.15.67:0/3254026317 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1079567479 0 0) 0x7f5e940035b0 con 0x7f5ea40d1fc0 2023-10-14T10:36:53.659 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.653+0000 7f5ea8df3700 1 -- 172.21.15.67:0/3254026317 >> v1:172.21.15.67:6789/0 conn(0x7f5ea40d1fc0 legacy=0x7f5ea4161800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:53.660 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.653+0000 7f5ea8df3700 1 -- 172.21.15.67:0/3254026317 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ea40ce5f0 con 0x7f5ea415d9e0 2023-10-14T10:36:53.660 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.654+0000 7f5ea8df3700 1 --2- 172.21.15.67:0/3254026317 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5ea415d9e0 0x7f5ea415ddc0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5e98009840 tx=0x7f5e98009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1cd9dcd752fc7c0a server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:53.660 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.654+0000 7f5ea37fe700 1 -- 172.21.15.67:0/3254026317 <== mon.1 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e90001670 con 0x7f5ea415d9e0 2023-10-14T10:36:53.660 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.654+0000 7f5ea37fe700 1 -- 172.21.15.67:0/3254026317 <== mon.1 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e98004030 con 0x7f5ea415d9e0 2023-10-14T10:36:53.660 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.654+0000 7f5ea37fe700 1 -- 172.21.15.67:0/3254026317 <== mon.1 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e980043c0 con 0x7f5ea415d9e0 2023-10-14T10:36:53.661 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.654+0000 7f5eab057700 1 -- 172.21.15.67:0/3254026317 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5ea415d9e0 msgr2=0x7f5ea415ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:53.661 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.654+0000 7f5eab057700 1 --2- 172.21.15.67:0/3254026317 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5ea415d9e0 0x7f5ea415ddc0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5e98009840 tx=0x7f5e98009b50 comp rx=0 tx=0).stop 2023-10-14T10:36:53.661 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.654+0000 7f5eab057700 1 -- 172.21.15.67:0/3254026317 shutdown_connections 2023-10-14T10:36:53.661 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.654+0000 7f5eab057700 1 --2- 172.21.15.67:0/3254026317 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5ea415d9e0 0x7f5ea415ddc0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:53.661 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.654+0000 7f5eab057700 1 -- 172.21.15.67:0/3254026317 >> 172.21.15.67:0/3254026317 conn(0x7f5ea409cb40 msgr2=0x7f5ea40d37d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:53.662 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.654+0000 7f5eab057700 1 -- 172.21.15.67:0/3254026317 shutdown_connections 2023-10-14T10:36:53.662 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.654+0000 7f5eab057700 1 -- 172.21.15.67:0/3254026317 wait complete. 2023-10-14T10:36:53.664 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.656+0000 7f5eab057700 1 Processor -- start 2023-10-14T10:36:53.665 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.656+0000 7f5eab057700 1 -- start start 2023-10-14T10:36:53.665 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.656+0000 7f5eab057700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ea40d1fc0 0x7f5ea41f7590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:53.665 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.656+0000 7f5eab057700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5ea415d9e0 0x7f5ea41602d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:53.665 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.656+0000 7f5eab057700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f5ea4161f10 0x7f5ea40cf690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:53.665 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.657+0000 7f5eab057700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea4160bd0 con 0x7f5ea40d1fc0 2023-10-14T10:36:53.666 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.657+0000 7f5eab057700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea41f3a50 con 0x7f5ea415d9e0 2023-10-14T10:36:53.666 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.657+0000 7f5eab057700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea41f3d30 con 0x7f5ea4161f10 2023-10-14T10:36:53.666 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.657+0000 7f5ea8df3700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ea40d1fc0 0x7f5ea41f7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:53.666 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.657+0000 7f5ea8df3700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ea40d1fc0 0x7f5ea41f7590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45542/0 (socket says 172.21.15.67:45542) 2023-10-14T10:36:53.666 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.657+0000 7f5ea8df3700 1 -- 172.21.15.67:0/3576593303 learned_addr learned my addr 172.21.15.67:0/3576593303 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:53.667 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.657+0000 7f5ea3fff700 1 --2- 172.21.15.67:0/3576593303 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5ea415d9e0 0x7f5ea41602d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:53.667 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.657+0000 7f5ea95f4700 1 --2- 172.21.15.67:0/3576593303 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f5ea4161f10 0x7f5ea40cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:53.667 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.657+0000 7f5ea8df3700 1 -- 172.21.15.67:0/3576593303 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f5ea4161f10 msgr2=0x7f5ea40cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:53.667 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.657+0000 7f5ea8df3700 1 --2- 172.21.15.67:0/3576593303 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f5ea4161f10 0x7f5ea40cf690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:53.667 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.657+0000 7f5ea8df3700 1 -- 172.21.15.67:0/3576593303 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5ea415d9e0 msgr2=0x7f5ea41602d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:53.668 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.657+0000 7f5ea8df3700 1 --2- 172.21.15.67:0/3576593303 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5ea415d9e0 0x7f5ea41602d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:53.668 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.657+0000 7f5ea8df3700 1 -- 172.21.15.67:0/3576593303 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e980094a0 con 0x7f5ea40d1fc0 2023-10-14T10:36:53.668 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.657+0000 7f5ea3fff700 1 --2- 172.21.15.67:0/3576593303 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5ea415d9e0 0x7f5ea41602d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-14T10:36:53.669 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.657+0000 7f5ea8df3700 1 --2- 172.21.15.67:0/3576593303 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ea40d1fc0 0x7f5ea41f7590 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f5ea4067580 tx=0x7f5e98004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:53.669 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.657+0000 7f5ea17fa700 1 -- 172.21.15.67:0/3576593303 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e98010790 con 0x7f5ea40d1fc0 2023-10-14T10:36:53.669 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.657+0000 7f5ea17fa700 1 -- 172.21.15.67:0/3576593303 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e98010910 con 0x7f5ea40d1fc0 2023-10-14T10:36:53.669 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.657+0000 7f5eab057700 1 -- 172.21.15.67:0/3576593303 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ea41f4010 con 0x7f5ea40d1fc0 2023-10-14T10:36:53.670 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.657+0000 7f5ea17fa700 1 -- 172.21.15.67:0/3576593303 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e98010cc0 con 0x7f5ea40d1fc0 2023-10-14T10:36:53.670 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.657+0000 7f5eab057700 1 -- 172.21.15.67:0/3576593303 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ea41f7ad0 con 0x7f5ea40d1fc0 2023-10-14T10:36:53.670 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.658+0000 7f5eab057700 1 -- 172.21.15.67:0/3576593303 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ea4067d20 con 0x7f5ea40d1fc0 2023-10-14T10:36:53.670 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.659+0000 7f5ea17fa700 1 -- 172.21.15.67:0/3576593303 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f5e9801c420 con 0x7f5ea40d1fc0 2023-10-14T10:36:53.671 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.659+0000 7f5ea17fa700 1 --2- 172.21.15.67:0/3576593303 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f5e8806af00 0x7f5e8806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:53.671 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.659+0000 7f5ea17fa700 1 -- 172.21.15.67:0/3576593303 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5e9808acc0 con 0x7f5ea40d1fc0 2023-10-14T10:36:53.671 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.659+0000 7f5ea3fff700 1 --2- 172.21.15.67:0/3576593303 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f5e8806af00 0x7f5e8806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:53.671 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.660+0000 7f5ea3fff700 1 --2- 172.21.15.67:0/3576593303 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f5e8806af00 0x7f5e8806d3c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5e90000f60 tx=0x7f5e90009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:53.672 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.664+0000 7f5ea17fa700 1 -- 172.21.15.67:0/3576593303 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5e98056270 con 0x7f5ea40d1fc0 2023-10-14T10:36:53.949 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.947+0000 7f5eab057700 1 -- 172.21.15.67:0/3576593303 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f5ea4161560 con 0x7f5ea40d1fc0 2023-10-14T10:36:53.951 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.949+0000 7f5ea17fa700 1 -- 172.21.15.67:0/3576593303 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7f5e980243c0 con 0x7f5ea40d1fc0 2023-10-14T10:36:53.954 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.952+0000 7f5eab057700 1 -- 172.21.15.67:0/3576593303 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f5e8806af00 msgr2=0x7f5e8806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:53.954 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.952+0000 7f5eab057700 1 --2- 172.21.15.67:0/3576593303 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f5e8806af00 0x7f5e8806d3c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5e90000f60 tx=0x7f5e90009040 comp rx=0 tx=0).stop 2023-10-14T10:36:53.955 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.952+0000 7f5eab057700 1 -- 172.21.15.67:0/3576593303 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ea40d1fc0 msgr2=0x7f5ea41f7590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:53.955 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.952+0000 7f5eab057700 1 --2- 172.21.15.67:0/3576593303 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ea40d1fc0 0x7f5ea41f7590 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f5ea4067580 tx=0x7f5e98004750 comp rx=0 tx=0).stop 2023-10-14T10:36:53.955 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.952+0000 7f5eab057700 1 -- 172.21.15.67:0/3576593303 shutdown_connections 2023-10-14T10:36:53.955 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.952+0000 7f5eab057700 1 --2- 172.21.15.67:0/3576593303 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f5e8806af00 0x7f5e8806d3c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:53.955 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.952+0000 7f5eab057700 1 --2- 172.21.15.67:0/3576593303 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f5ea4161f10 0x7f5ea40cf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:53.956 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.952+0000 7f5eab057700 1 --2- 172.21.15.67:0/3576593303 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ea40d1fc0 0x7f5ea41f7590 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:53.956 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.952+0000 7f5eab057700 1 --2- 172.21.15.67:0/3576593303 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5ea415d9e0 0x7f5ea41602d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:53.956 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.952+0000 7f5eab057700 1 -- 172.21.15.67:0/3576593303 >> 172.21.15.67:0/3576593303 conn(0x7f5ea409cb40 msgr2=0x7f5ea415cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:53.956 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.952+0000 7f5eab057700 1 -- 172.21.15.67:0/3576593303 shutdown_connections 2023-10-14T10:36:53.956 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:53.952+0000 7f5eab057700 1 -- 172.21.15.67:0/3576593303 wait complete. 2023-10-14T10:36:53.966 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-10-14T10:36:54.095 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.092+0000 7f74e25f7700 1 Processor -- start 2023-10-14T10:36:54.095 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.093+0000 7f74e25f7700 1 -- start start 2023-10-14T10:36:54.097 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.093+0000 7f74e25f7700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f74dc0ce4c0 0x7f74dc0d20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:54.098 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.093+0000 7f74e25f7700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f74dc15bfa0 0x7f74dc0b0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:54.098 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.093+0000 7f74e25f7700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74dc0cce60 con 0x7f74dc15bbc0 2023-10-14T10:36:54.098 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.093+0000 7f74e25f7700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74dc0cd4e0 con 0x7f74dc0ce4c0 2023-10-14T10:36:54.098 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.093+0000 7f74e25f7700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74dc0b2ba0 con 0x7f74dc15bfa0 2023-10-14T10:36:54.099 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.093+0000 7f74dbfff700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f74dc15bbc0 0x7f74dc0cddb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:53324/0 (socket says 172.21.15.67:53324) 2023-10-14T10:36:54.099 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.093+0000 7f74dbfff700 1 -- 172.21.15.67:0/1072889425 learned_addr learned my addr 172.21.15.67:0/1072889425 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:36:54.099 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.094+0000 7f74e0b94700 1 --2- 172.21.15.67:0/1072889425 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f74dc15bfa0 0x7f74dc0b0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:54.099 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.094+0000 7f74db7fe700 1 --2- 172.21.15.67:0/1072889425 >> v2:172.21.15.72:3300/0 conn(0x7f74dc0ce4c0 0x7f74dc0d20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:54.100 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.094+0000 7f74daffd700 1 -- 172.21.15.67:0/1072889425 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1909578213 0 0) 0x7f74dc0cce60 con 0x7f74dc15bbc0 2023-10-14T10:36:54.100 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.094+0000 7f74daffd700 1 -- 172.21.15.67:0/1072889425 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74c80035b0 con 0x7f74dc15bbc0 2023-10-14T10:36:54.100 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.094+0000 7f74e0b94700 1 -- 172.21.15.67:0/1072889425 >> v2:172.21.15.72:3300/0 conn(0x7f74dc0ce4c0 msgr2=0x7f74dc0d20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:54.100 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.094+0000 7f74e0b94700 1 --2- 172.21.15.67:0/1072889425 >> v2:172.21.15.72:3300/0 conn(0x7f74dc0ce4c0 0x7f74dc0d20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:54.101 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.094+0000 7f74e0b94700 1 -- 172.21.15.67:0/1072889425 >> v1:172.21.15.67:6789/0 conn(0x7f74dc15bbc0 legacy=0x7f74dc0cddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:54.101 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.094+0000 7f74e0b94700 1 -- 172.21.15.67:0/1072889425 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74dc0cd100 con 0x7f74dc15bfa0 2023-10-14T10:36:54.101 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.094+0000 7f74db7fe700 1 --2- 172.21.15.67:0/1072889425 >> v2:172.21.15.72:3300/0 conn(0x7f74dc0ce4c0 0x7f74dc0d20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-14T10:36:54.101 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.094+0000 7f74e0b94700 1 --2- 172.21.15.67:0/1072889425 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f74dc15bfa0 0x7f74dc0b0d40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f74d4009d90 tx=0x7f74d400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7dffe880602993c server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:54.101 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.094+0000 7f74daffd700 1 -- 172.21.15.67:0/1072889425 <== mon.2 v2:172.21.15.72:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d400e070 con 0x7f74dc15bfa0 2023-10-14T10:36:54.102 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.094+0000 7f74daffd700 1 -- 172.21.15.67:0/1072889425 <== mon.2 v2:172.21.15.72:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74d400bb20 con 0x7f74dc15bfa0 2023-10-14T10:36:54.102 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.094+0000 7f74daffd700 1 -- 172.21.15.67:0/1072889425 <== mon.2 v2:172.21.15.72:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d4012430 con 0x7f74dc15bfa0 2023-10-14T10:36:54.102 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.095+0000 7f74e25f7700 1 -- 172.21.15.67:0/1072889425 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f74dc15bfa0 msgr2=0x7f74dc0b0d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:54.102 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.095+0000 7f74e25f7700 1 --2- 172.21.15.67:0/1072889425 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f74dc15bfa0 0x7f74dc0b0d40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f74d4009d90 tx=0x7f74d400b330 comp rx=0 tx=0).stop 2023-10-14T10:36:54.102 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.095+0000 7f74e25f7700 1 -- 172.21.15.67:0/1072889425 shutdown_connections 2023-10-14T10:36:54.102 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.095+0000 7f74e25f7700 1 --2- 172.21.15.67:0/1072889425 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f74dc15bfa0 0x7f74dc0b0d40 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:54.103 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.095+0000 7f74e25f7700 1 --2- 172.21.15.67:0/1072889425 >> v2:172.21.15.72:3300/0 conn(0x7f74dc0ce4c0 0x7f74dc0d20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:54.103 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.095+0000 7f74e25f7700 1 -- 172.21.15.67:0/1072889425 >> 172.21.15.67:0/1072889425 conn(0x7f74dc0d15f0 msgr2=0x7f74dc0d19d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:54.103 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.095+0000 7f74e25f7700 1 -- 172.21.15.67:0/1072889425 shutdown_connections 2023-10-14T10:36:54.103 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.095+0000 7f74e25f7700 1 -- 172.21.15.67:0/1072889425 wait complete. 2023-10-14T10:36:54.107 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.096+0000 7f74e25f7700 1 Processor -- start 2023-10-14T10:36:54.108 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.096+0000 7f74e25f7700 1 -- start start 2023-10-14T10:36:54.109 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.097+0000 7f74e25f7700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f74dc0ce4c0 0x7f74dc0c6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:54.109 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.097+0000 7f74e25f7700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f74dc15bbc0 0x7f74dc14e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:54.109 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.097+0000 7f74e25f7700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74dc15bfa0 0x7f74dc14e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:54.109 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.097+0000 7f74e25f7700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74dc14f4c0 con 0x7f74dc15bfa0 2023-10-14T10:36:54.110 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.097+0000 7f74e25f7700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74dc14f7a0 con 0x7f74dc0ce4c0 2023-10-14T10:36:54.110 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.097+0000 7f74e25f7700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74dc14fa80 con 0x7f74dc15bbc0 2023-10-14T10:36:54.110 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.097+0000 7f74e0b94700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74dc15bfa0 0x7f74dc14e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:54.111 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.097+0000 7f74e0b94700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74dc15bfa0 0x7f74dc14e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45564/0 (socket says 172.21.15.67:45564) 2023-10-14T10:36:54.111 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.097+0000 7f74e0b94700 1 -- 172.21.15.67:0/1651972280 learned_addr learned my addr 172.21.15.67:0/1651972280 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:54.111 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.097+0000 7f74db7fe700 1 --2- 172.21.15.67:0/1651972280 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f74dc15bbc0 0x7f74dc14e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:54.111 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.097+0000 7f74dbfff700 1 --2- 172.21.15.67:0/1651972280 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f74dc0ce4c0 0x7f74dc0c6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:54.112 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.097+0000 7f74e0b94700 1 -- 172.21.15.67:0/1651972280 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f74dc15bbc0 msgr2=0x7f74dc14e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:54.112 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.097+0000 7f74e0b94700 1 --2- 172.21.15.67:0/1651972280 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f74dc15bbc0 0x7f74dc14e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:54.112 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.097+0000 7f74e0b94700 1 -- 172.21.15.67:0/1651972280 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f74dc0ce4c0 msgr2=0x7f74dc0c6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:54.112 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.097+0000 7f74e0b94700 1 --2- 172.21.15.67:0/1651972280 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f74dc0ce4c0 0x7f74dc0c6250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:54.112 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.097+0000 7f74e0b94700 1 -- 172.21.15.67:0/1651972280 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74cc0094a0 con 0x7f74dc15bfa0 2023-10-14T10:36:54.112 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.097+0000 7f74db7fe700 1 --2- 172.21.15.67:0/1651972280 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f74dc15bbc0 0x7f74dc14e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-14T10:36:54.113 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.097+0000 7f74dbfff700 1 --2- 172.21.15.67:0/1651972280 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f74dc0ce4c0 0x7f74dc0c6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-14T10:36:54.113 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.098+0000 7f74e0b94700 1 --2- 172.21.15.67:0/1651972280 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74dc15bfa0 0x7f74dc14e950 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f74d4014040 tx=0x7f74d4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:54.113 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.098+0000 7f74d8ff9700 1 -- 172.21.15.67:0/1651972280 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d40129f0 con 0x7f74dc15bfa0 2023-10-14T10:36:54.113 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.098+0000 7f74d8ff9700 1 -- 172.21.15.67:0/1651972280 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74d4020070 con 0x7f74dc15bfa0 2023-10-14T10:36:54.113 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.098+0000 7f74e25f7700 1 -- 172.21.15.67:0/1651972280 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74d4009a70 con 0x7f74dc15bfa0 2023-10-14T10:36:54.114 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.098+0000 7f74d8ff9700 1 -- 172.21.15.67:0/1651972280 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d4012db0 con 0x7f74dc15bfa0 2023-10-14T10:36:54.114 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.098+0000 7f74e25f7700 1 -- 172.21.15.67:0/1651972280 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74dc0c5170 con 0x7f74dc15bfa0 2023-10-14T10:36:54.114 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.099+0000 7f74e25f7700 1 -- 172.21.15.67:0/1651972280 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74dc06dfb0 con 0x7f74dc15bfa0 2023-10-14T10:36:54.114 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.099+0000 7f74d8ff9700 1 -- 172.21.15.67:0/1651972280 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f74d4004040 con 0x7f74dc15bfa0 2023-10-14T10:36:54.114 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.100+0000 7f74d8ff9700 1 --2- 172.21.15.67:0/1651972280 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f74bc06af00 0x7f74bc06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:54.115 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.100+0000 7f74d8ff9700 1 -- 172.21.15.67:0/1651972280 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f74d4022070 con 0x7f74dc15bfa0 2023-10-14T10:36:54.115 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.100+0000 7f74dbfff700 1 --2- 172.21.15.67:0/1651972280 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f74bc06af00 0x7f74bc06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:54.115 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.100+0000 7f74dbfff700 1 --2- 172.21.15.67:0/1651972280 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f74bc06af00 0x7f74bc06d3c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f74c40011e0 tx=0x7f74c4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:54.115 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.104+0000 7f74d8ff9700 1 -- 172.21.15.67:0/1651972280 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f74d40563c0 con 0x7f74dc15bfa0 2023-10-14T10:36:54.389 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.387+0000 7f74e25f7700 1 -- 172.21.15.67:0/1651972280 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f74dc0b45f0 con 0x7f74dc15bfa0 2023-10-14T10:36:54.390 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.389+0000 7f74d8ff9700 1 -- 172.21.15.67:0/1651972280 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7f74dc0b45f0 con 0x7f74dc15bfa0 2023-10-14T10:36:54.393 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.391+0000 7f74e25f7700 1 -- 172.21.15.67:0/1651972280 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f74bc06af00 msgr2=0x7f74bc06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:54.393 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.391+0000 7f74e25f7700 1 --2- 172.21.15.67:0/1651972280 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f74bc06af00 0x7f74bc06d3c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f74c40011e0 tx=0x7f74c4009040 comp rx=0 tx=0).stop 2023-10-14T10:36:54.393 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.391+0000 7f74e25f7700 1 -- 172.21.15.67:0/1651972280 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74dc15bfa0 msgr2=0x7f74dc14e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:54.394 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.391+0000 7f74e25f7700 1 --2- 172.21.15.67:0/1651972280 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74dc15bfa0 0x7f74dc14e950 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f74d4014040 tx=0x7f74d4004ad0 comp rx=0 tx=0).stop 2023-10-14T10:36:54.394 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.391+0000 7f74e25f7700 1 -- 172.21.15.67:0/1651972280 shutdown_connections 2023-10-14T10:36:54.394 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.391+0000 7f74e25f7700 1 --2- 172.21.15.67:0/1651972280 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f74bc06af00 0x7f74bc06d3c0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:54.394 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.391+0000 7f74e25f7700 1 --2- 172.21.15.67:0/1651972280 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74dc15bfa0 0x7f74dc14e950 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:54.394 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.391+0000 7f74e25f7700 1 --2- 172.21.15.67:0/1651972280 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f74dc0ce4c0 0x7f74dc0c6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:54.395 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.391+0000 7f74e25f7700 1 --2- 172.21.15.67:0/1651972280 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f74dc15bbc0 0x7f74dc14e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:54.395 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.391+0000 7f74e25f7700 1 -- 172.21.15.67:0/1651972280 >> 172.21.15.67:0/1651972280 conn(0x7f74dc0d15f0 msgr2=0x7f74dc0d1c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:54.395 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.391+0000 7f74e25f7700 1 -- 172.21.15.67:0/1651972280 shutdown_connections 2023-10-14T10:36:54.395 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.391+0000 7f74e25f7700 1 -- 172.21.15.67:0/1651972280 wait complete. 2023-10-14T10:36:54.405 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-10-14T10:36:54.530 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.528+0000 7f22fb611700 1 Processor -- start 2023-10-14T10:36:54.531 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.529+0000 7f22fb611700 1 -- start start 2023-10-14T10:36:54.533 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.529+0000 7f22fb611700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f22f415a850 0x7f22f40cbc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:54.533 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.529+0000 7f22fb611700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f22f40cd4b0 0x7f22f40ce840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:54.534 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.529+0000 7f22fb611700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22f40ce3b0 con 0x7f22f40b2940 2023-10-14T10:36:54.534 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.529+0000 7f22fb611700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22f40b2dc0 con 0x7f22f40cd4b0 2023-10-14T10:36:54.534 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.529+0000 7f22fb611700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22f40b6290 con 0x7f22f415a850 2023-10-14T10:36:54.534 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.530+0000 7f22f9bae700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f22f40cd4b0 0x7f22f40ce840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:54.535 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.530+0000 7f22f93ad700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f22f415a850 0x7f22f40cbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:54.535 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.530+0000 7f22f8bac700 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f22f40b2940 0x7f22f40ccda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.67:33346/0 (socket says 172.21.15.67:33346) 2023-10-14T10:36:54.535 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.530+0000 7f22f8bac700 1 -- 172.21.15.67:0/2741642855 learned_addr learned my addr 172.21.15.67:0/2741642855 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:36:54.535 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.530+0000 7f22f93ad700 1 -- 172.21.15.67:0/2741642855 >> v2:172.21.15.72:3300/0 conn(0x7f22f40cd4b0 msgr2=0x7f22f40ce840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:54.535 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.530+0000 7f22f93ad700 1 --2- 172.21.15.67:0/2741642855 >> v2:172.21.15.72:3300/0 conn(0x7f22f40cd4b0 0x7f22f40ce840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:54.536 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.530+0000 7f22f93ad700 1 -- 172.21.15.67:0/2741642855 >> v1:172.21.15.72:6789/0 conn(0x7f22f40b2940 legacy=0x7f22f40ccda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:54.536 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.530+0000 7f22f93ad700 1 -- 172.21.15.67:0/2741642855 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22f40cc220 con 0x7f22f415a850 2023-10-14T10:36:54.536 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.530+0000 7f22f9bae700 1 --2- 172.21.15.67:0/2741642855 >> v2:172.21.15.72:3300/0 conn(0x7f22f40cd4b0 0x7f22f40ce840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-14T10:36:54.536 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.530+0000 7f22f93ad700 1 --2- 172.21.15.67:0/2741642855 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f22f415a850 0x7f22f40cbc40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f22e8009840 tx=0x7f22e8009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76df34569452bdd server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:54.536 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.530+0000 7f22e7fff700 1 -- 172.21.15.67:0/2741642855 <== mon.2 v2:172.21.15.72:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e8011830 con 0x7f22f415a850 2023-10-14T10:36:54.537 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.530+0000 7f22e7fff700 1 -- 172.21.15.67:0/2741642855 <== mon.2 v2:172.21.15.72:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22e8011990 con 0x7f22f415a850 2023-10-14T10:36:54.537 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.530+0000 7f22e7fff700 1 -- 172.21.15.67:0/2741642855 <== mon.2 v2:172.21.15.72:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e8011d20 con 0x7f22f415a850 2023-10-14T10:36:54.537 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.531+0000 7f22fb611700 1 -- 172.21.15.67:0/2741642855 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f22f415a850 msgr2=0x7f22f40cbc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:54.537 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.531+0000 7f22fb611700 1 --2- 172.21.15.67:0/2741642855 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f22f415a850 0x7f22f40cbc40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f22e8009840 tx=0x7f22e8009b50 comp rx=0 tx=0).stop 2023-10-14T10:36:54.537 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.531+0000 7f22fb611700 1 -- 172.21.15.67:0/2741642855 shutdown_connections 2023-10-14T10:36:54.538 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.531+0000 7f22fb611700 1 --2- 172.21.15.67:0/2741642855 >> v2:172.21.15.72:3300/0 conn(0x7f22f40cd4b0 0x7f22f40ce840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:54.538 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.531+0000 7f22fb611700 1 --2- 172.21.15.67:0/2741642855 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f22f415a850 0x7f22f40cbc40 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:54.538 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.531+0000 7f22fb611700 1 -- 172.21.15.67:0/2741642855 >> 172.21.15.67:0/2741642855 conn(0x7f22f406dac0 msgr2=0x7f22f406dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:54.538 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.531+0000 7f22fb611700 1 -- 172.21.15.67:0/2741642855 shutdown_connections 2023-10-14T10:36:54.538 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.531+0000 7f22fb611700 1 -- 172.21.15.67:0/2741642855 wait complete. 2023-10-14T10:36:54.541 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.533+0000 7f22fb611700 1 Processor -- start 2023-10-14T10:36:54.541 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.533+0000 7f22fb611700 1 -- start start 2023-10-14T10:36:54.541 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.534+0000 7f22fb611700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22f40b2940 0x7f22f40ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:54.542 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.534+0000 7f22fb611700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f22f40cd4b0 0x7f22f4157e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:54.542 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.534+0000 7f22fb611700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f22f415a850 0x7f22f40b58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:54.542 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.534+0000 7f22fb611700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22f41583c0 con 0x7f22f40b2940 2023-10-14T10:36:54.542 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.534+0000 7f22fb611700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22f40c85c0 con 0x7f22f40cd4b0 2023-10-14T10:36:54.542 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.534+0000 7f22fb611700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22f40b5e20 con 0x7f22f415a850 2023-10-14T10:36:54.543 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.534+0000 7f22f93ad700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22f40b2940 0x7f22f40ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:54.543 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.534+0000 7f22f93ad700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22f40b2940 0x7f22f40ac8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45574/0 (socket says 172.21.15.67:45574) 2023-10-14T10:36:54.543 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.534+0000 7f22f93ad700 1 -- 172.21.15.67:0/495769634 learned_addr learned my addr 172.21.15.67:0/495769634 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:54.543 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.534+0000 7f22f9bae700 1 --2- 172.21.15.67:0/495769634 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f22f415a850 0x7f22f40b58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:54.543 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.534+0000 7f22f8bac700 1 --2- 172.21.15.67:0/495769634 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f22f40cd4b0 0x7f22f4157e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:54.544 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.534+0000 7f22f93ad700 1 -- 172.21.15.67:0/495769634 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f22f415a850 msgr2=0x7f22f40b58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:54.544 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.534+0000 7f22f93ad700 1 --2- 172.21.15.67:0/495769634 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f22f415a850 0x7f22f40b58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:54.544 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.534+0000 7f22f93ad700 1 -- 172.21.15.67:0/495769634 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f22f40cd4b0 msgr2=0x7f22f4157e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:54.544 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.534+0000 7f22f93ad700 1 --2- 172.21.15.67:0/495769634 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f22f40cd4b0 0x7f22f4157e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:54.544 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.534+0000 7f22f93ad700 1 -- 172.21.15.67:0/495769634 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22ec009a70 con 0x7f22f40b2940 2023-10-14T10:36:54.545 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.534+0000 7f22f93ad700 1 --2- 172.21.15.67:0/495769634 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22f40b2940 0x7f22f40ac8b0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f22e8009fd0 tx=0x7f22e8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:54.545 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.534+0000 7f22e5ffb700 1 -- 172.21.15.67:0/495769634 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e80045d0 con 0x7f22f40b2940 2023-10-14T10:36:54.545 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.534+0000 7f22e5ffb700 1 -- 172.21.15.67:0/495769634 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22e801f070 con 0x7f22f40b2940 2023-10-14T10:36:54.545 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.534+0000 7f22fb611700 1 -- 172.21.15.67:0/495769634 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22e80094a0 con 0x7f22f40b2940 2023-10-14T10:36:54.545 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.535+0000 7f22e5ffb700 1 -- 172.21.15.67:0/495769634 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e80038e0 con 0x7f22f40b2940 2023-10-14T10:36:54.546 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.535+0000 7f22fb611700 1 -- 172.21.15.67:0/495769634 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22f40ad220 con 0x7f22f40b2940 2023-10-14T10:36:54.546 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.536+0000 7f22fb611700 1 -- 172.21.15.67:0/495769634 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22f40c8780 con 0x7f22f40b2940 2023-10-14T10:36:54.546 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.536+0000 7f22e5ffb700 1 -- 172.21.15.67:0/495769634 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f22e8003a40 con 0x7f22f40b2940 2023-10-14T10:36:54.546 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.536+0000 7f22e5ffb700 1 --2- 172.21.15.67:0/495769634 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f22d806af00 0x7f22d806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:54.546 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.537+0000 7f22e5ffb700 1 -- 172.21.15.67:0/495769634 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f22e8021070 con 0x7f22f40b2940 2023-10-14T10:36:54.547 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.537+0000 7f22f8bac700 1 --2- 172.21.15.67:0/495769634 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f22d806af00 0x7f22d806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:54.547 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.537+0000 7f22f8bac700 1 --2- 172.21.15.67:0/495769634 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f22d806af00 0x7f22d806d3c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f22f0005910 tx=0x7f22f0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:54.547 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.541+0000 7f22e5ffb700 1 -- 172.21.15.67:0/495769634 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f22e8055ea0 con 0x7f22f40b2940 2023-10-14T10:36:54.827 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.825+0000 7f22fb611700 1 -- 172.21.15.67:0/495769634 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f22f4067d20 con 0x7f22f40b2940 2023-10-14T10:36:54.831 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.830+0000 7f22e5ffb700 1 -- 172.21.15.67:0/495769634 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v9) v1 ==== 90+0+0 (secure 0 0 0) 0x7f22e80150a0 con 0x7f22f40b2940 2023-10-14T10:36:54.833 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.831+0000 7f22fb611700 1 -- 172.21.15.67:0/495769634 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f22d806af00 msgr2=0x7f22d806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:54.834 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.831+0000 7f22fb611700 1 --2- 172.21.15.67:0/495769634 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f22d806af00 0x7f22d806d3c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f22f0005910 tx=0x7f22f0009040 comp rx=0 tx=0).stop 2023-10-14T10:36:54.834 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.831+0000 7f22fb611700 1 -- 172.21.15.67:0/495769634 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22f40b2940 msgr2=0x7f22f40ac8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:54.834 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.831+0000 7f22fb611700 1 --2- 172.21.15.67:0/495769634 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22f40b2940 0x7f22f40ac8b0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f22e8009fd0 tx=0x7f22e8004750 comp rx=0 tx=0).stop 2023-10-14T10:36:54.834 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.831+0000 7f22fb611700 1 -- 172.21.15.67:0/495769634 shutdown_connections 2023-10-14T10:36:54.835 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.831+0000 7f22fb611700 1 --2- 172.21.15.67:0/495769634 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f22d806af00 0x7f22d806d3c0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:54.835 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.831+0000 7f22fb611700 1 --2- 172.21.15.67:0/495769634 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f22f415a850 0x7f22f40b58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:54.835 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.831+0000 7f22fb611700 1 --2- 172.21.15.67:0/495769634 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22f40b2940 0x7f22f40ac8b0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:54.835 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.831+0000 7f22fb611700 1 --2- 172.21.15.67:0/495769634 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f22f40cd4b0 0x7f22f4157e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:54.835 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.831+0000 7f22fb611700 1 -- 172.21.15.67:0/495769634 >> 172.21.15.67:0/495769634 conn(0x7f22f406dac0 msgr2=0x7f22f40aff10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:54.836 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.831+0000 7f22fb611700 1 -- 172.21.15.67:0/495769634 shutdown_connections 2023-10-14T10:36:54.836 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.831+0000 7f22fb611700 1 -- 172.21.15.67:0/495769634 wait complete. 2023-10-14T10:36:54.845 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-10-14T10:36:54.970 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.968+0000 7f1e8423e700 1 Processor -- start 2023-10-14T10:36:54.971 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.969+0000 7f1e8423e700 1 -- start start 2023-10-14T10:36:54.974 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.970+0000 7f1e8423e700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f1e7c0ccc20 0x7f1e7c0cd080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:54.974 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.970+0000 7f1e8423e700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f1e7c0af140 0x7f1e7c0af5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:54.974 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.970+0000 7f1e8423e700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e7c0ae130 con 0x7f1e7c0cdf60 2023-10-14T10:36:54.974 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.970+0000 7f1e8423e700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e7c0ae380 con 0x7f1e7c0af140 2023-10-14T10:36:54.975 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.970+0000 7f1e8423e700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e7c0ae660 con 0x7f1e7c0ccc20 2023-10-14T10:36:54.975 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.970+0000 7f1e817d9700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f1e7c0ccc20 0x7f1e7c0cd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:54.975 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.970+0000 7f1e827db700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f1e7c0af140 0x7f1e7c0af5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:54.975 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.970+0000 7f1e81fda700 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f1e7c0cdf60 0x7f1e7c0ce340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.67:33358/0 (socket says 172.21.15.67:33358) 2023-10-14T10:36:54.976 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.970+0000 7f1e817d9700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f1e7c0ccc20 0x7f1e7c0cd080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3301/0 says I am v2:172.21.15.67:33632/0 (socket says 172.21.15.67:33632) 2023-10-14T10:36:54.976 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.970+0000 7f1e81fda700 1 -- 172.21.15.67:0/1260433424 learned_addr learned my addr 172.21.15.67:0/1260433424 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:36:54.976 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.970+0000 7f1e817d9700 1 -- 172.21.15.67:0/1260433424 >> v2:172.21.15.72:3300/0 conn(0x7f1e7c0af140 msgr2=0x7f1e7c0af5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:54.976 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.970+0000 7f1e817d9700 1 --2- 172.21.15.67:0/1260433424 >> v2:172.21.15.72:3300/0 conn(0x7f1e7c0af140 0x7f1e7c0af5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:54.977 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.970+0000 7f1e817d9700 1 -- 172.21.15.67:0/1260433424 >> v1:172.21.15.72:6789/0 conn(0x7f1e7c0cdf60 legacy=0x7f1e7c0ce340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:54.977 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.970+0000 7f1e817d9700 1 -- 172.21.15.67:0/1260433424 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e7c0ae940 con 0x7f1e7c0ccc20 2023-10-14T10:36:54.977 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.971+0000 7f1e817d9700 1 --2- 172.21.15.67:0/1260433424 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f1e7c0ccc20 0x7f1e7c0cd080 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1e78009840 tx=0x7f1e78009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d0c11dc32a275b1 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:54.977 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.971+0000 7f1e80fd8700 1 -- 172.21.15.67:0/1260433424 <== mon.2 v2:172.21.15.72:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e78011830 con 0x7f1e7c0ccc20 2023-10-14T10:36:54.977 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.971+0000 7f1e80fd8700 1 -- 172.21.15.67:0/1260433424 <== mon.2 v2:172.21.15.72:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e78011990 con 0x7f1e7c0ccc20 2023-10-14T10:36:54.978 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.971+0000 7f1e80fd8700 1 -- 172.21.15.67:0/1260433424 <== mon.2 v2:172.21.15.72:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e78011d20 con 0x7f1e7c0ccc20 2023-10-14T10:36:54.978 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.971+0000 7f1e8423e700 1 -- 172.21.15.67:0/1260433424 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f1e7c0ccc20 msgr2=0x7f1e7c0cd080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:54.978 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.971+0000 7f1e8423e700 1 --2- 172.21.15.67:0/1260433424 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f1e7c0ccc20 0x7f1e7c0cd080 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1e78009840 tx=0x7f1e78009b50 comp rx=0 tx=0).stop 2023-10-14T10:36:54.978 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.971+0000 7f1e8423e700 1 -- 172.21.15.67:0/1260433424 shutdown_connections 2023-10-14T10:36:54.978 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.971+0000 7f1e8423e700 1 --2- 172.21.15.67:0/1260433424 >> v2:172.21.15.72:3300/0 conn(0x7f1e7c0af140 0x7f1e7c0af5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:54.979 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.971+0000 7f1e8423e700 1 --2- 172.21.15.67:0/1260433424 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f1e7c0ccc20 0x7f1e7c0cd080 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:54.979 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.971+0000 7f1e8423e700 1 -- 172.21.15.67:0/1260433424 >> 172.21.15.67:0/1260433424 conn(0x7f1e7c0d1570 msgr2=0x7f1e7c0d1980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:54.979 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.971+0000 7f1e8423e700 1 -- 172.21.15.67:0/1260433424 shutdown_connections 2023-10-14T10:36:54.979 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.971+0000 7f1e8423e700 1 -- 172.21.15.67:0/1260433424 wait complete. 2023-10-14T10:36:54.982 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.973+0000 7f1e8423e700 1 Processor -- start 2023-10-14T10:36:54.982 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.973+0000 7f1e8423e700 1 -- start start 2023-10-14T10:36:54.983 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.973+0000 7f1e8423e700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e7c0af140 0x7f1e7c06b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:54.983 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.973+0000 7f1e8423e700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f1e7c0ccc20 0x7f1e7c153aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:54.983 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.973+0000 7f1e8423e700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1e7c0cdf60 0x7f1e7c159bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:54.983 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.973+0000 7f1e8423e700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e7c06c7d0 con 0x7f1e7c0af140 2023-10-14T10:36:54.983 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.973+0000 7f1e8423e700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e7c159860 con 0x7f1e7c0cdf60 2023-10-14T10:36:54.984 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.973+0000 7f1e8423e700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e7c1e3730 con 0x7f1e7c0ccc20 2023-10-14T10:36:54.984 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.974+0000 7f1e81fda700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e7c0af140 0x7f1e7c06b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:54.984 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.974+0000 7f1e81fda700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e7c0af140 0x7f1e7c06b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45588/0 (socket says 172.21.15.67:45588) 2023-10-14T10:36:54.984 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.974+0000 7f1e81fda700 1 -- 172.21.15.67:0/4108731063 learned_addr learned my addr 172.21.15.67:0/4108731063 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:54.984 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.974+0000 7f1e827db700 1 --2- 172.21.15.67:0/4108731063 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1e7c0cdf60 0x7f1e7c159bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:54.985 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.974+0000 7f1e817d9700 1 --2- 172.21.15.67:0/4108731063 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f1e7c0ccc20 0x7f1e7c153aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:54.985 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.974+0000 7f1e81fda700 1 -- 172.21.15.67:0/4108731063 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f1e7c0ccc20 msgr2=0x7f1e7c153aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:54.985 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.974+0000 7f1e81fda700 1 --2- 172.21.15.67:0/4108731063 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f1e7c0ccc20 0x7f1e7c153aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:54.985 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.974+0000 7f1e81fda700 1 -- 172.21.15.67:0/4108731063 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1e7c0cdf60 msgr2=0x7f1e7c159bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:54.985 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.974+0000 7f1e81fda700 1 --2- 172.21.15.67:0/4108731063 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1e7c0cdf60 0x7f1e7c159bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:54.986 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.974+0000 7f1e81fda700 1 -- 172.21.15.67:0/4108731063 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e780094a0 con 0x7f1e7c0af140 2023-10-14T10:36:54.986 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.974+0000 7f1e81fda700 1 --2- 172.21.15.67:0/4108731063 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e7c0af140 0x7f1e7c06b950 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c00c480 tx=0x7f1e6c00c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:54.986 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.974+0000 7f1e727fc700 1 -- 172.21.15.67:0/4108731063 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e6c005d70 con 0x7f1e7c0af140 2023-10-14T10:36:54.986 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.974+0000 7f1e727fc700 1 -- 172.21.15.67:0/4108731063 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e6c00ce90 con 0x7f1e7c0af140 2023-10-14T10:36:54.986 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.974+0000 7f1e8423e700 1 -- 172.21.15.67:0/4108731063 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e7c1e3400 con 0x7f1e7c0af140 2023-10-14T10:36:54.987 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.974+0000 7f1e727fc700 1 -- 172.21.15.67:0/4108731063 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e6c0072d0 con 0x7f1e7c0af140 2023-10-14T10:36:54.987 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.974+0000 7f1e8423e700 1 -- 172.21.15.67:0/4108731063 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e7c06bf30 con 0x7f1e7c0af140 2023-10-14T10:36:54.987 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.975+0000 7f1e8423e700 1 -- 172.21.15.67:0/4108731063 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e7c04ebb0 con 0x7f1e7c0af140 2023-10-14T10:36:54.987 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.976+0000 7f1e727fc700 1 -- 172.21.15.67:0/4108731063 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f1e6c007430 con 0x7f1e7c0af140 2023-10-14T10:36:54.987 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.976+0000 7f1e727fc700 1 --2- 172.21.15.67:0/4108731063 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f1e6006af00 0x7f1e6006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:54.988 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.976+0000 7f1e727fc700 1 -- 172.21.15.67:0/4108731063 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1e6c088490 con 0x7f1e7c0af140 2023-10-14T10:36:54.988 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.977+0000 7f1e817d9700 1 --2- 172.21.15.67:0/4108731063 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f1e6006af00 0x7f1e6006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:54.988 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.977+0000 7f1e817d9700 1 --2- 172.21.15.67:0/4108731063 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f1e6006af00 0x7f1e6006d3c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1e78000c00 tx=0x7f1e78008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:54.988 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:54.981+0000 7f1e727fc700 1 -- 172.21.15.67:0/4108731063 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1e6c053940 con 0x7f1e7c0af140 2023-10-14T10:36:55.267 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.265+0000 7f1e8423e700 1 -- 172.21.15.67:0/4108731063 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f1e7c06fa40 con 0x7f1e7c0af140 2023-10-14T10:36:55.269 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.267+0000 7f1e727fc700 1 -- 172.21.15.67:0/4108731063 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7f1e6c056f60 con 0x7f1e7c0af140 2023-10-14T10:36:55.272 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.269+0000 7f1e8423e700 1 -- 172.21.15.67:0/4108731063 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f1e6006af00 msgr2=0x7f1e6006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:55.272 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.269+0000 7f1e8423e700 1 --2- 172.21.15.67:0/4108731063 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f1e6006af00 0x7f1e6006d3c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1e78000c00 tx=0x7f1e78008030 comp rx=0 tx=0).stop 2023-10-14T10:36:55.272 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.269+0000 7f1e8423e700 1 -- 172.21.15.67:0/4108731063 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e7c0af140 msgr2=0x7f1e7c06b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:55.272 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.269+0000 7f1e8423e700 1 --2- 172.21.15.67:0/4108731063 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e7c0af140 0x7f1e7c06b950 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c00c480 tx=0x7f1e6c00c790 comp rx=0 tx=0).stop 2023-10-14T10:36:55.273 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.269+0000 7f1e8423e700 1 -- 172.21.15.67:0/4108731063 shutdown_connections 2023-10-14T10:36:55.273 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.269+0000 7f1e8423e700 1 --2- 172.21.15.67:0/4108731063 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f1e6006af00 0x7f1e6006d3c0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:55.273 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.269+0000 7f1e8423e700 1 --2- 172.21.15.67:0/4108731063 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1e7c0cdf60 0x7f1e7c159bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:55.273 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.269+0000 7f1e8423e700 1 --2- 172.21.15.67:0/4108731063 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e7c0af140 0x7f1e7c06b950 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:55.273 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.269+0000 7f1e8423e700 1 --2- 172.21.15.67:0/4108731063 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f1e7c0ccc20 0x7f1e7c153aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:55.274 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.269+0000 7f1e8423e700 1 -- 172.21.15.67:0/4108731063 >> 172.21.15.67:0/4108731063 conn(0x7f1e7c0d1570 msgr2=0x7f1e7c1e2820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:55.274 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.269+0000 7f1e8423e700 1 -- 172.21.15.67:0/4108731063 shutdown_connections 2023-10-14T10:36:55.274 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.270+0000 7f1e8423e700 1 -- 172.21.15.67:0/4108731063 wait complete. 2023-10-14T10:36:55.284 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-10-14T10:36:55.410 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.408+0000 7fa05d6b2700 1 Processor -- start 2023-10-14T10:36:55.411 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.408+0000 7fa05d6b2700 1 -- start start 2023-10-14T10:36:55.414 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.409+0000 7fa05d6b2700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa05815dbc0 0x7fa05815dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:55.414 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.409+0000 7fa05d6b2700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fa0580d3fc0 0x7fa0580d1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:55.414 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.409+0000 7fa05d6b2700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa05815e4e0 con 0x7fa0580b7050 2023-10-14T10:36:55.414 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.409+0000 7fa05d6b2700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0580b7560 con 0x7fa05815dbc0 2023-10-14T10:36:55.415 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.409+0000 7fa05d6b2700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0580b7840 con 0x7fa0580d3fc0 2023-10-14T10:36:55.415 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.409+0000 7fa056ffd700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa05815dbc0 0x7fa05815dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:55.415 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.409+0000 7fa056ffd700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa05815dbc0 0x7fa05815dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45596/0 (socket says 172.21.15.67:45596) 2023-10-14T10:36:55.415 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.409+0000 7fa056ffd700 1 -- 172.21.15.67:0/1008455350 learned_addr learned my addr 172.21.15.67:0/1008455350 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:55.415 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.409+0000 7fa056ffd700 1 -- 172.21.15.67:0/1008455350 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fa0580d3fc0 msgr2=0x7fa0580d1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:55.416 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.409+0000 7fa0567fc700 1 --2- 172.21.15.67:0/1008455350 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fa0580d3fc0 0x7fa0580d1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:55.416 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.409+0000 7fa056ffd700 1 --2- 172.21.15.67:0/1008455350 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fa0580d3fc0 0x7fa0580d1ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:55.416 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.409+0000 7fa056ffd700 1 -- 172.21.15.67:0/1008455350 >> v1:172.21.15.72:6789/0 conn(0x7fa0580b7050 legacy=0x7fa0580d2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:55.416 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.409+0000 7fa056ffd700 1 -- 172.21.15.67:0/1008455350 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0580b7b20 con 0x7fa05815dbc0 2023-10-14T10:36:55.416 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.409+0000 7fa056ffd700 1 --2- 172.21.15.67:0/1008455350 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa05815dbc0 0x7fa05815dfa0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa040006fd0 tx=0x7fa040009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fe65f7e3b75c8da server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:55.417 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.410+0000 7fa055ffb700 1 -- 172.21.15.67:0/1008455350 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa040005e30 con 0x7fa05815dbc0 2023-10-14T10:36:55.417 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.410+0000 7fa055ffb700 1 -- 172.21.15.67:0/1008455350 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa040006030 con 0x7fa05815dbc0 2023-10-14T10:36:55.417 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.410+0000 7fa055ffb700 1 -- 172.21.15.67:0/1008455350 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0400063c0 con 0x7fa05815dbc0 2023-10-14T10:36:55.417 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.410+0000 7fa05d6b2700 1 -- 172.21.15.67:0/1008455350 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa05815dbc0 msgr2=0x7fa05815dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:55.417 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.410+0000 7fa05d6b2700 1 --2- 172.21.15.67:0/1008455350 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa05815dbc0 0x7fa05815dfa0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa040006fd0 tx=0x7fa040009bd0 comp rx=0 tx=0).stop 2023-10-14T10:36:55.418 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.410+0000 7fa05d6b2700 1 -- 172.21.15.67:0/1008455350 shutdown_connections 2023-10-14T10:36:55.418 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.410+0000 7fa05d6b2700 1 --2- 172.21.15.67:0/1008455350 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa05815dbc0 0x7fa05815dfa0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:55.418 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.410+0000 7fa05d6b2700 1 --2- 172.21.15.67:0/1008455350 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fa0580d3fc0 0x7fa0580d1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:55.418 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.410+0000 7fa05d6b2700 1 -- 172.21.15.67:0/1008455350 >> 172.21.15.67:0/1008455350 conn(0x7fa05809cb40 msgr2=0x7fa0580d3820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:55.418 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.410+0000 7fa05d6b2700 1 -- 172.21.15.67:0/1008455350 shutdown_connections 2023-10-14T10:36:55.419 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.410+0000 7fa05d6b2700 1 -- 172.21.15.67:0/1008455350 wait complete. 2023-10-14T10:36:55.419 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.412+0000 7fa05d6b2700 1 Processor -- start 2023-10-14T10:36:55.419 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.412+0000 7fa05d6b2700 1 -- start start 2023-10-14T10:36:55.422 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.412+0000 7fa05d6b2700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa0580b7050 0x7fa0581589a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:55.422 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.412+0000 7fa05d6b2700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fa0580d3fc0 0x7fa0581582a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:55.422 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.412+0000 7fa05d6b2700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa05815dbc0 0x7fa0581f6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:55.423 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.412+0000 7fa05d6b2700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0581587e0 con 0x7fa05815dbc0 2023-10-14T10:36:55.423 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.412+0000 7fa05d6b2700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0581f7500 con 0x7fa0580b7050 2023-10-14T10:36:55.423 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.412+0000 7fa05d6b2700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa058159030 con 0x7fa0580d3fc0 2023-10-14T10:36:55.423 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.412+0000 7fa0577fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa05815dbc0 0x7fa0581f6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:55.423 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.412+0000 7fa0577fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa05815dbc0 0x7fa0581f6fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45604/0 (socket says 172.21.15.67:45604) 2023-10-14T10:36:55.424 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.412+0000 7fa0577fe700 1 -- 172.21.15.67:0/2455972990 learned_addr learned my addr 172.21.15.67:0/2455972990 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:55.424 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.413+0000 7fa056ffd700 1 --2- 172.21.15.67:0/2455972990 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa0580b7050 0x7fa0581589a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:55.424 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.413+0000 7fa0567fc700 1 --2- 172.21.15.67:0/2455972990 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fa0580d3fc0 0x7fa0581582a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:55.424 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.413+0000 7fa0577fe700 1 -- 172.21.15.67:0/2455972990 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fa0580d3fc0 msgr2=0x7fa0581582a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:55.424 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.413+0000 7fa0577fe700 1 --2- 172.21.15.67:0/2455972990 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fa0580d3fc0 0x7fa0581582a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:55.425 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.413+0000 7fa0577fe700 1 -- 172.21.15.67:0/2455972990 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa0580b7050 msgr2=0x7fa0581589a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:55.425 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.413+0000 7fa0577fe700 1 --2- 172.21.15.67:0/2455972990 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa0580b7050 0x7fa0581589a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:55.425 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.413+0000 7fa0577fe700 1 -- 172.21.15.67:0/2455972990 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa040009500 con 0x7fa05815dbc0 2023-10-14T10:36:55.425 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.413+0000 7fa0577fe700 1 --2- 172.21.15.67:0/2455972990 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa05815dbc0 0x7fa0581f6fc0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fa05000cba0 tx=0x7fa05000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:55.426 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.413+0000 7fa04f7fe700 1 -- 172.21.15.67:0/2455972990 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa05000ab80 con 0x7fa05815dbc0 2023-10-14T10:36:55.426 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.413+0000 7fa04f7fe700 1 -- 172.21.15.67:0/2455972990 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa05000ace0 con 0x7fa05815dbc0 2023-10-14T10:36:55.426 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.413+0000 7fa04f7fe700 1 -- 172.21.15.67:0/2455972990 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa050012660 con 0x7fa05815dbc0 2023-10-14T10:36:55.426 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.413+0000 7fa05d6b2700 1 -- 172.21.15.67:0/2455972990 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa058154050 con 0x7fa05815dbc0 2023-10-14T10:36:55.426 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.413+0000 7fa05d6b2700 1 -- 172.21.15.67:0/2455972990 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa058154610 con 0x7fa05815dbc0 2023-10-14T10:36:55.427 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.414+0000 7fa05d6b2700 1 -- 172.21.15.67:0/2455972990 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa05804ebb0 con 0x7fa05815dbc0 2023-10-14T10:36:55.427 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.415+0000 7fa04f7fe700 1 -- 172.21.15.67:0/2455972990 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fa0500127c0 con 0x7fa05815dbc0 2023-10-14T10:36:55.427 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.415+0000 7fa04f7fe700 1 --2- 172.21.15.67:0/2455972990 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fa03806af00 0x7fa03806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:55.427 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.415+0000 7fa04f7fe700 1 -- 172.21.15.67:0/2455972990 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa050089a30 con 0x7fa05815dbc0 2023-10-14T10:36:55.428 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.415+0000 7fa056ffd700 1 --2- 172.21.15.67:0/2455972990 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fa03806af00 0x7fa03806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:55.428 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.416+0000 7fa056ffd700 1 --2- 172.21.15.67:0/2455972990 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fa03806af00 0x7fa03806d3c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa040013040 tx=0x7fa040008070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:55.428 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.419+0000 7fa04f7fe700 1 -- 172.21.15.67:0/2455972990 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa050054e30 con 0x7fa05815dbc0 2023-10-14T10:36:55.703 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.701+0000 7fa05d6b2700 1 -- 172.21.15.67:0/2455972990 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7fa0580c6ca0 con 0x7fa05815dbc0 2023-10-14T10:36:55.705 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.704+0000 7fa04f7fe700 1 -- 172.21.15.67:0/2455972990 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v11) v1 ==== 97+0+0 (secure 0 0 0) 0x7fa050016070 con 0x7fa05815dbc0 2023-10-14T10:36:55.708 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.706+0000 7fa05d6b2700 1 -- 172.21.15.67:0/2455972990 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fa03806af00 msgr2=0x7fa03806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:55.709 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.706+0000 7fa05d6b2700 1 --2- 172.21.15.67:0/2455972990 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fa03806af00 0x7fa03806d3c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa040013040 tx=0x7fa040008070 comp rx=0 tx=0).stop 2023-10-14T10:36:55.709 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.706+0000 7fa05d6b2700 1 -- 172.21.15.67:0/2455972990 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa05815dbc0 msgr2=0x7fa0581f6fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:55.709 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.706+0000 7fa05d6b2700 1 --2- 172.21.15.67:0/2455972990 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa05815dbc0 0x7fa0581f6fc0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fa05000cba0 tx=0x7fa05000cf60 comp rx=0 tx=0).stop 2023-10-14T10:36:55.709 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.706+0000 7fa05d6b2700 1 -- 172.21.15.67:0/2455972990 shutdown_connections 2023-10-14T10:36:55.710 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.706+0000 7fa05d6b2700 1 --2- 172.21.15.67:0/2455972990 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fa03806af00 0x7fa03806d3c0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:55.710 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.706+0000 7fa05d6b2700 1 --2- 172.21.15.67:0/2455972990 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa05815dbc0 0x7fa0581f6fc0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:55.710 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.706+0000 7fa05d6b2700 1 --2- 172.21.15.67:0/2455972990 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa0580b7050 0x7fa0581589a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:55.710 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.706+0000 7fa05d6b2700 1 --2- 172.21.15.67:0/2455972990 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fa0580d3fc0 0x7fa0581582a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:55.711 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.706+0000 7fa05d6b2700 1 -- 172.21.15.67:0/2455972990 >> 172.21.15.67:0/2455972990 conn(0x7fa05809cb40 msgr2=0x7fa0580d0bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:55.711 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.706+0000 7fa05d6b2700 1 -- 172.21.15.67:0/2455972990 shutdown_connections 2023-10-14T10:36:55.711 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.706+0000 7fa05d6b2700 1 -- 172.21.15.67:0/2455972990 wait complete. 2023-10-14T10:36:55.721 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-10-14T10:36:55.847 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.845+0000 7f14575c4700 1 Processor -- start 2023-10-14T10:36:55.847 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.845+0000 7f14575c4700 1 -- start start 2023-10-14T10:36:55.851 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.846+0000 7f14575c4700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f14500cd4b0 0x7f14500cbc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:55.851 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.846+0000 7f14575c4700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f14500b2940 0x7f14500ce840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:55.852 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.846+0000 7f14575c4700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14500ce3b0 con 0x7f145015a850 2023-10-14T10:36:55.852 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.846+0000 7f14575c4700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14500b6290 con 0x7f14500b2940 2023-10-14T10:36:55.852 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.846+0000 7f14575c4700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14500cc220 con 0x7f14500cd4b0 2023-10-14T10:36:55.852 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.846+0000 7f1455360700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f145015a850 0x7f14500ccda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:53376/0 (socket says 172.21.15.67:53376) 2023-10-14T10:36:55.852 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.846+0000 7f1455360700 1 -- 172.21.15.67:0/1168273895 learned_addr learned my addr 172.21.15.67:0/1168273895 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:36:55.853 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.846+0000 7f1455b61700 1 --2- 172.21.15.67:0/1168273895 >> v2:172.21.15.72:3300/0 conn(0x7f14500b2940 0x7f14500ce840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:55.853 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.846+0000 7f1454b5f700 1 --2- 172.21.15.67:0/1168273895 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f14500cd4b0 0x7f14500cbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:55.853 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.846+0000 7f1447fff700 1 -- 172.21.15.67:0/1168273895 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 162650843 0 0) 0x7f14500ce3b0 con 0x7f145015a850 2023-10-14T10:36:55.853 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.846+0000 7f1447fff700 1 -- 172.21.15.67:0/1168273895 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f143c0035b0 con 0x7f145015a850 2023-10-14T10:36:55.854 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.846+0000 7f1447fff700 1 -- 172.21.15.67:0/1168273895 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4090909779 0 0) 0x7f143c0035b0 con 0x7f145015a850 2023-10-14T10:36:55.854 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.847+0000 7f1447fff700 1 -- 172.21.15.67:0/1168273895 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f14500cd4b0 msgr2=0x7f14500cbc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:55.854 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.847+0000 7f1447fff700 1 --2- 172.21.15.67:0/1168273895 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f14500cd4b0 0x7f14500cbc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:55.854 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.847+0000 7f1447fff700 1 -- 172.21.15.67:0/1168273895 >> v2:172.21.15.72:3300/0 conn(0x7f14500b2940 msgr2=0x7f14500ce840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:55.854 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.847+0000 7f1447fff700 1 --2- 172.21.15.67:0/1168273895 >> v2:172.21.15.72:3300/0 conn(0x7f14500b2940 0x7f14500ce840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:55.855 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.847+0000 7f1447fff700 1 -- 172.21.15.67:0/1168273895 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14500cc500 con 0x7f145015a850 2023-10-14T10:36:55.855 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.847+0000 7f1447fff700 1 -- 172.21.15.67:0/1168273895 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3180043755 0 0) 0x7f1440001670 con 0x7f145015a850 2023-10-14T10:36:55.855 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.847+0000 7f1447fff700 1 -- 172.21.15.67:0/1168273895 >> v1:172.21.15.67:6789/0 conn(0x7f145015a850 legacy=0x7f14500ccda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:55.855 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.847+0000 7f1447fff700 1 --2- 172.21.15.67:0/1168273895 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f14500cd4b0 0x7f143c003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:55.855 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.847+0000 7f1447fff700 1 --2- 172.21.15.67:0/1168273895 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f143c003f80 0x7f143c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:55.856 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.847+0000 7f1447fff700 1 --2- 172.21.15.67:0/1168273895 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f143c004d90 0x7f143c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:55.856 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.847+0000 7f1447fff700 1 -- 172.21.15.67:0/1168273895 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14500b6290 con 0x7f143c003f80 2023-10-14T10:36:55.856 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.847+0000 7f1447fff700 1 -- 172.21.15.67:0/1168273895 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14500cc220 con 0x7f143c004d90 2023-10-14T10:36:55.856 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.847+0000 7f1447fff700 1 -- 172.21.15.67:0/1168273895 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f143c00dc70 con 0x7f14500cd4b0 2023-10-14T10:36:55.856 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.847+0000 7f1454b5f700 1 --2- 172.21.15.67:0/1168273895 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f143c003f80 0x7f143c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:55.856 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.847+0000 7f1455360700 1 --2- 172.21.15.67:0/1168273895 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f14500cd4b0 0x7f143c003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:55.857 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.847+0000 7f1455b61700 1 --2- 172.21.15.67:0/1168273895 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f143c004d90 0x7f143c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:55.857 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.847+0000 7f1454b5f700 1 -- 172.21.15.67:0/1168273895 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f14500cd4b0 msgr2=0x7f143c003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:55.857 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.847+0000 7f1454b5f700 1 --2- 172.21.15.67:0/1168273895 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f14500cd4b0 0x7f143c003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:55.857 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.847+0000 7f1454b5f700 1 -- 172.21.15.67:0/1168273895 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f143c004d90 msgr2=0x7f143c005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:55.857 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.847+0000 7f1454b5f700 1 --2- 172.21.15.67:0/1168273895 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f143c004d90 0x7f143c005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:55.858 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.847+0000 7f1454b5f700 1 -- 172.21.15.67:0/1168273895 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f143c0035b0 con 0x7f143c003f80 2023-10-14T10:36:55.858 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.847+0000 7f1454b5f700 1 --2- 172.21.15.67:0/1168273895 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f143c003f80 0x7f143c004850 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f144c009b90 tx=0x7f143c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df6e639712e2c5f8 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:55.858 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.847+0000 7f1447fff700 1 -- 172.21.15.67:0/1168273895 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f144c00e6c0 con 0x7f143c003f80 2023-10-14T10:36:55.858 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.847+0000 7f1447fff700 1 -- 172.21.15.67:0/1168273895 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f144c00e820 con 0x7f143c003f80 2023-10-14T10:36:55.858 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.847+0000 7f1447fff700 1 -- 172.21.15.67:0/1168273895 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f144c00eb90 con 0x7f143c003f80 2023-10-14T10:36:55.859 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.848+0000 7f14575c4700 1 -- 172.21.15.67:0/1168273895 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f143c003f80 msgr2=0x7f143c004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:55.859 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.848+0000 7f14575c4700 1 --2- 172.21.15.67:0/1168273895 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f143c003f80 0x7f143c004850 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f144c009b90 tx=0x7f143c005880 comp rx=0 tx=0).stop 2023-10-14T10:36:55.859 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.848+0000 7f1455b61700 1 -- 172.21.15.67:0/1168273895 reap_dead start 2023-10-14T10:36:55.859 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.848+0000 7f14575c4700 1 -- 172.21.15.67:0/1168273895 shutdown_connections 2023-10-14T10:36:55.859 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.848+0000 7f14575c4700 1 -- 172.21.15.67:0/1168273895 >> 172.21.15.67:0/1168273895 conn(0x7f145006dac0 msgr2=0x7f145006dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:55.860 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.848+0000 7f14575c4700 1 -- 172.21.15.67:0/1168273895 shutdown_connections 2023-10-14T10:36:55.860 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.848+0000 7f14575c4700 1 -- 172.21.15.67:0/1168273895 wait complete. 2023-10-14T10:36:55.860 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.849+0000 7f14575c4700 1 Processor -- start 2023-10-14T10:36:55.860 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.849+0000 7f14575c4700 1 -- start start 2023-10-14T10:36:55.863 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.850+0000 7f14575c4700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f143c003f80 0x7f1450155c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:55.863 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.850+0000 7f14575c4700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f145015b390 0x7f145015b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:55.863 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.850+0000 7f14575c4700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14500b4c60 0x7f14500b50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:55.863 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.850+0000 7f14575c4700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f145015bd50 con 0x7f14500b4c60 2023-10-14T10:36:55.864 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.850+0000 7f14575c4700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14500c3fa0 con 0x7f145015b390 2023-10-14T10:36:55.864 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.850+0000 7f14575c4700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1450156190 con 0x7f143c003f80 2023-10-14T10:36:55.864 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.850+0000 7f1455b61700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14500b4c60 0x7f14500b50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:55.864 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.850+0000 7f1455b61700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14500b4c60 0x7f14500b50e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45624/0 (socket says 172.21.15.67:45624) 2023-10-14T10:36:55.864 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.850+0000 7f1455b61700 1 -- 172.21.15.67:0/140649037 learned_addr learned my addr 172.21.15.67:0/140649037 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:55.865 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.850+0000 7f1454b5f700 1 --2- 172.21.15.67:0/140649037 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f145015b390 0x7f145015b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:55.865 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.850+0000 7f1455360700 1 --2- 172.21.15.67:0/140649037 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f143c003f80 0x7f1450155c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:55.865 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.851+0000 7f1455b61700 1 -- 172.21.15.67:0/140649037 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f143c003f80 msgr2=0x7f1450155c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:55.865 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.851+0000 7f1455b61700 1 --2- 172.21.15.67:0/140649037 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f143c003f80 0x7f1450155c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:55.866 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.851+0000 7f1455b61700 1 -- 172.21.15.67:0/140649037 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f145015b390 msgr2=0x7f145015b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:55.866 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.851+0000 7f1455b61700 1 --2- 172.21.15.67:0/140649037 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f145015b390 0x7f145015b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:55.866 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.851+0000 7f1455b61700 1 -- 172.21.15.67:0/140649037 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14500addc0 con 0x7f14500b4c60 2023-10-14T10:36:55.866 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.851+0000 7f1455b61700 1 --2- 172.21.15.67:0/140649037 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14500b4c60 0x7f14500b50e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f1448011bf0 tx=0x7f1448011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:55.866 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.851+0000 7f1445ffb700 1 -- 172.21.15.67:0/140649037 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f144800e040 con 0x7f14500b4c60 2023-10-14T10:36:55.867 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.851+0000 7f1445ffb700 1 -- 172.21.15.67:0/140649037 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f144800f040 con 0x7f14500b4c60 2023-10-14T10:36:55.867 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.851+0000 7f1445ffb700 1 -- 172.21.15.67:0/140649037 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f144800fcd0 con 0x7f14500b4c60 2023-10-14T10:36:55.867 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.851+0000 7f14575c4700 1 -- 172.21.15.67:0/140649037 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14500b5a00 con 0x7f14500b4c60 2023-10-14T10:36:55.867 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.851+0000 7f14575c4700 1 -- 172.21.15.67:0/140649037 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14500ad770 con 0x7f14500b4c60 2023-10-14T10:36:55.867 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.852+0000 7f14575c4700 1 -- 172.21.15.67:0/140649037 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14500adf50 con 0x7f14500b4c60 2023-10-14T10:36:55.868 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.853+0000 7f1445ffb700 1 -- 172.21.15.67:0/140649037 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f1448022460 con 0x7f14500b4c60 2023-10-14T10:36:55.868 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.853+0000 7f1445ffb700 1 --2- 172.21.15.67:0/140649037 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f143406af30 0x7f143406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:55.868 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.853+0000 7f1445ffb700 1 -- 172.21.15.67:0/140649037 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1448089650 con 0x7f14500b4c60 2023-10-14T10:36:55.868 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.853+0000 7f1455360700 1 --2- 172.21.15.67:0/140649037 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f143406af30 0x7f143406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:55.868 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.854+0000 7f1455360700 1 --2- 172.21.15.67:0/140649037 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f143406af30 0x7f143406d3f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1440000c00 tx=0x7f1440007980 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:55.869 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:55.857+0000 7f1445ffb700 1 -- 172.21.15.67:0/140649037 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1448054a50 con 0x7f14500b4c60 2023-10-14T10:36:56.141 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.139+0000 7f14575c4700 1 -- 172.21.15.67:0/140649037 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f14500cd270 con 0x7f14500b4c60 2023-10-14T10:36:56.143 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.141+0000 7f1445ffb700 1 -- 172.21.15.67:0/140649037 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v12) v1 ==== 90+0+0 (secure 0 0 0) 0x7f1448058070 con 0x7f14500b4c60 2023-10-14T10:36:56.145 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.143+0000 7f14575c4700 1 -- 172.21.15.67:0/140649037 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f143406af30 msgr2=0x7f143406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:56.145 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.143+0000 7f14575c4700 1 --2- 172.21.15.67:0/140649037 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f143406af30 0x7f143406d3f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1440000c00 tx=0x7f1440007980 comp rx=0 tx=0).stop 2023-10-14T10:36:56.146 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.143+0000 7f14575c4700 1 -- 172.21.15.67:0/140649037 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14500b4c60 msgr2=0x7f14500b50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:56.146 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.143+0000 7f14575c4700 1 --2- 172.21.15.67:0/140649037 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14500b4c60 0x7f14500b50e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f1448011bf0 tx=0x7f1448011f00 comp rx=0 tx=0).stop 2023-10-14T10:36:56.146 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.143+0000 7f14575c4700 1 -- 172.21.15.67:0/140649037 shutdown_connections 2023-10-14T10:36:56.146 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.143+0000 7f14575c4700 1 --2- 172.21.15.67:0/140649037 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f143406af30 0x7f143406d3f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:56.146 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.143+0000 7f14575c4700 1 --2- 172.21.15.67:0/140649037 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14500b4c60 0x7f14500b50e0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:56.147 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.143+0000 7f14575c4700 1 --2- 172.21.15.67:0/140649037 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f143c003f80 0x7f1450155c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:56.147 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.143+0000 7f14575c4700 1 --2- 172.21.15.67:0/140649037 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f145015b390 0x7f145015b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:56.147 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.143+0000 7f14575c4700 1 -- 172.21.15.67:0/140649037 >> 172.21.15.67:0/140649037 conn(0x7f145006dac0 msgr2=0x7f1450158e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:56.147 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.143+0000 7f14575c4700 1 -- 172.21.15.67:0/140649037 shutdown_connections 2023-10-14T10:36:56.147 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.143+0000 7f14575c4700 1 -- 172.21.15.67:0/140649037 wait complete. 2023-10-14T10:36:56.157 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-14T10:36:56.282 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.280+0000 7f79c0e8a700 1 Processor -- start 2023-10-14T10:36:56.282 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.280+0000 7f79c0e8a700 1 -- start start 2023-10-14T10:36:56.286 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.281+0000 7f79c0e8a700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f79bc15eab0 0x7f79bc0cf190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:56.286 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.281+0000 7f79c0e8a700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79bc0d2170 con 0x7f79bc0cf6d0 2023-10-14T10:36:56.286 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.281+0000 7f79c0e8a700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79bc15ee90 con 0x7f79bc0d1d90 2023-10-14T10:36:56.286 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.281+0000 7f79c0e8a700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79bc0cfb60 con 0x7f79bc15eab0 2023-10-14T10:36:56.287 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.281+0000 7f79b9d9b700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f79bc0cf6d0 0x7f79bc0d1680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:53398/0 (socket says 172.21.15.67:53398) 2023-10-14T10:36:56.287 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.281+0000 7f79b9d9b700 1 -- 172.21.15.67:0/337300548 learned_addr learned my addr 172.21.15.67:0/337300548 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:36:56.287 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.281+0000 7f79ba59c700 1 --2- 172.21.15.67:0/337300548 >> v2:172.21.15.72:3300/0 conn(0x7f79bc15eab0 0x7f79bc0cf190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:56.287 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.281+0000 7f79b959a700 1 -- 172.21.15.67:0/337300548 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2465615347 0 0) 0x7f79ac001110 con 0x7f79bc0cf6d0 2023-10-14T10:36:56.287 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.281+0000 7f79b959a700 1 -- 172.21.15.67:0/337300548 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79a40035b0 con 0x7f79bc0cf6d0 2023-10-14T10:36:56.288 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.282+0000 7f79b959a700 1 -- 172.21.15.67:0/337300548 <== mon.1 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3753464079 0 0) 0x7f79bc15ee90 con 0x7f79bc0d1d90 2023-10-14T10:36:56.288 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.282+0000 7f79b959a700 1 -- 172.21.15.67:0/337300548 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79ac001110 con 0x7f79bc0d1d90 2023-10-14T10:36:56.288 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.282+0000 7f79b959a700 1 -- 172.21.15.67:0/337300548 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2289417315 0 0) 0x7f79a40035b0 con 0x7f79bc0cf6d0 2023-10-14T10:36:56.288 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.282+0000 7f79b959a700 1 -- 172.21.15.67:0/337300548 >> v2:172.21.15.72:3300/0 conn(0x7f79bc15eab0 msgr2=0x7f79bc0cf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:56.288 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.282+0000 7f79b959a700 1 --2- 172.21.15.67:0/337300548 >> v2:172.21.15.72:3300/0 conn(0x7f79bc15eab0 0x7f79bc0cf190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:56.289 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.282+0000 7f79b959a700 1 -- 172.21.15.67:0/337300548 >> v1:172.21.15.72:6789/0 conn(0x7f79bc0d1d90 legacy=0x7f79bc0cbc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:56.289 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.282+0000 7f79b959a700 1 -- 172.21.15.67:0/337300548 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79bc158370 con 0x7f79bc0cf6d0 2023-10-14T10:36:56.289 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.282+0000 7f79b959a700 1 -- 172.21.15.67:0/337300548 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3180043755 0 0) 0x7f79ac001800 con 0x7f79bc0cf6d0 2023-10-14T10:36:56.289 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.282+0000 7f79b959a700 1 -- 172.21.15.67:0/337300548 >> v1:172.21.15.67:6789/0 conn(0x7f79bc0cf6d0 legacy=0x7f79bc0d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:56.289 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.282+0000 7f79b959a700 1 --2- 172.21.15.67:0/337300548 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f79a4003be0 0x7f79a4003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:56.290 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.282+0000 7f79b959a700 1 --2- 172.21.15.67:0/337300548 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f79a4004850 0x7f79a4004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:56.290 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.282+0000 7f79b959a700 1 --2- 172.21.15.67:0/337300548 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f79a4005210 0x7f79a400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:56.290 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.282+0000 7f79b959a700 1 -- 172.21.15.67:0/337300548 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79bc0cfb60 con 0x7f79a4004850 2023-10-14T10:36:56.290 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.282+0000 7f79b959a700 1 -- 172.21.15.67:0/337300548 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79a400c0e0 con 0x7f79a4005210 2023-10-14T10:36:56.290 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.282+0000 7f79b959a700 1 -- 172.21.15.67:0/337300548 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79a400c2a0 con 0x7f79a4003be0 2023-10-14T10:36:56.291 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.282+0000 7f79b9d9b700 1 --2- 172.21.15.67:0/337300548 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f79a4004850 0x7f79a4004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:56.291 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.282+0000 7f79ba59c700 1 --2- 172.21.15.67:0/337300548 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f79a4003be0 0x7f79a4003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:56.291 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.282+0000 7f79bad9d700 1 --2- 172.21.15.67:0/337300548 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f79a4005210 0x7f79a400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:56.291 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.282+0000 7f79b9d9b700 1 -- 172.21.15.67:0/337300548 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f79a4003be0 msgr2=0x7f79a4003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:56.291 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.282+0000 7f79b9d9b700 1 --2- 172.21.15.67:0/337300548 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f79a4003be0 0x7f79a4003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:56.292 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.282+0000 7f79b9d9b700 1 -- 172.21.15.67:0/337300548 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f79a4005210 msgr2=0x7f79a400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:56.292 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.282+0000 7f79b9d9b700 1 --2- 172.21.15.67:0/337300548 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f79a4005210 0x7f79a400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:56.292 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.282+0000 7f79bad9d700 1 -- 172.21.15.67:0/337300548 reap_dead start 2023-10-14T10:36:56.292 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.282+0000 7f79b9d9b700 1 -- 172.21.15.67:0/337300548 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79a40035b0 con 0x7f79a4004850 2023-10-14T10:36:56.292 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.282+0000 7f79b9d9b700 1 --2- 172.21.15.67:0/337300548 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f79a4004850 0x7f79a4004cd0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f79ac00c110 tx=0x7f79a4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=558defd0dcc0a3c5 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:56.293 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.282+0000 7f79b959a700 1 -- 172.21.15.67:0/337300548 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79ac00cc50 con 0x7f79a4004850 2023-10-14T10:36:56.293 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.282+0000 7f79b959a700 1 -- 172.21.15.67:0/337300548 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79ac00cdd0 con 0x7f79a4004850 2023-10-14T10:36:56.293 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.283+0000 7f79b959a700 1 -- 172.21.15.67:0/337300548 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79ac005ba0 con 0x7f79a4004850 2023-10-14T10:36:56.293 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.283+0000 7f79c0e8a700 1 -- 172.21.15.67:0/337300548 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f79a4004850 msgr2=0x7f79a4004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:56.293 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.283+0000 7f79c0e8a700 1 --2- 172.21.15.67:0/337300548 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f79a4004850 0x7f79a4004cd0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f79ac00c110 tx=0x7f79a4005880 comp rx=0 tx=0).stop 2023-10-14T10:36:56.294 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.283+0000 7f79c0e8a700 1 -- 172.21.15.67:0/337300548 shutdown_connections 2023-10-14T10:36:56.294 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.283+0000 7f79c0e8a700 1 --2- 172.21.15.67:0/337300548 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f79a4004850 0x7f79a4004cd0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:56.294 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.283+0000 7f79c0e8a700 1 -- 172.21.15.67:0/337300548 >> 172.21.15.67:0/337300548 conn(0x7f79bc0665a0 msgr2=0x7f79bc066980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:56.294 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.283+0000 7f79c0e8a700 1 -- 172.21.15.67:0/337300548 shutdown_connections 2023-10-14T10:36:56.294 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.283+0000 7f79c0e8a700 1 -- 172.21.15.67:0/337300548 wait complete. 2023-10-14T10:36:56.297 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.284+0000 7f79c0e8a700 1 Processor -- start 2023-10-14T10:36:56.297 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.285+0000 7f79c0e8a700 1 -- start start 2023-10-14T10:36:56.298 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.285+0000 7f79c0e8a700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f79a4004850 0x7f79bc0c9b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:56.298 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.285+0000 7f79c0e8a700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f79bc1f3b80 0x7f79bc157990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:56.298 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.285+0000 7f79c0e8a700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f79bc0c73f0 0x7f79bc0c78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:56.298 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.285+0000 7f79c0e8a700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79bc0c7de0 con 0x7f79bc0c73f0 2023-10-14T10:36:56.298 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.285+0000 7f79c0e8a700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79bc0cadf0 con 0x7f79bc1f3b80 2023-10-14T10:36:56.299 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.285+0000 7f79c0e8a700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79bc1f4000 con 0x7f79a4004850 2023-10-14T10:36:56.299 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.285+0000 7f79bad9d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f79bc0c73f0 0x7f79bc0c78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:56.299 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.285+0000 7f79bad9d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f79bc0c73f0 0x7f79bc0c78a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45644/0 (socket says 172.21.15.67:45644) 2023-10-14T10:36:56.299 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.285+0000 7f79bad9d700 1 -- 172.21.15.67:0/1534311228 learned_addr learned my addr 172.21.15.67:0/1534311228 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:56.299 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.286+0000 7f79b9d9b700 1 --2- 172.21.15.67:0/1534311228 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f79bc1f3b80 0x7f79bc157990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:56.300 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.286+0000 7f79ba59c700 1 --2- 172.21.15.67:0/1534311228 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f79a4004850 0x7f79bc0c9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:56.300 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.286+0000 7f79bad9d700 1 -- 172.21.15.67:0/1534311228 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f79a4004850 msgr2=0x7f79bc0c9b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:56.300 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.286+0000 7f79bad9d700 1 --2- 172.21.15.67:0/1534311228 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f79a4004850 0x7f79bc0c9b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:56.300 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.286+0000 7f79bad9d700 1 -- 172.21.15.67:0/1534311228 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f79bc1f3b80 msgr2=0x7f79bc157990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:56.300 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.286+0000 7f79bad9d700 1 --2- 172.21.15.67:0/1534311228 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f79bc1f3b80 0x7f79bc157990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:56.300 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.286+0000 7f79bad9d700 1 -- 172.21.15.67:0/1534311228 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79bc0ca1e0 con 0x7f79bc0c73f0 2023-10-14T10:36:56.301 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.286+0000 7f79bad9d700 1 --2- 172.21.15.67:0/1534311228 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f79bc0c73f0 0x7f79bc0c78a0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f79bc0b3280 tx=0x7f79b400d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:56.301 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.286+0000 7f79aaffd700 1 -- 172.21.15.67:0/1534311228 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b4003cd0 con 0x7f79bc0c73f0 2023-10-14T10:36:56.301 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.286+0000 7f79aaffd700 1 -- 172.21.15.67:0/1534311228 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79b4003e30 con 0x7f79bc0c73f0 2023-10-14T10:36:56.301 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.286+0000 7f79c0e8a700 1 -- 172.21.15.67:0/1534311228 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79bc0ca4c0 con 0x7f79bc0c73f0 2023-10-14T10:36:56.301 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.286+0000 7f79aaffd700 1 -- 172.21.15.67:0/1534311228 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b400db70 con 0x7f79bc0c73f0 2023-10-14T10:36:56.302 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.286+0000 7f79c0e8a700 1 -- 172.21.15.67:0/1534311228 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79bc0caa80 con 0x7f79bc0c73f0 2023-10-14T10:36:56.302 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.287+0000 7f79c0e8a700 1 -- 172.21.15.67:0/1534311228 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79bc0cac40 con 0x7f79bc0c73f0 2023-10-14T10:36:56.302 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.288+0000 7f79aaffd700 1 -- 172.21.15.67:0/1534311228 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f79b400dcd0 con 0x7f79bc0c73f0 2023-10-14T10:36:56.302 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.288+0000 7f79aaffd700 1 --2- 172.21.15.67:0/1534311228 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f799c06af00 0x7f799c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:56.303 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.288+0000 7f79aaffd700 1 -- 172.21.15.67:0/1534311228 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f79b4089640 con 0x7f79bc0c73f0 2023-10-14T10:36:56.303 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.288+0000 7f79ba59c700 1 --2- 172.21.15.67:0/1534311228 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f799c06af00 0x7f799c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:56.303 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.289+0000 7f79ba59c700 1 --2- 172.21.15.67:0/1534311228 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f799c06af00 0x7f799c06d3c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f79bc0b5570 tx=0x7f79b0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:56.303 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.292+0000 7f79aaffd700 1 -- 172.21.15.67:0/1534311228 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f79b4054a40 con 0x7f79bc0c73f0 2023-10-14T10:36:56.497 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.494+0000 7f79c0e8a700 1 -- 172.21.15.67:0/1534311228 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f79bc0c5050 con 0x7f79bc0c73f0 2023-10-14T10:36:56.497 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.495+0000 7f79aaffd700 1 -- 172.21.15.67:0/1534311228 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+12809 (secure 0 0 0) 0x7f79bc0c5050 con 0x7f79bc0c73f0 2023-10-14T10:36:56.498 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-14T10:36:56.498 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":29,"fsid":"37f9ab46-3082-4b71-8b24-08d39897b008","created":"2023-10-14T10:35:46.067554+0000","modified":"2023-10-14T10:36:53.046309+0000","last_up_change":"2023-10-14T10:35:56.668142+0000","last_in_change":"2023-10-14T10:35:48.551490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-14T10:35:57.809566+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2b59a07e-11ef-440c-b766-9f64ac885838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6801","nonce":1287255499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6803","nonce":1287255499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6807","nonce":1287255499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6805","nonce":1287255499}]},"public_addr":"172.21.15.67:6801/1287255499","cluster_addr":"172.21.15.67:6803/1287255499","heartbeat_back_addr":"172.21.15.67:6807/1287255499","heartbeat_front_addr":"172.21.15.67:6805/1287255499","state":["exists","up"]},{"osd":1,"uuid":"6f813bd6-fa0b-487d-a204-431eb1276a60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6818","nonce":2435538787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6822","nonce":2435538787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6830","nonce":2435538787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6826","nonce":2435538787}]},"public_addr":"172.21.15.67:6818/2435538787","cluster_addr":"172.21.15.67:6822/2435538787","heartbeat_back_addr":"172.21.15.67:6830/2435538787","heartbeat_front_addr":"172.21.15.67:6826/2435538787","state":["exists","up"]},{"osd":2,"uuid":"6af42c6a-1b59-4669-9c04-b6876788446d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6809","nonce":2827491791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6811","nonce":2827491791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6815","nonce":2827491791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6813","nonce":2827491791}]},"public_addr":"172.21.15.67:6809/2827491791","cluster_addr":"172.21.15.67:6811/2827491791","heartbeat_back_addr":"172.21.15.67:6815/2827491791","heartbeat_front_addr":"172.21.15.67:6813/2827491791","state":["exists","up"]},{"osd":3,"uuid":"1d889d76-a646-499d-bfff-ed5bae5434ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6819","nonce":1848001176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6821","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6823","nonce":1848001176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6831","nonce":1848001176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6825","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6827","nonce":1848001176}]},"public_addr":"172.21.15.67:6819/1848001176","cluster_addr":"172.21.15.67:6823/1848001176","heartbeat_back_addr":"172.21.15.67:6831/1848001176","heartbeat_front_addr":"172.21.15.67:6827/1848001176","state":["exists","up"]},{"osd":4,"uuid":"c3271dff-b437-4fdd-b13b-6e49a35473af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6804","nonce":1891230785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6808","nonce":1891230785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6815","nonce":1891230785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6811","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6813","nonce":1891230785}]},"public_addr":"172.21.15.72:6804/1891230785","cluster_addr":"172.21.15.72:6808/1891230785","heartbeat_back_addr":"172.21.15.72:6815/1891230785","heartbeat_front_addr":"172.21.15.72:6813/1891230785","state":["exists","up"]},{"osd":5,"uuid":"0337bb18-4b68-4eff-b528-a667275e6926","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6801","nonce":569686991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6803","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6805","nonce":569686991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6812","nonce":569686991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6807","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6809","nonce":569686991}]},"public_addr":"172.21.15.72:6801/569686991","cluster_addr":"172.21.15.72:6805/569686991","heartbeat_back_addr":"172.21.15.72:6812/569686991","heartbeat_front_addr":"172.21.15.72:6809/569686991","state":["exists","up"]},{"osd":6,"uuid":"8e92d9cf-0c8c-46eb-89a7-db2023ef1509","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6817","nonce":3767685934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6819","nonce":3767685934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6823","nonce":3767685934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6821","nonce":3767685934}]},"public_addr":"172.21.15.72:6817/3767685934","cluster_addr":"172.21.15.72:6819/3767685934","heartbeat_back_addr":"172.21.15.72:6823/3767685934","heartbeat_front_addr":"172.21.15.72:6821/3767685934","state":["exists","up"]},{"osd":7,"uuid":"3fd6aa7d-50a9-45c9-9561-5902a044a119","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6825","nonce":3567465926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6827","nonce":3567465926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6831","nonce":3567465926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6829","nonce":3567465926}]},"public_addr":"172.21.15.72:6825/3567465926","cluster_addr":"172.21.15.72:6827/3567465926","heartbeat_back_addr":"172.21.15.72:6831/3567465926","heartbeat_front_addr":"172.21.15.72:6829/3567465926","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.511466+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.795107+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.858674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.728895+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.794842+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.750064+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.732122+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.765185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-14T10:36:56.500 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.497+0000 7f79c0e8a700 1 -- 172.21.15.67:0/1534311228 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f799c06af00 msgr2=0x7f799c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:56.500 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.497+0000 7f79c0e8a700 1 --2- 172.21.15.67:0/1534311228 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f799c06af00 0x7f799c06d3c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f79bc0b5570 tx=0x7f79b0008040 comp rx=0 tx=0).stop 2023-10-14T10:36:56.500 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.497+0000 7f79c0e8a700 1 -- 172.21.15.67:0/1534311228 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f79bc0c73f0 msgr2=0x7f79bc0c78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:56.500 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.497+0000 7f79c0e8a700 1 --2- 172.21.15.67:0/1534311228 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f79bc0c73f0 0x7f79bc0c78a0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f79bc0b3280 tx=0x7f79b400d430 comp rx=0 tx=0).stop 2023-10-14T10:36:56.500 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.497+0000 7f79c0e8a700 1 -- 172.21.15.67:0/1534311228 shutdown_connections 2023-10-14T10:36:56.501 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.497+0000 7f79c0e8a700 1 --2- 172.21.15.67:0/1534311228 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f799c06af00 0x7f799c06d3c0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:56.501 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.497+0000 7f79c0e8a700 1 --2- 172.21.15.67:0/1534311228 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f79bc0c73f0 0x7f79bc0c78a0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:56.501 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.497+0000 7f79c0e8a700 1 --2- 172.21.15.67:0/1534311228 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f79a4004850 0x7f79bc0c9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:56.501 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.497+0000 7f79c0e8a700 1 --2- 172.21.15.67:0/1534311228 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f79bc1f3b80 0x7f79bc157990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:56.502 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.497+0000 7f79c0e8a700 1 -- 172.21.15.67:0/1534311228 >> 172.21.15.67:0/1534311228 conn(0x7f79bc0665a0 msgr2=0x7f79bc0b41c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:56.502 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.497+0000 7f79c0e8a700 1 -- 172.21.15.67:0/1534311228 shutdown_connections 2023-10-14T10:36:56.502 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.497+0000 7f79c0e8a700 1 -- 172.21.15.67:0/1534311228 wait complete. 2023-10-14T10:36:56.512 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-14T10:35:57.809566+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-14T10:36:56.512 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-14T10:36:56.640 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.638+0000 7f84a9f31700 1 Processor -- start 2023-10-14T10:36:56.641 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.638+0000 7f84a9f31700 1 -- start start 2023-10-14T10:36:56.643 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.639+0000 7f84a9f31700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f84a40cd950 0x7f84a415e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:56.643 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.639+0000 7f84a9f31700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f84a40cede0 0x7f84a415f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:56.644 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.639+0000 7f84a9f31700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84a40cf260 con 0x7f84a406a840 2023-10-14T10:36:56.644 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.639+0000 7f84a9f31700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84a41619a0 con 0x7f84a40cd950 2023-10-14T10:36:56.644 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.639+0000 7f84a9f31700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84a40b2d60 con 0x7f84a40cede0 2023-10-14T10:36:56.644 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.639+0000 7f84a37fe700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f84a40cd950 0x7f84a415e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:56.644 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.639+0000 7f84a37fe700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f84a40cd950 0x7f84a415e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45648/0 (socket says 172.21.15.67:45648) 2023-10-14T10:36:56.645 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.639+0000 7f84a3fff700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f84a406a840 0x7f84a4161df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:53420/0 (socket says 172.21.15.67:53420) 2023-10-14T10:36:56.645 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.639+0000 7f84a37fe700 1 -- 172.21.15.67:0/807657003 learned_addr learned my addr 172.21.15.67:0/807657003 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:56.645 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.639+0000 7f84a2ffd700 1 --2- 172.21.15.67:0/807657003 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f84a40cede0 0x7f84a415f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:56.645 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.639+0000 7f84a37fe700 1 -- 172.21.15.67:0/807657003 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f84a40cede0 msgr2=0x7f84a415f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:56.645 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.639+0000 7f84a37fe700 1 --2- 172.21.15.67:0/807657003 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f84a40cede0 0x7f84a415f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:56.646 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.639+0000 7f84a37fe700 1 -- 172.21.15.67:0/807657003 >> v1:172.21.15.67:6789/0 conn(0x7f84a406a840 legacy=0x7f84a4161df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:56.646 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.639+0000 7f84a37fe700 1 -- 172.21.15.67:0/807657003 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84a40cdd30 con 0x7f84a40cd950 2023-10-14T10:36:56.646 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.639+0000 7f84a27fc700 1 -- 172.21.15.67:0/807657003 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1680417651 0 0) 0x7f84a40cf260 con 0x7f84a406a840 2023-10-14T10:36:56.646 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.640+0000 7f84a37fe700 1 --2- 172.21.15.67:0/807657003 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84a40cd950 0x7f84a415e2f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f848c0098f0 tx=0x7f848c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36be290021f39573 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:56.646 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.640+0000 7f84a27fc700 1 -- 172.21.15.67:0/807657003 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f848c011830 con 0x7f84a40cd950 2023-10-14T10:36:56.647 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.640+0000 7f84a27fc700 1 -- 172.21.15.67:0/807657003 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f848c011990 con 0x7f84a40cd950 2023-10-14T10:36:56.647 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.640+0000 7f84a27fc700 1 -- 172.21.15.67:0/807657003 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f848c011d20 con 0x7f84a40cd950 2023-10-14T10:36:56.647 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.640+0000 7f84a9f31700 1 -- 172.21.15.67:0/807657003 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84a40cd950 msgr2=0x7f84a415e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:56.647 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.640+0000 7f84a9f31700 1 --2- 172.21.15.67:0/807657003 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84a40cd950 0x7f84a415e2f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f848c0098f0 tx=0x7f848c009c00 comp rx=0 tx=0).stop 2023-10-14T10:36:56.647 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.640+0000 7f84a9f31700 1 -- 172.21.15.67:0/807657003 shutdown_connections 2023-10-14T10:36:56.648 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.640+0000 7f84a9f31700 1 --2- 172.21.15.67:0/807657003 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84a40cd950 0x7f84a415e2f0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:56.648 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.640+0000 7f84a9f31700 1 --2- 172.21.15.67:0/807657003 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f84a40cede0 0x7f84a415f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:56.648 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.640+0000 7f84a9f31700 1 -- 172.21.15.67:0/807657003 >> 172.21.15.67:0/807657003 conn(0x7f84a406dae0 msgr2=0x7f84a406def0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:56.648 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.640+0000 7f84a9f31700 1 -- 172.21.15.67:0/807657003 shutdown_connections 2023-10-14T10:36:56.648 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.640+0000 7f84a9f31700 1 -- 172.21.15.67:0/807657003 wait complete. 2023-10-14T10:36:56.651 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.642+0000 7f84a9f31700 1 Processor -- start 2023-10-14T10:36:56.651 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.642+0000 7f84a9f31700 1 -- start start 2023-10-14T10:36:56.652 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.643+0000 7f84a9f31700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f84a406a840 0x7f84a4154440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:56.652 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.643+0000 7f84a9f31700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f84a40cd950 0x7f84a41f34c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:56.652 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.643+0000 7f84a9f31700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84a40cede0 0x7f84a40cb880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:56.652 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.643+0000 7f84a9f31700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84a415b220 con 0x7f84a40cede0 2023-10-14T10:36:56.652 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.643+0000 7f84a9f31700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84a41f3a00 con 0x7f84a40cd950 2023-10-14T10:36:56.653 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.643+0000 7f84a9f31700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84a40cbdc0 con 0x7f84a406a840 2023-10-14T10:36:56.653 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.643+0000 7f84a3fff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84a40cede0 0x7f84a40cb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:56.653 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.643+0000 7f84a3fff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84a40cede0 0x7f84a40cb880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45658/0 (socket says 172.21.15.67:45658) 2023-10-14T10:36:56.653 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.643+0000 7f84a3fff700 1 -- 172.21.15.67:0/1009310739 learned_addr learned my addr 172.21.15.67:0/1009310739 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:56.653 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.643+0000 7f84a2ffd700 1 --2- 172.21.15.67:0/1009310739 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f84a40cd950 0x7f84a41f34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:56.654 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.643+0000 7f84a37fe700 1 --2- 172.21.15.67:0/1009310739 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f84a406a840 0x7f84a4154440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:56.654 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.644+0000 7f84a3fff700 1 -- 172.21.15.67:0/1009310739 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f84a406a840 msgr2=0x7f84a4154440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:56.654 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.644+0000 7f84a3fff700 1 --2- 172.21.15.67:0/1009310739 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f84a406a840 0x7f84a4154440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:56.654 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.644+0000 7f84a3fff700 1 -- 172.21.15.67:0/1009310739 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f84a40cd950 msgr2=0x7f84a41f34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:56.654 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.644+0000 7f84a3fff700 1 --2- 172.21.15.67:0/1009310739 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f84a40cd950 0x7f84a41f34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:56.655 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.644+0000 7f84a3fff700 1 -- 172.21.15.67:0/1009310739 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f848c009550 con 0x7f84a40cede0 2023-10-14T10:36:56.655 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.644+0000 7f84a3fff700 1 --2- 172.21.15.67:0/1009310739 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84a40cede0 0x7f84a40cb880 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f849c002230 tx=0x7f849c00cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:56.655 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.644+0000 7f849bfff700 1 -- 172.21.15.67:0/1009310739 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f849c00aac0 con 0x7f84a40cede0 2023-10-14T10:36:56.655 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.644+0000 7f849bfff700 1 -- 172.21.15.67:0/1009310739 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f849c00ac40 con 0x7f84a40cede0 2023-10-14T10:36:56.655 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.644+0000 7f84a9f31700 1 -- 172.21.15.67:0/1009310739 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84a40cc060 con 0x7f84a40cede0 2023-10-14T10:36:56.656 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.644+0000 7f849bfff700 1 -- 172.21.15.67:0/1009310739 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f849c0070d0 con 0x7f84a40cede0 2023-10-14T10:36:56.656 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.644+0000 7f84a9f31700 1 -- 172.21.15.67:0/1009310739 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84a40cc620 con 0x7f84a40cede0 2023-10-14T10:36:56.656 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.645+0000 7f84a9f31700 1 -- 172.21.15.67:0/1009310739 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84a4067d20 con 0x7f84a40cede0 2023-10-14T10:36:56.656 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.645+0000 7f849bfff700 1 -- 172.21.15.67:0/1009310739 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f849c007250 con 0x7f84a40cede0 2023-10-14T10:36:56.656 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.646+0000 7f849bfff700 1 --2- 172.21.15.67:0/1009310739 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f848406af00 0x7f848406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:56.656 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.646+0000 7f849bfff700 1 -- 172.21.15.67:0/1009310739 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f849c0890d0 con 0x7f84a40cede0 2023-10-14T10:36:56.657 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.646+0000 7f84a37fe700 1 --2- 172.21.15.67:0/1009310739 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f848406af00 0x7f848406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:56.657 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.646+0000 7f84a37fe700 1 --2- 172.21.15.67:0/1009310739 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f848406af00 0x7f848406d3c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f848c009c40 tx=0x7f848c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:56.657 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.650+0000 7f849bfff700 1 -- 172.21.15.67:0/1009310739 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f849c054440 con 0x7f84a40cede0 2023-10-14T10:36:56.852 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.850+0000 7f84a9f31700 1 -- 172.21.15.67:0/1009310739 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f84a40b2630 con 0x7f84a40cede0 2023-10-14T10:36:56.852 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.850+0000 7f849bfff700 1 -- 172.21.15.67:0/1009310739 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7f849c0180a0 con 0x7f84a40cede0 2023-10-14T10:36:56.853 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2023-10-14T10:36:56.855 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.852+0000 7f84a9f31700 1 -- 172.21.15.67:0/1009310739 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f848406af00 msgr2=0x7f848406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:56.855 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.852+0000 7f84a9f31700 1 --2- 172.21.15.67:0/1009310739 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f848406af00 0x7f848406d3c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f848c009c40 tx=0x7f848c008030 comp rx=0 tx=0).stop 2023-10-14T10:36:56.855 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.852+0000 7f84a9f31700 1 -- 172.21.15.67:0/1009310739 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84a40cede0 msgr2=0x7f84a40cb880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:56.855 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.852+0000 7f84a9f31700 1 --2- 172.21.15.67:0/1009310739 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84a40cede0 0x7f84a40cb880 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f849c002230 tx=0x7f849c00cf50 comp rx=0 tx=0).stop 2023-10-14T10:36:56.855 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.852+0000 7f84a9f31700 1 -- 172.21.15.67:0/1009310739 shutdown_connections 2023-10-14T10:36:56.856 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.852+0000 7f84a9f31700 1 --2- 172.21.15.67:0/1009310739 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f848406af00 0x7f848406d3c0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:56.856 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.852+0000 7f84a9f31700 1 --2- 172.21.15.67:0/1009310739 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84a40cede0 0x7f84a40cb880 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:56.856 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.852+0000 7f84a9f31700 1 --2- 172.21.15.67:0/1009310739 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f84a406a840 0x7f84a4154440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:56.856 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.852+0000 7f84a9f31700 1 --2- 172.21.15.67:0/1009310739 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f84a40cd950 0x7f84a41f34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:56.856 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.852+0000 7f84a9f31700 1 -- 172.21.15.67:0/1009310739 >> 172.21.15.67:0/1009310739 conn(0x7f84a406dae0 msgr2=0x7f84a4156c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:56.857 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.852+0000 7f84a9f31700 1 -- 172.21.15.67:0/1009310739 shutdown_connections 2023-10-14T10:36:56.857 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.852+0000 7f84a9f31700 1 -- 172.21.15.67:0/1009310739 wait complete. 2023-10-14T10:36:56.867 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-14T10:36:56.991 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.989+0000 7faeefeef700 1 Processor -- start 2023-10-14T10:36:56.992 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.989+0000 7faeefeef700 1 -- start start 2023-10-14T10:36:56.994 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.990+0000 7faeefeef700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7faee815e2f0 0x7faee80cede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:56.994 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.990+0000 7faeefeef700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7faee806a840 0x7faee806acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:56.994 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.990+0000 7faeefeef700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faee80b2d60 con 0x7faee80cd950 2023-10-14T10:36:56.995 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.990+0000 7faeefeef700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faee81619a0 con 0x7faee815e2f0 2023-10-14T10:36:56.995 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.990+0000 7faeefeef700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faee806b1f0 con 0x7faee806a840 2023-10-14T10:36:56.995 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.991+0000 7faeed48a700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7faee815e2f0 0x7faee80cede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:56.995 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.991+0000 7faeee48c700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7faee806a840 0x7faee806acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:56.995 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.991+0000 7faeed48a700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7faee815e2f0 0x7faee80cede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.67:40454/0 (socket says 172.21.15.67:40454) 2023-10-14T10:36:56.996 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.991+0000 7faeedc8b700 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7faee80cd950 0x7faee815f570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.67:33438/0 (socket says 172.21.15.67:33438) 2023-10-14T10:36:56.996 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.991+0000 7faeed48a700 1 -- 172.21.15.67:0/3507813827 learned_addr learned my addr 172.21.15.67:0/3507813827 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:56.996 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.991+0000 7faeecc89700 1 -- 172.21.15.67:0/3507813827 <== mon.1 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 124131210 0 0) 0x7faee80b2d60 con 0x7faee80cd950 2023-10-14T10:36:56.996 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.991+0000 7faeee48c700 1 -- 172.21.15.67:0/3507813827 >> v2:172.21.15.72:3300/0 conn(0x7faee815e2f0 msgr2=0x7faee80cede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:56.996 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.991+0000 7faeee48c700 1 --2- 172.21.15.67:0/3507813827 >> v2:172.21.15.72:3300/0 conn(0x7faee815e2f0 0x7faee80cede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:56.997 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.991+0000 7faeee48c700 1 -- 172.21.15.67:0/3507813827 >> v1:172.21.15.72:6789/0 conn(0x7faee80cd950 legacy=0x7faee815f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:56.997 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.991+0000 7faeee48c700 1 -- 172.21.15.67:0/3507813827 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faee80cdd30 con 0x7faee806a840 2023-10-14T10:36:56.997 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.991+0000 7faeee48c700 1 --2- 172.21.15.67:0/3507813827 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7faee806a840 0x7faee806acb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7faee8161ae0 tx=0x7faee000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cda7037fe916706 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:56.997 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.991+0000 7faeecc89700 1 -- 172.21.15.67:0/3507813827 <== mon.2 v2:172.21.15.72:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee0016070 con 0x7faee806a840 2023-10-14T10:36:56.997 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.991+0000 7faeecc89700 1 -- 172.21.15.67:0/3507813827 <== mon.2 v2:172.21.15.72:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faee000bb20 con 0x7faee806a840 2023-10-14T10:36:56.998 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.991+0000 7faeecc89700 1 -- 172.21.15.67:0/3507813827 <== mon.2 v2:172.21.15.72:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee000e070 con 0x7faee806a840 2023-10-14T10:36:56.998 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.992+0000 7faeefeef700 1 -- 172.21.15.67:0/3507813827 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7faee806a840 msgr2=0x7faee806acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:56.998 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.992+0000 7faeefeef700 1 --2- 172.21.15.67:0/3507813827 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7faee806a840 0x7faee806acb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7faee8161ae0 tx=0x7faee000b330 comp rx=0 tx=0).stop 2023-10-14T10:36:56.998 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.992+0000 7faeefeef700 1 -- 172.21.15.67:0/3507813827 shutdown_connections 2023-10-14T10:36:56.998 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.992+0000 7faeefeef700 1 --2- 172.21.15.67:0/3507813827 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7faee806a840 0x7faee806acb0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:56.999 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.992+0000 7faeefeef700 1 --2- 172.21.15.67:0/3507813827 >> v2:172.21.15.72:3300/0 conn(0x7faee815e2f0 0x7faee80cede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:56.999 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.992+0000 7faeefeef700 1 -- 172.21.15.67:0/3507813827 >> 172.21.15.67:0/3507813827 conn(0x7faee806dae0 msgr2=0x7faee806def0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:56.999 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.992+0000 7faeefeef700 1 -- 172.21.15.67:0/3507813827 shutdown_connections 2023-10-14T10:36:56.999 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.992+0000 7faeefeef700 1 -- 172.21.15.67:0/3507813827 wait complete. 2023-10-14T10:36:57.002 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.993+0000 7faeefeef700 1 Processor -- start 2023-10-14T10:36:57.002 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.993+0000 7faeefeef700 1 -- start start 2023-10-14T10:36:57.002 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.994+0000 7faeefeef700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faee806a840 0x7faee80ad2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:57.002 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.994+0000 7faeefeef700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faee80cd950 0x7faee815c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:57.003 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.994+0000 7faeefeef700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7faee815e2f0 0x7faee80b4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:57.003 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.994+0000 7faeefeef700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faee815cb80 con 0x7faee80cd950 2023-10-14T10:36:57.003 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.994+0000 7faeefeef700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faee80ad950 con 0x7faee806a840 2023-10-14T10:36:57.003 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.994+0000 7faeefeef700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faee80adc30 con 0x7faee815e2f0 2023-10-14T10:36:57.003 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.994+0000 7faeed48a700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faee80cd950 0x7faee815c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:57.004 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.994+0000 7faeed48a700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faee80cd950 0x7faee815c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45670/0 (socket says 172.21.15.67:45670) 2023-10-14T10:36:57.004 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.994+0000 7faeed48a700 1 -- 172.21.15.67:0/4044277969 learned_addr learned my addr 172.21.15.67:0/4044277969 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:57.004 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.994+0000 7faeedc8b700 1 --2- 172.21.15.67:0/4044277969 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faee806a840 0x7faee80ad2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:57.004 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.994+0000 7faeee48c700 1 --2- 172.21.15.67:0/4044277969 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7faee815e2f0 0x7faee80b4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:57.004 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.994+0000 7faeed48a700 1 -- 172.21.15.67:0/4044277969 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7faee815e2f0 msgr2=0x7faee80b4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:57.005 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.994+0000 7faeed48a700 1 --2- 172.21.15.67:0/4044277969 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7faee815e2f0 0x7faee80b4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:57.005 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.994+0000 7faeed48a700 1 -- 172.21.15.67:0/4044277969 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faee806a840 msgr2=0x7faee80ad2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:57.005 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.994+0000 7faeed48a700 1 --2- 172.21.15.67:0/4044277969 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faee806a840 0x7faee80ad2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:57.005 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.994+0000 7faeed48a700 1 -- 172.21.15.67:0/4044277969 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faee0009a70 con 0x7faee80cd950 2023-10-14T10:36:57.005 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.995+0000 7faeed48a700 1 --2- 172.21.15.67:0/4044277969 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faee80cd950 0x7faee815c640 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7faee400e820 tx=0x7faee400eb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:57.006 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.995+0000 7faede7fc700 1 -- 172.21.15.67:0/4044277969 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee4013680 con 0x7faee80cd950 2023-10-14T10:36:57.006 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.995+0000 7faede7fc700 1 -- 172.21.15.67:0/4044277969 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faee40137e0 con 0x7faee80cd950 2023-10-14T10:36:57.006 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.995+0000 7faeefeef700 1 -- 172.21.15.67:0/4044277969 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faee80adf10 con 0x7faee80cd950 2023-10-14T10:36:57.006 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.995+0000 7faede7fc700 1 -- 172.21.15.67:0/4044277969 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee4013b90 con 0x7faee80cd950 2023-10-14T10:36:57.006 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.995+0000 7faeefeef700 1 -- 172.21.15.67:0/4044277969 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faee80ae4d0 con 0x7faee80cd950 2023-10-14T10:36:57.007 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.996+0000 7faeefeef700 1 -- 172.21.15.67:0/4044277969 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faee80ae690 con 0x7faee80cd950 2023-10-14T10:36:57.007 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.996+0000 7faede7fc700 1 -- 172.21.15.67:0/4044277969 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7faee4013cf0 con 0x7faee80cd950 2023-10-14T10:36:57.007 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.997+0000 7faede7fc700 1 --2- 172.21.15.67:0/4044277969 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7faecc06af00 0x7faecc06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:57.007 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.997+0000 7faede7fc700 1 -- 172.21.15.67:0/4044277969 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7faee40894d0 con 0x7faee80cd950 2023-10-14T10:36:57.007 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.997+0000 7faeedc8b700 1 --2- 172.21.15.67:0/4044277969 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7faecc06af00 0x7faecc06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:57.007 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:56.997+0000 7faeedc8b700 1 --2- 172.21.15.67:0/4044277969 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7faecc06af00 0x7faecc06d3c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7faed8007fd0 tx=0x7faed8007de0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:57.008 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:57.001+0000 7faede7fc700 1 -- 172.21.15.67:0/4044277969 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7faee4054840 con 0x7faee80cd950 2023-10-14T10:36:57.207 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:57.204+0000 7faeefeef700 1 -- 172.21.15.67:0/4044277969 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7faee8067d20 con 0x7faee80cd950 2023-10-14T10:36:57.207 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:57.205+0000 7faede7fc700 1 -- 172.21.15.67:0/4044277969 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v29) v1 ==== 84+0+65 (secure 0 0 0) 0x7faee4057e60 con 0x7faee80cd950 2023-10-14T10:36:57.207 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-14T10:36:57.207 INFO:teuthology.orchestra.run.smithi067.stdout:[ 2023-10-14T10:36:57.208 INFO:teuthology.orchestra.run.smithi067.stdout: { 2023-10-14T10:36:57.208 INFO:teuthology.orchestra.run.smithi067.stdout: "poolnum": 1, 2023-10-14T10:36:57.208 INFO:teuthology.orchestra.run.smithi067.stdout: "poolname": ".mgr" 2023-10-14T10:36:57.208 INFO:teuthology.orchestra.run.smithi067.stdout: } 2023-10-14T10:36:57.208 INFO:teuthology.orchestra.run.smithi067.stdout:] 2023-10-14T10:36:57.209 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:57.207+0000 7faeefeef700 1 -- 172.21.15.67:0/4044277969 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7faecc06af00 msgr2=0x7faecc06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:57.210 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:57.207+0000 7faeefeef700 1 --2- 172.21.15.67:0/4044277969 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7faecc06af00 0x7faecc06d3c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7faed8007fd0 tx=0x7faed8007de0 comp rx=0 tx=0).stop 2023-10-14T10:36:57.210 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:57.207+0000 7faeefeef700 1 -- 172.21.15.67:0/4044277969 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faee80cd950 msgr2=0x7faee815c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:57.210 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:57.207+0000 7faeefeef700 1 --2- 172.21.15.67:0/4044277969 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faee80cd950 0x7faee815c640 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7faee400e820 tx=0x7faee400eb30 comp rx=0 tx=0).stop 2023-10-14T10:36:57.210 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:57.207+0000 7faeefeef700 1 -- 172.21.15.67:0/4044277969 shutdown_connections 2023-10-14T10:36:57.211 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:57.207+0000 7faeefeef700 1 --2- 172.21.15.67:0/4044277969 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7faecc06af00 0x7faecc06d3c0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:57.212 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:57.207+0000 7faeefeef700 1 --2- 172.21.15.67:0/4044277969 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7faee815e2f0 0x7faee80b4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:57.213 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:57.207+0000 7faeefeef700 1 --2- 172.21.15.67:0/4044277969 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faee806a840 0x7faee80ad2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:57.213 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:57.207+0000 7faeefeef700 1 --2- 172.21.15.67:0/4044277969 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faee80cd950 0x7faee815c640 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:57.213 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:57.207+0000 7faeefeef700 1 -- 172.21.15.67:0/4044277969 >> 172.21.15.67:0/4044277969 conn(0x7faee806dae0 msgr2=0x7faee815d950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:57.213 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:57.207+0000 7faeefeef700 1 -- 172.21.15.67:0/4044277969 shutdown_connections 2023-10-14T10:36:57.213 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:57.207+0000 7faeefeef700 1 -- 172.21.15.67:0/4044277969 wait complete. 2023-10-14T10:36:57.221 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'testcephfs1' 2023-10-14T10:36:57.221 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create testcephfs1_metadata --pg_num_min 64 2023-10-14T10:36:57.350 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:57.349+0000 7fdc4858a700 1 Processor -- start 2023-10-14T10:36:57.350 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:57.349+0000 7fdc4858a700 1 -- start start 2023-10-14T10:36:57.354 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:57.349+0000 7fdc4858a700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fdc400cdf40 0x7fdc400d2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:57.354 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:57.349+0000 7fdc4858a700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fdc400cf430 0x7fdc400ce320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:57.354 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:57.349+0000 7fdc4858a700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc400d05e0 con 0x7fdc400cc320 2023-10-14T10:36:57.354 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:57.349+0000 7fdc4858a700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc400ce860 con 0x7fdc400cf430 2023-10-14T10:36:57.355 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:57.349+0000 7fdc4858a700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc400b08d0 con 0x7fdc400cdf40 2023-10-14T10:36:57.355 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:57.349+0000 7fdc46b27700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fdc400cc320 0x7fdc400cc7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:53448/0 (socket says 172.21.15.67:53448) 2023-10-14T10:36:57.355 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:57.349+0000 7fdc46b27700 1 -- 172.21.15.67:0/2941836141 learned_addr learned my addr 172.21.15.67:0/2941836141 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:36:57.355 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:57.349+0000 7fdc45b25700 1 --2- 172.21.15.67:0/2941836141 >> v2:172.21.15.72:3300/0 conn(0x7fdc400cf430 0x7fdc400ce320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:57.355 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:57.349+0000 7fdc45324700 1 -- 172.21.15.67:0/2941836141 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 231180792 0 0) 0x7fdc400d05e0 con 0x7fdc400cc320 2023-10-14T10:36:57.356 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:57.349+0000 7fdc46326700 1 --2- 172.21.15.67:0/2941836141 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fdc400cdf40 0x7fdc400d2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:57.356 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:57.349+0000 7fdc45324700 1 -- 172.21.15.67:0/2941836141 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc2c0035b0 con 0x7fdc400cc320 2023-10-14T10:36:57.356 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:57.349+0000 7fdc45324700 1 -- 172.21.15.67:0/2941836141 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3507961880 0 0) 0x7fdc2c0035b0 con 0x7fdc400cc320 2023-10-14T10:36:57.356 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:57.349+0000 7fdc45324700 1 -- 172.21.15.67:0/2941836141 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fdc400cdf40 msgr2=0x7fdc400d2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:57.356 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:57.349+0000 7fdc45324700 1 --2- 172.21.15.67:0/2941836141 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fdc400cdf40 0x7fdc400d2050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:57.357 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:57.349+0000 7fdc45324700 1 -- 172.21.15.67:0/2941836141 >> v2:172.21.15.72:3300/0 conn(0x7fdc400cf430 msgr2=0x7fdc400ce320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:57.357 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:57.349+0000 7fdc45324700 1 --2- 172.21.15.67:0/2941836141 >> v2:172.21.15.72:3300/0 conn(0x7fdc400cf430 0x7fdc400ce320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:57.357 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:57.349+0000 7fdc45324700 1 -- 172.21.15.67:0/2941836141 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc400cfcb0 con 0x7fdc400cc320 2023-10-14T10:36:57.357 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:57.349+0000 7fdc45324700 1 -- 172.21.15.67:0/2941836141 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3180043755 0 0) 0x7fdc380019f0 con 0x7fdc400cc320 2023-10-14T10:36:57.357 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:57.349+0000 7fdc45324700 1 -- 172.21.15.67:0/2941836141 >> v1:172.21.15.67:6789/0 conn(0x7fdc400cc320 legacy=0x7fdc400cc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:57.358 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:57.350+0000 7fdc45324700 1 --2- 172.21.15.67:0/2941836141 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fdc400cdf40 0x7fdc2c003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:57.358 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:57.350+0000 7fdc45324700 1 --2- 172.21.15.67:0/2941836141 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc2c003f80 0x7fdc2c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:57.358 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:57.350+0000 7fdc45324700 1 --2- 172.21.15.67:0/2941836141 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdc2c004d90 0x7fdc2c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:57.358 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:57.350+0000 7fdc45324700 1 -- 172.21.15.67:0/2941836141 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc400ce860 con 0x7fdc2c003f80 2023-10-14T10:36:57.358 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:57.350+0000 7fdc45324700 1 -- 172.21.15.67:0/2941836141 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc400b08d0 con 0x7fdc2c004d90 2023-10-14T10:36:57.359 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:57.350+0000 7fdc45324700 1 -- 172.21.15.67:0/2941836141 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2c00ba10 con 0x7fdc400cdf40 2023-10-14T10:36:57.359 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:57.350+0000 7fdc45b25700 1 --2- 172.21.15.67:0/2941836141 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc2c003f80 0x7fdc2c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:57.359 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:57.350+0000 7fdc46326700 1 --2- 172.21.15.67:0/2941836141 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fdc400cdf40 0x7fdc2c003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:57.359 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:57.350+0000 7fdc46b27700 1 --2- 172.21.15.67:0/2941836141 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdc2c004d90 0x7fdc2c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:57.359 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:57.350+0000 7fdc45b25700 1 -- 172.21.15.67:0/2941836141 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fdc400cdf40 msgr2=0x7fdc2c003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:57.360 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:57.350+0000 7fdc45b25700 1 --2- 172.21.15.67:0/2941836141 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fdc400cdf40 0x7fdc2c003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:57.360 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:57.350+0000 7fdc45b25700 1 -- 172.21.15.67:0/2941836141 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdc2c004d90 msgr2=0x7fdc2c005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:57.360 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:57.350+0000 7fdc45b25700 1 --2- 172.21.15.67:0/2941836141 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdc2c004d90 0x7fdc2c005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:57.360 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:57.350+0000 7fdc45b25700 1 -- 172.21.15.67:0/2941836141 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc2c0035b0 con 0x7fdc2c003f80 2023-10-14T10:36:57.360 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:57.350+0000 7fdc45b25700 1 --2- 172.21.15.67:0/2941836141 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc2c003f80 0x7fdc2c004850 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fdc30009b60 tx=0x7fdc2c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87fc1d7651f63d23 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:57.361 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:57.350+0000 7fdc45324700 1 -- 172.21.15.67:0/2941836141 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc3000e660 con 0x7fdc2c003f80 2023-10-14T10:36:57.361 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:57.350+0000 7fdc45324700 1 -- 172.21.15.67:0/2941836141 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc3000e7c0 con 0x7fdc2c003f80 2023-10-14T10:36:57.361 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:57.350+0000 7fdc45324700 1 -- 172.21.15.67:0/2941836141 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc3000eb30 con 0x7fdc2c003f80 2023-10-14T10:36:57.361 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:57.351+0000 7fdc4858a700 1 -- 172.21.15.67:0/2941836141 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc2c003f80 msgr2=0x7fdc2c004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:57.361 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:57.351+0000 7fdc4858a700 1 --2- 172.21.15.67:0/2941836141 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc2c003f80 0x7fdc2c004850 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fdc30009b60 tx=0x7fdc2c005880 comp rx=0 tx=0).stop 2023-10-14T10:36:57.362 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:57.351+0000 7fdc46b27700 1 -- 172.21.15.67:0/2941836141 reap_dead start 2023-10-14T10:36:57.362 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:57.351+0000 7fdc4858a700 1 -- 172.21.15.67:0/2941836141 shutdown_connections 2023-10-14T10:36:57.362 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:57.351+0000 7fdc4858a700 1 -- 172.21.15.67:0/2941836141 >> 172.21.15.67:0/2941836141 conn(0x7fdc400d1530 msgr2=0x7fdc400d1940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:57.362 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:57.351+0000 7fdc4858a700 1 -- 172.21.15.67:0/2941836141 shutdown_connections 2023-10-14T10:36:57.362 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:57.351+0000 7fdc4858a700 1 -- 172.21.15.67:0/2941836141 wait complete. 2023-10-14T10:36:57.365 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:57.352+0000 7fdc4858a700 1 Processor -- start 2023-10-14T10:36:57.365 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:57.353+0000 7fdc4858a700 1 -- start start 2023-10-14T10:36:57.366 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:57.353+0000 7fdc4858a700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc2c003f80 0x7fdc400c42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:57.366 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:57.353+0000 7fdc4858a700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fdc40150000 0x7fdc400ae4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:57.366 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:57.354+0000 7fdc4858a700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdc40158920 0x7fdc400ae9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:57.366 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:57.354+0000 7fdc4858a700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc400c6610 con 0x7fdc2c003f80 2023-10-14T10:36:57.366 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:57.354+0000 7fdc4858a700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc4006c600 con 0x7fdc40158920 2023-10-14T10:36:57.367 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:57.354+0000 7fdc4858a700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc4006b9f0 con 0x7fdc40150000 2023-10-14T10:36:57.367 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:57.354+0000 7fdc46326700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc2c003f80 0x7fdc400c42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:57.367 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:57.354+0000 7fdc46326700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc2c003f80 0x7fdc400c42c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45688/0 (socket says 172.21.15.67:45688) 2023-10-14T10:36:57.367 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:57.354+0000 7fdc46326700 1 -- 172.21.15.67:0/3503976625 learned_addr learned my addr 172.21.15.67:0/3503976625 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:57.367 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:57.354+0000 7fdc46b27700 1 --2- 172.21.15.67:0/3503976625 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdc40158920 0x7fdc400ae9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:57.368 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:57.354+0000 7fdc46326700 1 -- 172.21.15.67:0/3503976625 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fdc40150000 msgr2=0x7fdc400ae4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:57.368 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:57.354+0000 7fdc46326700 1 --2- 172.21.15.67:0/3503976625 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fdc40150000 0x7fdc400ae4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:57.368 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:57.354+0000 7fdc46326700 1 -- 172.21.15.67:0/3503976625 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdc40158920 msgr2=0x7fdc400ae9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:57.368 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:57.354+0000 7fdc46326700 1 --2- 172.21.15.67:0/3503976625 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdc40158920 0x7fdc400ae9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:57.368 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:57.354+0000 7fdc46326700 1 -- 172.21.15.67:0/3503976625 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc4006bcd0 con 0x7fdc2c003f80 2023-10-14T10:36:57.369 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:57.354+0000 7fdc46326700 1 --2- 172.21.15.67:0/3503976625 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc2c003f80 0x7fdc400c42c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c0119f0 tx=0x7fdc3c011db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:57.369 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:57.354+0000 7fdc36ffd700 1 -- 172.21.15.67:0/3503976625 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc3c005c30 con 0x7fdc2c003f80 2023-10-14T10:36:57.369 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:57.354+0000 7fdc36ffd700 1 -- 172.21.15.67:0/3503976625 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc3c005d90 con 0x7fdc2c003f80 2023-10-14T10:36:57.369 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:57.354+0000 7fdc36ffd700 1 -- 172.21.15.67:0/3503976625 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc3c00f950 con 0x7fdc2c003f80 2023-10-14T10:36:57.369 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:57.354+0000 7fdc4858a700 1 -- 172.21.15.67:0/3503976625 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc4006bfb0 con 0x7fdc2c003f80 2023-10-14T10:36:57.369 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:57.354+0000 7fdc4858a700 1 -- 172.21.15.67:0/3503976625 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc4006d5d0 con 0x7fdc2c003f80 2023-10-14T10:36:57.370 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:57.355+0000 7fdc4858a700 1 -- 172.21.15.67:0/3503976625 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc40067d20 con 0x7fdc2c003f80 2023-10-14T10:36:57.370 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:57.356+0000 7fdc36ffd700 1 -- 172.21.15.67:0/3503976625 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fdc3c00fab0 con 0x7fdc2c003f80 2023-10-14T10:36:57.370 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:57.356+0000 7fdc36ffd700 1 --2- 172.21.15.67:0/3503976625 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fdc2406af00 0x7fdc2406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:57.370 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:57.356+0000 7fdc36ffd700 1 -- 172.21.15.67:0/3503976625 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fdc3c089990 con 0x7fdc2c003f80 2023-10-14T10:36:57.370 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:57.356+0000 7fdc45b25700 1 --2- 172.21.15.67:0/3503976625 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fdc2406af00 0x7fdc2406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:57.371 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:57.358+0000 7fdc45b25700 1 --2- 172.21.15.67:0/3503976625 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fdc2406af00 0x7fdc2406d3c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fdc400af050 tx=0x7fdc30013000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:57.371 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:57.361+0000 7fdc36ffd700 1 -- 172.21.15.67:0/3503976625 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdc3c054d90 con 0x7fdc2c003f80 2023-10-14T10:36:57.562 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:57.561+0000 7fdc4858a700 1 -- 172.21.15.67:0/3503976625 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "testcephfs1_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fdc400704d0 con 0x7fdc2c003f80 2023-10-14T10:36:58.042 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:58.041+0000 7fdc36ffd700 1 -- 172.21.15.67:0/3503976625 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "testcephfs1_metadata", "pg_num_min": 64}]=0 pool 'testcephfs1_metadata' created v30) v1 ==== 148+0+0 (secure 0 0 0) 0x7fdc3c0583b0 con 0x7fdc2c003f80 2023-10-14T10:36:58.045 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:58.044+0000 7fdc4858a700 1 -- 172.21.15.67:0/3503976625 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fdc2406af00 msgr2=0x7fdc2406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:58.045 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:58.044+0000 7fdc4858a700 1 --2- 172.21.15.67:0/3503976625 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fdc2406af00 0x7fdc2406d3c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fdc400af050 tx=0x7fdc30013000 comp rx=0 tx=0).stop 2023-10-14T10:36:58.046 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:58.044+0000 7fdc4858a700 1 -- 172.21.15.67:0/3503976625 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc2c003f80 msgr2=0x7fdc400c42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:58.046 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:58.044+0000 7fdc4858a700 1 --2- 172.21.15.67:0/3503976625 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc2c003f80 0x7fdc400c42c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c0119f0 tx=0x7fdc3c011db0 comp rx=0 tx=0).stop 2023-10-14T10:36:58.046 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:58.044+0000 7fdc4858a700 1 -- 172.21.15.67:0/3503976625 shutdown_connections 2023-10-14T10:36:58.046 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:58.044+0000 7fdc4858a700 1 --2- 172.21.15.67:0/3503976625 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fdc2406af00 0x7fdc2406d3c0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:58.046 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:58.044+0000 7fdc4858a700 1 --2- 172.21.15.67:0/3503976625 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdc40158920 0x7fdc400ae9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:58.047 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:58.044+0000 7fdc4858a700 1 --2- 172.21.15.67:0/3503976625 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc2c003f80 0x7fdc400c42c0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:58.047 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:58.044+0000 7fdc4858a700 1 --2- 172.21.15.67:0/3503976625 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fdc40150000 0x7fdc400ae4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:58.047 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:58.044+0000 7fdc4858a700 1 -- 172.21.15.67:0/3503976625 >> 172.21.15.67:0/3503976625 conn(0x7fdc400d1530 msgr2=0x7fdc400b0060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:58.047 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:58.044+0000 7fdc4858a700 1 -- 172.21.15.67:0/3503976625 shutdown_connections 2023-10-14T10:36:58.047 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:58.044+0000 7fdc4858a700 1 -- 172.21.15.67:0/3503976625 wait complete. 2023-10-14T10:36:58.048 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'testcephfs1_metadata' created 2023-10-14T10:36:58.060 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create testcephfs1_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-14T10:36:58.194 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:58.193+0000 7f8858a2d700 1 Processor -- start 2023-10-14T10:36:58.195 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:58.193+0000 7f8858a2d700 1 -- start start 2023-10-14T10:36:58.197 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:58.194+0000 7f8858a2d700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f885415a730 0x7f88540b5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:58.197 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:58.194+0000 7f8858a2d700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f88540cecf0 0x7f885415e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:58.197 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:58.194+0000 7f8858a2d700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f885415e860 0x7f88540cce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:58.197 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:58.194+0000 7f8858a2d700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88540b1fc0 con 0x7f885415e860 2023-10-14T10:36:58.198 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:58.194+0000 7f8858a2d700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88540b6400 con 0x7f88540cecf0 2023-10-14T10:36:58.198 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:58.194+0000 7f8858a2d700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88540ce450 con 0x7f885415a730 2023-10-14T10:36:58.198 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:58.194+0000 7f8852d9d700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f885415e860 0x7f88540cce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:58.198 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:58.194+0000 7f8852d9d700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f885415e860 0x7f88540cce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45700/0 (socket says 172.21.15.67:45700) 2023-10-14T10:36:58.199 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:58.194+0000 7f8852d9d700 1 -- 172.21.15.67:0/2271564750 learned_addr learned my addr 172.21.15.67:0/2271564750 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:58.199 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:58.194+0000 7f8851d9b700 1 --2- 172.21.15.67:0/2271564750 >> v2:172.21.15.72:3300/0 conn(0x7f88540cecf0 0x7f885415e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:58.199 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:58.194+0000 7f885259c700 1 --2- 172.21.15.67:0/2271564750 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f885415a730 0x7f88540b5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:58.199 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:58.195+0000 7f8852d9d700 1 -- 172.21.15.67:0/2271564750 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f885415a730 msgr2=0x7f88540b5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:58.199 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:58.195+0000 7f8852d9d700 1 --2- 172.21.15.67:0/2271564750 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f885415a730 0x7f88540b5e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:58.199 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:58.195+0000 7f8852d9d700 1 -- 172.21.15.67:0/2271564750 >> v2:172.21.15.72:3300/0 conn(0x7f88540cecf0 msgr2=0x7f885415e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:58.200 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:58.195+0000 7f8852d9d700 1 --2- 172.21.15.67:0/2271564750 >> v2:172.21.15.72:3300/0 conn(0x7f88540cecf0 0x7f885415e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:58.200 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:58.195+0000 7f8852d9d700 1 -- 172.21.15.67:0/2271564750 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88540ce730 con 0x7f885415e860 2023-10-14T10:36:58.200 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:58.195+0000 7f8851d9b700 1 --2- 172.21.15.67:0/2271564750 >> v2:172.21.15.72:3300/0 conn(0x7f88540cecf0 0x7f885415e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-14T10:36:58.200 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:58.195+0000 7f885259c700 1 --2- 172.21.15.67:0/2271564750 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f885415a730 0x7f88540b5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-14T10:36:58.200 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:58.195+0000 7f8852d9d700 1 --2- 172.21.15.67:0/2271564750 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f885415e860 0x7f88540cce90 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f88540b3060 tx=0x7f884c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dce0f3a775b897f9 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:58.201 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:58.195+0000 7f885159a700 1 -- 172.21.15.67:0/2271564750 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f884c00e070 con 0x7f885415e860 2023-10-14T10:36:58.201 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:58.195+0000 7f885159a700 1 -- 172.21.15.67:0/2271564750 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f884c00b9c0 con 0x7f885415e860 2023-10-14T10:36:58.201 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:58.195+0000 7f885159a700 1 -- 172.21.15.67:0/2271564750 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f884c00bd70 con 0x7f885415e860 2023-10-14T10:36:58.201 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:58.195+0000 7f8858a2d700 1 -- 172.21.15.67:0/2271564750 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f885415e860 msgr2=0x7f88540cce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:58.201 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:58.195+0000 7f8858a2d700 1 --2- 172.21.15.67:0/2271564750 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f885415e860 0x7f88540cce90 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f88540b3060 tx=0x7f884c00b330 comp rx=0 tx=0).stop 2023-10-14T10:36:58.202 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:58.196+0000 7f8858a2d700 1 -- 172.21.15.67:0/2271564750 shutdown_connections 2023-10-14T10:36:58.202 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:58.196+0000 7f8858a2d700 1 --2- 172.21.15.67:0/2271564750 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f885415e860 0x7f88540cce90 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:58.202 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:58.196+0000 7f8858a2d700 1 --2- 172.21.15.67:0/2271564750 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f885415a730 0x7f88540b5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:58.202 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:58.196+0000 7f8858a2d700 1 --2- 172.21.15.67:0/2271564750 >> v2:172.21.15.72:3300/0 conn(0x7f88540cecf0 0x7f885415e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:58.202 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:58.196+0000 7f8858a2d700 1 -- 172.21.15.67:0/2271564750 >> 172.21.15.67:0/2271564750 conn(0x7f885406dac0 msgr2=0x7f885406dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:58.203 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:58.196+0000 7f8858a2d700 1 -- 172.21.15.67:0/2271564750 shutdown_connections 2023-10-14T10:36:58.203 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:58.196+0000 7f8858a2d700 1 -- 172.21.15.67:0/2271564750 wait complete. 2023-10-14T10:36:58.205 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:58.197+0000 7f8858a2d700 1 Processor -- start 2023-10-14T10:36:58.206 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:58.197+0000 7f8858a2d700 1 -- start start 2023-10-14T10:36:58.206 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:58.198+0000 7f8858a2d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f88540cecf0 0x7f88541575e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:58.206 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:58.198+0000 7f8858a2d700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f885415a730 0x7f8854159620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:58.206 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:58.198+0000 7f8858a2d700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f88540c6b20 0x7f8854159b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:58.206 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:58.198+0000 7f8858a2d700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8854157b20 con 0x7f88540cecf0 2023-10-14T10:36:58.207 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:58.198+0000 7f8858a2d700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88540c6690 con 0x7f88540c6b20 2023-10-14T10:36:58.207 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:58.198+0000 7f8858a2d700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88540c5a80 con 0x7f885415a730 2023-10-14T10:36:58.207 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:58.198+0000 7f885259c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f88540cecf0 0x7f88541575e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:58.207 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:58.198+0000 7f885259c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f88540cecf0 0x7f88541575e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45702/0 (socket says 172.21.15.67:45702) 2023-10-14T10:36:58.207 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:58.198+0000 7f885259c700 1 -- 172.21.15.67:0/3267166270 learned_addr learned my addr 172.21.15.67:0/3267166270 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:58.208 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:58.198+0000 7f8852d9d700 1 --2- 172.21.15.67:0/3267166270 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f88540c6b20 0x7f8854159b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:58.208 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:58.198+0000 7f8851d9b700 1 --2- 172.21.15.67:0/3267166270 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f885415a730 0x7f8854159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:58.208 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:58.198+0000 7f885259c700 1 -- 172.21.15.67:0/3267166270 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f885415a730 msgr2=0x7f8854159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:58.208 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:58.198+0000 7f885259c700 1 --2- 172.21.15.67:0/3267166270 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f885415a730 0x7f8854159620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:58.208 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:58.198+0000 7f885259c700 1 -- 172.21.15.67:0/3267166270 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f88540c6b20 msgr2=0x7f8854159b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:58.209 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:58.198+0000 7f885259c700 1 --2- 172.21.15.67:0/3267166270 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f88540c6b20 0x7f8854159b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:58.209 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:58.198+0000 7f885259c700 1 -- 172.21.15.67:0/3267166270 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f884c009b30 con 0x7f88540cecf0 2023-10-14T10:36:58.209 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:58.198+0000 7f885259c700 1 --2- 172.21.15.67:0/3267166270 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f88540cecf0 0x7f88541575e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f884800c750 tx=0x7f884800cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:58.209 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:58.198+0000 7f8842ffd700 1 -- 172.21.15.67:0/3267166270 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8848011800 con 0x7f88540cecf0 2023-10-14T10:36:58.209 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:58.199+0000 7f8842ffd700 1 -- 172.21.15.67:0/3267166270 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8848011960 con 0x7f88540cecf0 2023-10-14T10:36:58.210 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:58.199+0000 7f8858a2d700 1 -- 172.21.15.67:0/3267166270 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f885415ac90 con 0x7f88540cecf0 2023-10-14T10:36:58.210 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:58.199+0000 7f8842ffd700 1 -- 172.21.15.67:0/3267166270 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8848011cd0 con 0x7f88540cecf0 2023-10-14T10:36:58.210 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:58.199+0000 7f8858a2d700 1 -- 172.21.15.67:0/3267166270 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88540c6040 con 0x7f88540cecf0 2023-10-14T10:36:58.210 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:58.200+0000 7f8858a2d700 1 -- 172.21.15.67:0/3267166270 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8854067d20 con 0x7f88540cecf0 2023-10-14T10:36:58.210 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:58.200+0000 7f8842ffd700 1 -- 172.21.15.67:0/3267166270 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f8848021350 con 0x7f88540cecf0 2023-10-14T10:36:58.211 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:58.201+0000 7f8842ffd700 1 --2- 172.21.15.67:0/3267166270 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f883406af00 0x7f883406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:58.211 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:58.201+0000 7f8842ffd700 1 -- 172.21.15.67:0/3267166270 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5232+0+0 (secure 0 0 0) 0x7f8848089380 con 0x7f88540cecf0 2023-10-14T10:36:58.211 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:58.201+0000 7f8851d9b700 1 --2- 172.21.15.67:0/3267166270 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f883406af00 0x7f883406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:58.211 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:58.201+0000 7f8851d9b700 1 --2- 172.21.15.67:0/3267166270 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f883406af00 0x7f883406d3c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f88540af990 tx=0x7f8844008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:58.212 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:58.205+0000 7f8842ffd700 1 -- 172.21.15.67:0/3267166270 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f88480545a0 con 0x7f88540cecf0 2023-10-14T10:36:58.407 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:58.406+0000 7f8858a2d700 1 -- 172.21.15.67:0/3267166270 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "testcephfs1_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f885406e8c0 con 0x7f88540cecf0 2023-10-14T10:36:59.044 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:59.043+0000 7f8842ffd700 1 -- 172.21.15.67:0/3267166270 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "testcephfs1_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'testcephfs1_data' created v31) v1 ==== 180+0+0 (secure 0 0 0) 0x7f8848057bc0 con 0x7f88540cecf0 2023-10-14T10:36:59.047 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:59.046+0000 7f8858a2d700 1 -- 172.21.15.67:0/3267166270 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f883406af00 msgr2=0x7f883406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:59.047 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:59.046+0000 7f8858a2d700 1 --2- 172.21.15.67:0/3267166270 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f883406af00 0x7f883406d3c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f88540af990 tx=0x7f8844008040 comp rx=0 tx=0).stop 2023-10-14T10:36:59.048 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:59.046+0000 7f8858a2d700 1 -- 172.21.15.67:0/3267166270 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f88540cecf0 msgr2=0x7f88541575e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:59.048 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:59.046+0000 7f8858a2d700 1 --2- 172.21.15.67:0/3267166270 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f88540cecf0 0x7f88541575e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f884800c750 tx=0x7f884800cb10 comp rx=0 tx=0).stop 2023-10-14T10:36:59.048 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:59.046+0000 7f8858a2d700 1 -- 172.21.15.67:0/3267166270 shutdown_connections 2023-10-14T10:36:59.048 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:59.046+0000 7f8858a2d700 1 --2- 172.21.15.67:0/3267166270 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f883406af00 0x7f883406d3c0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:59.048 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:59.046+0000 7f8858a2d700 1 --2- 172.21.15.67:0/3267166270 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f88540c6b20 0x7f8854159b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:59.049 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:59.046+0000 7f8858a2d700 1 --2- 172.21.15.67:0/3267166270 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f88540cecf0 0x7f88541575e0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:59.049 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:59.046+0000 7f8858a2d700 1 --2- 172.21.15.67:0/3267166270 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f885415a730 0x7f8854159620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:59.049 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:59.046+0000 7f8858a2d700 1 -- 172.21.15.67:0/3267166270 >> 172.21.15.67:0/3267166270 conn(0x7f885406dac0 msgr2=0x7f88540b3b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:59.049 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:59.046+0000 7f8858a2d700 1 -- 172.21.15.67:0/3267166270 shutdown_connections 2023-10-14T10:36:59.049 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:59.046+0000 7f8858a2d700 1 -- 172.21.15.67:0/3267166270 wait complete. 2023-10-14T10:36:59.050 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'testcephfs1_data' created 2023-10-14T10:36:59.069 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new testcephfs1 testcephfs1_metadata testcephfs1_data 2023-10-14T10:36:59.201 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:59.200+0000 7fb5d9d3a700 1 Processor -- start 2023-10-14T10:36:59.202 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:59.200+0000 7fb5d9d3a700 1 -- start start 2023-10-14T10:36:59.205 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:59.201+0000 7fb5d9d3a700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fb5d40aeaa0 0x7fb5d40cda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:59.205 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:59.201+0000 7fb5d9d3a700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fb5d40b3f70 0x7fb5d415e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:59.205 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:59.201+0000 7fb5d9d3a700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5d40b29d0 con 0x7fb5d40ae6c0 2023-10-14T10:36:59.206 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:59.201+0000 7fb5d9d3a700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5d40b2cb0 con 0x7fb5d40b3f70 2023-10-14T10:36:59.206 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:59.201+0000 7fb5d9d3a700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d40b2f90 con 0x7fb5d40aeaa0 2023-10-14T10:36:59.206 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:59.201+0000 7fb5d37fe700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fb5d40ae6c0 0x7fb5d40cf850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:53478/0 (socket says 172.21.15.67:53478) 2023-10-14T10:36:59.206 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:59.201+0000 7fb5d37fe700 1 -- 172.21.15.67:0/992517112 learned_addr learned my addr 172.21.15.67:0/992517112 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:36:59.206 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:59.201+0000 7fb5d3fff700 1 --2- 172.21.15.67:0/992517112 >> v2:172.21.15.72:3300/0 conn(0x7fb5d40b3f70 0x7fb5d415e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:59.207 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:59.201+0000 7fb5d27fc700 1 -- 172.21.15.67:0/992517112 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3247834039 0 0) 0x7fb5d40b29d0 con 0x7fb5d40ae6c0 2023-10-14T10:36:59.207 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:59.201+0000 7fb5d27fc700 1 -- 172.21.15.67:0/992517112 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5c00057c0 con 0x7fb5d40ae6c0 2023-10-14T10:36:59.207 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:59.201+0000 7fb5d2ffd700 1 --2- 172.21.15.67:0/992517112 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fb5d40aeaa0 0x7fb5d40cda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:59.207 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:59.202+0000 7fb5d27fc700 1 -- 172.21.15.67:0/992517112 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 889465797 0 0) 0x7fb5c00057c0 con 0x7fb5d40ae6c0 2023-10-14T10:36:59.207 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:59.202+0000 7fb5d27fc700 1 -- 172.21.15.67:0/992517112 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fb5d40aeaa0 msgr2=0x7fb5d40cda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:59.208 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:59.202+0000 7fb5d27fc700 1 --2- 172.21.15.67:0/992517112 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fb5d40aeaa0 0x7fb5d40cda10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:59.208 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:59.202+0000 7fb5d27fc700 1 -- 172.21.15.67:0/992517112 >> v2:172.21.15.72:3300/0 conn(0x7fb5d40b3f70 msgr2=0x7fb5d415e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:59.208 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:59.202+0000 7fb5d27fc700 1 --2- 172.21.15.67:0/992517112 >> v2:172.21.15.72:3300/0 conn(0x7fb5d40b3f70 0x7fb5d415e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:59.208 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:59.202+0000 7fb5d27fc700 1 -- 172.21.15.67:0/992517112 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5d40b3270 con 0x7fb5d40ae6c0 2023-10-14T10:36:59.208 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:59.202+0000 7fb5d27fc700 1 -- 172.21.15.67:0/992517112 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3180043755 0 0) 0x7fb5bc001670 con 0x7fb5d40ae6c0 2023-10-14T10:36:59.209 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:59.202+0000 7fb5d27fc700 1 -- 172.21.15.67:0/992517112 >> v1:172.21.15.67:6789/0 conn(0x7fb5d40ae6c0 legacy=0x7fb5d40cf850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:59.209 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:59.202+0000 7fb5d27fc700 1 --2- 172.21.15.67:0/992517112 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb5c0005c10 0x7fb5c0006030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:59.209 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:59.202+0000 7fb5d27fc700 1 --2- 172.21.15.67:0/992517112 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fb5c0006a60 0x7fb5c0006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:59.209 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:59.202+0000 7fb5d27fc700 1 --2- 172.21.15.67:0/992517112 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb5c00073e0 0x7fb5c000dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:59.209 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:59.202+0000 7fb5d27fc700 1 -- 172.21.15.67:0/992517112 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d40b2cb0 con 0x7fb5c0005c10 2023-10-14T10:36:59.210 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:59.202+0000 7fb5d27fc700 1 -- 172.21.15.67:0/992517112 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d40b2f90 con 0x7fb5c00073e0 2023-10-14T10:36:59.210 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:59.202+0000 7fb5d27fc700 1 -- 172.21.15.67:0/992517112 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c0006860 con 0x7fb5c0006a60 2023-10-14T10:36:59.210 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:59.202+0000 7fb5d37fe700 1 --2- 172.21.15.67:0/992517112 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb5c0005c10 0x7fb5c0006030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:59.210 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:59.202+0000 7fb5d2ffd700 1 --2- 172.21.15.67:0/992517112 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fb5c0006a60 0x7fb5c0006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:59.211 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:59.202+0000 7fb5d3fff700 1 --2- 172.21.15.67:0/992517112 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb5c00073e0 0x7fb5c000dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:59.211 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:59.202+0000 7fb5d37fe700 1 -- 172.21.15.67:0/992517112 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fb5c0006a60 msgr2=0x7fb5c0006ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:59.211 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:59.202+0000 7fb5d37fe700 1 --2- 172.21.15.67:0/992517112 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fb5c0006a60 0x7fb5c0006ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:59.211 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:59.202+0000 7fb5d37fe700 1 -- 172.21.15.67:0/992517112 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb5c00073e0 msgr2=0x7fb5c000dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:36:59.211 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:59.202+0000 7fb5d37fe700 1 --2- 172.21.15.67:0/992517112 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb5c00073e0 0x7fb5c000dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:59.212 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:59.202+0000 7fb5d37fe700 1 -- 172.21.15.67:0/992517112 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5c00057c0 con 0x7fb5c0005c10 2023-10-14T10:36:59.212 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:59.202+0000 7fb5d3fff700 1 --2- 172.21.15.67:0/992517112 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb5c00073e0 0x7fb5c000dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-14T10:36:59.212 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:59.202+0000 7fb5d37fe700 1 --2- 172.21.15.67:0/992517112 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb5c0005c10 0x7fb5c0006030 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc00c370 tx=0x7fb5c0007a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f13881a524b0b243 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:59.212 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:59.202+0000 7fb5d27fc700 1 -- 172.21.15.67:0/992517112 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5bc007c60 con 0x7fb5c0005c10 2023-10-14T10:36:59.212 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:59.202+0000 7fb5d27fc700 1 -- 172.21.15.67:0/992517112 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5bc005960 con 0x7fb5c0005c10 2023-10-14T10:36:59.213 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:59.202+0000 7fb5d27fc700 1 -- 172.21.15.67:0/992517112 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5bc005d10 con 0x7fb5c0005c10 2023-10-14T10:36:59.213 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:59.203+0000 7fb5d9d3a700 1 -- 172.21.15.67:0/992517112 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb5c0005c10 msgr2=0x7fb5c0006030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:59.213 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:59.203+0000 7fb5d9d3a700 1 --2- 172.21.15.67:0/992517112 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb5c0005c10 0x7fb5c0006030 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc00c370 tx=0x7fb5c0007a90 comp rx=0 tx=0).stop 2023-10-14T10:36:59.213 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:59.203+0000 7fb5d3fff700 1 -- 172.21.15.67:0/992517112 reap_dead start 2023-10-14T10:36:59.213 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:59.203+0000 7fb5d9d3a700 1 -- 172.21.15.67:0/992517112 shutdown_connections 2023-10-14T10:36:59.213 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:59.203+0000 7fb5d9d3a700 1 -- 172.21.15.67:0/992517112 >> 172.21.15.67:0/992517112 conn(0x7fb5d406ac60 msgr2=0x7fb5d406b070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:36:59.214 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:59.203+0000 7fb5d9d3a700 1 -- 172.21.15.67:0/992517112 shutdown_connections 2023-10-14T10:36:59.214 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:59.203+0000 7fb5d9d3a700 1 -- 172.21.15.67:0/992517112 wait complete. 2023-10-14T10:36:59.214 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:59.204+0000 7fb5d9d3a700 1 Processor -- start 2023-10-14T10:36:59.214 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:59.204+0000 7fb5d9d3a700 1 -- start start 2023-10-14T10:36:59.217 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:59.205+0000 7fb5d9d3a700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb5c0005c10 0x7fb5d41f6f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:59.217 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:59.205+0000 7fb5d9d3a700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb5d4158c90 0x7fb5d4159110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:59.217 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:59.205+0000 7fb5d9d3a700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fb5d415c890 0x7fb5d415cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:59.217 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:59.205+0000 7fb5d9d3a700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d415d630 con 0x7fb5c0005c10 2023-10-14T10:36:59.218 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:59.205+0000 7fb5d9d3a700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d4159650 con 0x7fb5d4158c90 2023-10-14T10:36:59.218 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:59.205+0000 7fb5d9d3a700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d415ae30 con 0x7fb5d415c890 2023-10-14T10:36:59.218 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:59.205+0000 7fb5d37fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb5c0005c10 0x7fb5d41f6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:59.218 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:59.205+0000 7fb5d37fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb5c0005c10 0x7fb5d41f6f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45722/0 (socket says 172.21.15.67:45722) 2023-10-14T10:36:59.218 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:59.205+0000 7fb5d37fe700 1 -- 172.21.15.67:0/1376087467 learned_addr learned my addr 172.21.15.67:0/1376087467 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:36:59.219 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:59.205+0000 7fb5d2ffd700 1 --2- 172.21.15.67:0/1376087467 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb5d4158c90 0x7fb5d4159110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:59.219 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:59.205+0000 7fb5d37fe700 1 -- 172.21.15.67:0/1376087467 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fb5d415c890 msgr2=0x7fb5d415cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:59.219 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:59.206+0000 7fb5d37fe700 1 --2- 172.21.15.67:0/1376087467 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fb5d415c890 0x7fb5d415cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:59.219 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:59.206+0000 7fb5d37fe700 1 -- 172.21.15.67:0/1376087467 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb5d4158c90 msgr2=0x7fb5d4159110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:36:59.219 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:59.206+0000 7fb5d37fe700 1 --2- 172.21.15.67:0/1376087467 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb5d4158c90 0x7fb5d4159110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:36:59.220 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:59.206+0000 7fb5d37fe700 1 -- 172.21.15.67:0/1376087467 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5d415b9b0 con 0x7fb5c0005c10 2023-10-14T10:36:59.220 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:59.206+0000 7fb5d37fe700 1 --2- 172.21.15.67:0/1376087467 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb5c0005c10 0x7fb5d41f6f40 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc011000 tx=0x7fb5bc000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:59.220 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:59.206+0000 7fb5cbfff700 1 -- 172.21.15.67:0/1376087467 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5bc004870 con 0x7fb5c0005c10 2023-10-14T10:36:59.220 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:59.206+0000 7fb5cbfff700 1 -- 172.21.15.67:0/1376087467 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5bc0049d0 con 0x7fb5c0005c10 2023-10-14T10:36:59.220 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:59.206+0000 7fb5d9d3a700 1 -- 172.21.15.67:0/1376087467 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5d415b080 con 0x7fb5c0005c10 2023-10-14T10:36:59.220 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:59.206+0000 7fb5cbfff700 1 -- 172.21.15.67:0/1376087467 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5bc004d80 con 0x7fb5c0005c10 2023-10-14T10:36:59.221 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:59.206+0000 7fb5d9d3a700 1 -- 172.21.15.67:0/1376087467 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5d415b640 con 0x7fb5c0005c10 2023-10-14T10:36:59.221 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:59.207+0000 7fb5d9d3a700 1 -- 172.21.15.67:0/1376087467 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5d404ebb0 con 0x7fb5c0005c10 2023-10-14T10:36:59.221 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:59.207+0000 7fb5cbfff700 1 -- 172.21.15.67:0/1376087467 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fb5bc0070d0 con 0x7fb5c0005c10 2023-10-14T10:36:59.221 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:59.208+0000 7fb5cbfff700 1 --2- 172.21.15.67:0/1376087467 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fb5b406af00 0x7fb5b406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:36:59.221 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:59.208+0000 7fb5cbfff700 1 -- 172.21.15.67:0/1376087467 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5619+0+0 (secure 0 0 0) 0x7fb5bc08ab30 con 0x7fb5c0005c10 2023-10-14T10:36:59.222 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:59.208+0000 7fb5d2ffd700 1 --2- 172.21.15.67:0/1376087467 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fb5b406af00 0x7fb5b406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:36:59.222 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:59.209+0000 7fb5d2ffd700 1 --2- 172.21.15.67:0/1376087467 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fb5b406af00 0x7fb5b406d3c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb5d40b99b0 tx=0x7fb5c4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:36:59.222 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:59.213+0000 7fb5cbfff700 1 -- 172.21.15.67:0/1376087467 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb5bc055de0 con 0x7fb5c0005c10 2023-10-14T10:36:59.463 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:36:59.462+0000 7fb5d9d3a700 1 -- 172.21.15.67:0/1376087467 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "testcephfs1", "metadata": "testcephfs1_metadata", "data": "testcephfs1_data"} v 0) v1 -- 0x7fb5d4072800 con 0x7fb5c0005c10 2023-10-14T10:37:00.073 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.072+0000 7fb5cbfff700 1 -- 172.21.15.67:0/1376087467 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "testcephfs1", "metadata": "testcephfs1_metadata", "data": "testcephfs1_data"}]=0 new fs with metadata pool 4 and data pool 5 v13) v1 ==== 187+0+0 (secure 0 0 0) 0x7fb5bc007dc0 con 0x7fb5c0005c10 2023-10-14T10:37:00.085 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.084+0000 7fb5d9d3a700 1 -- 172.21.15.67:0/1376087467 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fb5b406af00 msgr2=0x7fb5b406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:00.085 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.084+0000 7fb5d9d3a700 1 --2- 172.21.15.67:0/1376087467 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fb5b406af00 0x7fb5b406d3c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb5d40b99b0 tx=0x7fb5c4008040 comp rx=0 tx=0).stop 2023-10-14T10:37:00.085 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.084+0000 7fb5d9d3a700 1 -- 172.21.15.67:0/1376087467 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb5c0005c10 msgr2=0x7fb5d41f6f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:00.086 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.084+0000 7fb5d9d3a700 1 --2- 172.21.15.67:0/1376087467 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb5c0005c10 0x7fb5d41f6f40 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc011000 tx=0x7fb5bc000fe0 comp rx=0 tx=0).stop 2023-10-14T10:37:00.086 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.084+0000 7fb5d9d3a700 1 -- 172.21.15.67:0/1376087467 shutdown_connections 2023-10-14T10:37:00.086 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.084+0000 7fb5d9d3a700 1 --2- 172.21.15.67:0/1376087467 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fb5b406af00 0x7fb5b406d3c0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:00.086 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.084+0000 7fb5d9d3a700 1 --2- 172.21.15.67:0/1376087467 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fb5d415c890 0x7fb5d415cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:00.086 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.084+0000 7fb5d9d3a700 1 --2- 172.21.15.67:0/1376087467 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb5c0005c10 0x7fb5d41f6f40 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:00.087 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.084+0000 7fb5d9d3a700 1 --2- 172.21.15.67:0/1376087467 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb5d4158c90 0x7fb5d4159110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:00.087 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.084+0000 7fb5d9d3a700 1 -- 172.21.15.67:0/1376087467 >> 172.21.15.67:0/1376087467 conn(0x7fb5d406ac60 msgr2=0x7fb5d415a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:37:00.087 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.084+0000 7fb5d9d3a700 1 -- 172.21.15.67:0/1376087467 shutdown_connections 2023-10-14T10:37:00.088 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.084+0000 7fb5d9d3a700 1 -- 172.21.15.67:0/1376087467 wait complete. 2023-10-14T10:37:00.088 INFO:teuthology.orchestra.run.smithi067.stderr:new fs with metadata pool 4 and data pool 5 2023-10-14T10:37:00.104 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-14T10:37:00.235 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.234+0000 7f4265948700 1 Processor -- start 2023-10-14T10:37:00.235 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.234+0000 7f4265948700 1 -- start start 2023-10-14T10:37:00.237 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.234+0000 7f4265948700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f426015b6c0 0x7f426015f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:00.238 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.234+0000 7f4265948700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f426015f930 0x7f42600cc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:00.238 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.234+0000 7f4265948700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f42600cc6f0 0x7f42600ccf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:00.238 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.235+0000 7f4265948700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42600b03a0 con 0x7f42600cc6f0 2023-10-14T10:37:00.238 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.235+0000 7f4265948700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f426015fe70 con 0x7f426015f930 2023-10-14T10:37:00.238 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.235+0000 7f4265948700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4260160150 con 0x7f426015b6c0 2023-10-14T10:37:00.239 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.235+0000 7f425f7fe700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f42600cc6f0 0x7f42600ccf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:00.239 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.235+0000 7f425f7fe700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f42600cc6f0 0x7f42600ccf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45748/0 (socket says 172.21.15.67:45748) 2023-10-14T10:37:00.239 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.235+0000 7f425f7fe700 1 -- 172.21.15.67:0/3112885446 learned_addr learned my addr 172.21.15.67:0/3112885446 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:37:00.239 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.235+0000 7f425effd700 1 --2- 172.21.15.67:0/3112885446 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f426015b6c0 0x7f426015f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:00.239 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.235+0000 7f425e7fc700 1 --2- 172.21.15.67:0/3112885446 >> v2:172.21.15.72:3300/0 conn(0x7f426015f930 0x7f42600cc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:00.240 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.235+0000 7f425f7fe700 1 -- 172.21.15.67:0/3112885446 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f426015b6c0 msgr2=0x7f426015f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:37:00.240 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.235+0000 7f425f7fe700 1 --2- 172.21.15.67:0/3112885446 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f426015b6c0 0x7f426015f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:00.240 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.235+0000 7f425f7fe700 1 -- 172.21.15.67:0/3112885446 >> v2:172.21.15.72:3300/0 conn(0x7f426015f930 msgr2=0x7f42600cc1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:37:00.240 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.235+0000 7f425f7fe700 1 --2- 172.21.15.67:0/3112885446 >> v2:172.21.15.72:3300/0 conn(0x7f426015f930 0x7f42600cc1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:00.240 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.235+0000 7f425f7fe700 1 -- 172.21.15.67:0/3112885446 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42600b0030 con 0x7f42600cc6f0 2023-10-14T10:37:00.241 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.235+0000 7f425f7fe700 1 --2- 172.21.15.67:0/3112885446 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f42600cc6f0 0x7f42600ccf00 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f4260070580 tx=0x7f425800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=228a28d9aed7c219 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:00.241 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.235+0000 7f425dffb700 1 -- 172.21.15.67:0/3112885446 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f425800e070 con 0x7f42600cc6f0 2023-10-14T10:37:00.241 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.235+0000 7f425dffb700 1 -- 172.21.15.67:0/3112885446 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f425800b910 con 0x7f42600cc6f0 2023-10-14T10:37:00.241 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.235+0000 7f425dffb700 1 -- 172.21.15.67:0/3112885446 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f425800bcc0 con 0x7f42600cc6f0 2023-10-14T10:37:00.242 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.236+0000 7f4265948700 1 -- 172.21.15.67:0/3112885446 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f42600cc6f0 msgr2=0x7f42600ccf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:00.242 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.236+0000 7f4265948700 1 --2- 172.21.15.67:0/3112885446 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f42600cc6f0 0x7f42600ccf00 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f4260070580 tx=0x7f425800b330 comp rx=0 tx=0).stop 2023-10-14T10:37:00.242 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.236+0000 7f4265948700 1 -- 172.21.15.67:0/3112885446 shutdown_connections 2023-10-14T10:37:00.242 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.236+0000 7f4265948700 1 --2- 172.21.15.67:0/3112885446 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f42600cc6f0 0x7f42600ccf00 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:00.242 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.236+0000 7f4265948700 1 --2- 172.21.15.67:0/3112885446 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f426015b6c0 0x7f426015f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:00.243 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.236+0000 7f4265948700 1 --2- 172.21.15.67:0/3112885446 >> v2:172.21.15.72:3300/0 conn(0x7f426015f930 0x7f42600cc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:00.243 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.236+0000 7f4265948700 1 -- 172.21.15.67:0/3112885446 >> 172.21.15.67:0/3112885446 conn(0x7f4260099a30 msgr2=0x7f42600d0530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:37:00.243 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.236+0000 7f4265948700 1 -- 172.21.15.67:0/3112885446 shutdown_connections 2023-10-14T10:37:00.243 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.236+0000 7f4265948700 1 -- 172.21.15.67:0/3112885446 wait complete. 2023-10-14T10:37:00.246 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.237+0000 7f4265948700 1 Processor -- start 2023-10-14T10:37:00.246 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.237+0000 7f4265948700 1 -- start start 2023-10-14T10:37:00.246 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.238+0000 7f4265948700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f42600cc6f0 0x7f4260156450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:00.246 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.238+0000 7f4265948700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f426015b6c0 0x7f42600c9950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:00.247 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.238+0000 7f4265948700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f426015f930 0x7f42600b1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:00.247 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.238+0000 7f4265948700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4260156990 con 0x7f426015f930 2023-10-14T10:37:00.247 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.238+0000 7f4265948700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42600ca2c0 con 0x7f426015b6c0 2023-10-14T10:37:00.247 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.238+0000 7f4265948700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42600c87d0 con 0x7f42600cc6f0 2023-10-14T10:37:00.247 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.238+0000 7f425f7fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f426015f930 0x7f42600b1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:00.248 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.238+0000 7f425f7fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f426015f930 0x7f42600b1fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45754/0 (socket says 172.21.15.67:45754) 2023-10-14T10:37:00.248 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.238+0000 7f425f7fe700 1 -- 172.21.15.67:0/149037867 learned_addr learned my addr 172.21.15.67:0/149037867 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:37:00.248 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.238+0000 7f425e7fc700 1 --2- 172.21.15.67:0/149037867 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f426015b6c0 0x7f42600c9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:00.248 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.238+0000 7f425effd700 1 --2- 172.21.15.67:0/149037867 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f42600cc6f0 0x7f4260156450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:00.248 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.239+0000 7f425f7fe700 1 -- 172.21.15.67:0/149037867 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f42600cc6f0 msgr2=0x7f4260156450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:00.249 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.239+0000 7f425f7fe700 1 --2- 172.21.15.67:0/149037867 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f42600cc6f0 0x7f4260156450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:00.249 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.239+0000 7f425f7fe700 1 -- 172.21.15.67:0/149037867 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f426015b6c0 msgr2=0x7f42600c9950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:00.249 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.239+0000 7f425f7fe700 1 --2- 172.21.15.67:0/149037867 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f426015b6c0 0x7f42600c9950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:00.249 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.239+0000 7f425f7fe700 1 -- 172.21.15.67:0/149037867 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4258009b30 con 0x7f426015f930 2023-10-14T10:37:00.249 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.239+0000 7f425f7fe700 1 --2- 172.21.15.67:0/149037867 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f426015f930 0x7f42600b1fc0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f42600ce0b0 tx=0x7f4258004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:00.250 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.239+0000 7f4247fff700 1 -- 172.21.15.67:0/149037867 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42580099b0 con 0x7f426015f930 2023-10-14T10:37:00.250 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.239+0000 7f4247fff700 1 -- 172.21.15.67:0/149037867 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f425800ce80 con 0x7f426015f930 2023-10-14T10:37:00.250 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.239+0000 7f4265948700 1 -- 172.21.15.67:0/149037867 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42600c8ab0 con 0x7f426015f930 2023-10-14T10:37:00.250 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.239+0000 7f4247fff700 1 -- 172.21.15.67:0/149037867 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4258004490 con 0x7f426015f930 2023-10-14T10:37:00.250 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.239+0000 7f4265948700 1 -- 172.21.15.67:0/149037867 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42600c9e90 con 0x7f426015f930 2023-10-14T10:37:00.251 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.240+0000 7f4265948700 1 -- 172.21.15.67:0/149037867 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42600ca050 con 0x7f426015f930 2023-10-14T10:37:00.251 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.240+0000 7f4247fff700 1 -- 172.21.15.67:0/149037867 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f42580045f0 con 0x7f426015f930 2023-10-14T10:37:00.251 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.241+0000 7f4247fff700 1 --2- 172.21.15.67:0/149037867 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f424c06af00 0x7f424c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:00.251 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.241+0000 7f4247fff700 1 -- 172.21.15.67:0/149037867 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5729+0+0 (secure 0 0 0) 0x7f425808a560 con 0x7f426015f930 2023-10-14T10:37:00.251 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.241+0000 7f425effd700 1 --2- 172.21.15.67:0/149037867 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f424c06af00 0x7f424c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:00.252 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.241+0000 7f425effd700 1 --2- 172.21.15.67:0/149037867 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f424c06af00 0x7f424c06d3c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4250009b40 tx=0x7f42500091c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:00.252 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.245+0000 7f4247fff700 1 -- 172.21.15.67:0/149037867 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4258055600 con 0x7f426015f930 2023-10-14T10:37:00.445 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.443+0000 7f4265948700 1 -- 172.21.15.67:0/149037867 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f426004ebb0 con 0x7f426015f930 2023-10-14T10:37:00.446 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.445+0000 7f4247fff700 1 -- 172.21.15.67:0/149037867 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+16554 (secure 0 0 0) 0x7f4258058c20 con 0x7f426015f930 2023-10-14T10:37:00.446 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-14T10:37:00.447 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":32,"fsid":"37f9ab46-3082-4b71-8b24-08d39897b008","created":"2023-10-14T10:35:46.067554+0000","modified":"2023-10-14T10:36:59.462557+0000","last_up_change":"2023-10-14T10:35:56.668142+0000","last_in_change":"2023-10-14T10:35:48.551490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-14T10:35:57.809566+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"testcephfs1_metadata","create_time":"2023-10-14T10:36:57.591719+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"testcephfs1"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"testcephfs1_data","create_time":"2023-10-14T10:36:58.436708+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"testcephfs1"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2b59a07e-11ef-440c-b766-9f64ac885838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6801","nonce":1287255499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6803","nonce":1287255499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6807","nonce":1287255499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6805","nonce":1287255499}]},"public_addr":"172.21.15.67:6801/1287255499","cluster_addr":"172.21.15.67:6803/1287255499","heartbeat_back_addr":"172.21.15.67:6807/1287255499","heartbeat_front_addr":"172.21.15.67:6805/1287255499","state":["exists","up"]},{"osd":1,"uuid":"6f813bd6-fa0b-487d-a204-431eb1276a60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6818","nonce":2435538787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6822","nonce":2435538787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6830","nonce":2435538787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6826","nonce":2435538787}]},"public_addr":"172.21.15.67:6818/2435538787","cluster_addr":"172.21.15.67:6822/2435538787","heartbeat_back_addr":"172.21.15.67:6830/2435538787","heartbeat_front_addr":"172.21.15.67:6826/2435538787","state":["exists","up"]},{"osd":2,"uuid":"6af42c6a-1b59-4669-9c04-b6876788446d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6809","nonce":2827491791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6811","nonce":2827491791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6815","nonce":2827491791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6813","nonce":2827491791}]},"public_addr":"172.21.15.67:6809/2827491791","cluster_addr":"172.21.15.67:6811/2827491791","heartbeat_back_addr":"172.21.15.67:6815/2827491791","heartbeat_front_addr":"172.21.15.67:6813/2827491791","state":["exists","up"]},{"osd":3,"uuid":"1d889d76-a646-499d-bfff-ed5bae5434ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6819","nonce":1848001176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6821","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6823","nonce":1848001176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6831","nonce":1848001176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6825","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6827","nonce":1848001176}]},"public_addr":"172.21.15.67:6819/1848001176","cluster_addr":"172.21.15.67:6823/1848001176","heartbeat_back_addr":"172.21.15.67:6831/1848001176","heartbeat_front_addr":"172.21.15.67:6827/1848001176","state":["exists","up"]},{"osd":4,"uuid":"c3271dff-b437-4fdd-b13b-6e49a35473af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6804","nonce":1891230785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6808","nonce":1891230785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6815","nonce":1891230785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6811","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6813","nonce":1891230785}]},"public_addr":"172.21.15.72:6804/1891230785","cluster_addr":"172.21.15.72:6808/1891230785","heartbeat_back_addr":"172.21.15.72:6815/1891230785","heartbeat_front_addr":"172.21.15.72:6813/1891230785","state":["exists","up"]},{"osd":5,"uuid":"0337bb18-4b68-4eff-b528-a667275e6926","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6801","nonce":569686991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6803","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6805","nonce":569686991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6812","nonce":569686991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6807","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6809","nonce":569686991}]},"public_addr":"172.21.15.72:6801/569686991","cluster_addr":"172.21.15.72:6805/569686991","heartbeat_back_addr":"172.21.15.72:6812/569686991","heartbeat_front_addr":"172.21.15.72:6809/569686991","state":["exists","up"]},{"osd":6,"uuid":"8e92d9cf-0c8c-46eb-89a7-db2023ef1509","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6817","nonce":3767685934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6819","nonce":3767685934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6823","nonce":3767685934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6821","nonce":3767685934}]},"public_addr":"172.21.15.72:6817/3767685934","cluster_addr":"172.21.15.72:6819/3767685934","heartbeat_back_addr":"172.21.15.72:6823/3767685934","heartbeat_front_addr":"172.21.15.72:6821/3767685934","state":["exists","up"]},{"osd":7,"uuid":"3fd6aa7d-50a9-45c9-9561-5902a044a119","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6825","nonce":3567465926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6827","nonce":3567465926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6831","nonce":3567465926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6829","nonce":3567465926}]},"public_addr":"172.21.15.72:6825/3567465926","cluster_addr":"172.21.15.72:6827/3567465926","heartbeat_back_addr":"172.21.15.72:6831/3567465926","heartbeat_front_addr":"172.21.15.72:6829/3567465926","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.511466+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.795107+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.858674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.728895+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.794842+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.750064+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.732122+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.765185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-14T10:37:00.448 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.447+0000 7f4265948700 1 -- 172.21.15.67:0/149037867 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f424c06af00 msgr2=0x7f424c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:00.449 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.447+0000 7f4265948700 1 --2- 172.21.15.67:0/149037867 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f424c06af00 0x7f424c06d3c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4250009b40 tx=0x7f42500091c0 comp rx=0 tx=0).stop 2023-10-14T10:37:00.449 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.447+0000 7f4265948700 1 -- 172.21.15.67:0/149037867 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f426015f930 msgr2=0x7f42600b1fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:00.449 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.447+0000 7f4265948700 1 --2- 172.21.15.67:0/149037867 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f426015f930 0x7f42600b1fc0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f42600ce0b0 tx=0x7f4258004ad0 comp rx=0 tx=0).stop 2023-10-14T10:37:00.449 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.447+0000 7f4265948700 1 -- 172.21.15.67:0/149037867 shutdown_connections 2023-10-14T10:37:00.449 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.447+0000 7f4265948700 1 --2- 172.21.15.67:0/149037867 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f424c06af00 0x7f424c06d3c0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:00.450 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.447+0000 7f4265948700 1 --2- 172.21.15.67:0/149037867 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f426015f930 0x7f42600b1fc0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:00.450 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.447+0000 7f4265948700 1 --2- 172.21.15.67:0/149037867 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f42600cc6f0 0x7f4260156450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:00.450 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.447+0000 7f4265948700 1 --2- 172.21.15.67:0/149037867 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f426015b6c0 0x7f42600c9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:00.450 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.447+0000 7f4265948700 1 -- 172.21.15.67:0/149037867 >> 172.21.15.67:0/149037867 conn(0x7f4260099a30 msgr2=0x7f4260159630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:37:00.451 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.447+0000 7f4265948700 1 -- 172.21.15.67:0/149037867 shutdown_connections 2023-10-14T10:37:00.451 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.447+0000 7f4265948700 1 -- 172.21.15.67:0/149037867 wait complete. 2023-10-14T10:37:00.461 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-14T10:37:00.586 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.584+0000 7f3fabc61700 1 Processor -- start 2023-10-14T10:37:00.587 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.584+0000 7f3fabc61700 1 -- start start 2023-10-14T10:37:00.590 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.585+0000 7f3fabc61700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f3fa40cd870 0x7f3fa40cc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:00.591 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.585+0000 7f3fabc61700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fa40afd30 con 0x7f3fa40af740 2023-10-14T10:37:00.591 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.585+0000 7f3fabc61700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fa40cce20 con 0x7f3fa40b1c00 2023-10-14T10:37:00.591 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.585+0000 7f3fabc61700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fa4161cd0 con 0x7f3fa40cd870 2023-10-14T10:37:00.591 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.585+0000 7f3fa91fc700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f3fa40af740 0x7f3fa40b14f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:53526/0 (socket says 172.21.15.67:53526) 2023-10-14T10:37:00.592 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.585+0000 7f3fa91fc700 1 -- 172.21.15.67:0/991865702 learned_addr learned my addr 172.21.15.67:0/991865702 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:37:00.592 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.585+0000 7f3fa99fd700 1 --2- 172.21.15.67:0/991865702 >> v2:172.21.15.72:3300/0 conn(0x7f3fa40cd870 0x7f3fa40cc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:00.592 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.586+0000 7f3fa89fb700 1 -- 172.21.15.67:0/991865702 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3603150232 0 0) 0x7f3fa40afd30 con 0x7f3fa40af740 2023-10-14T10:37:00.592 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.586+0000 7f3fa89fb700 1 -- 172.21.15.67:0/991865702 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f900035b0 con 0x7f3fa40af740 2023-10-14T10:37:00.593 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.586+0000 7f3fa89fb700 1 -- 172.21.15.67:0/991865702 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2610831365 0 0) 0x7f3f900035b0 con 0x7f3fa40af740 2023-10-14T10:37:00.593 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.586+0000 7f3fa89fb700 1 -- 172.21.15.67:0/991865702 >> v2:172.21.15.72:3300/0 conn(0x7f3fa40cd870 msgr2=0x7f3fa40cc8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:37:00.593 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.586+0000 7f3fa89fb700 1 --2- 172.21.15.67:0/991865702 >> v2:172.21.15.72:3300/0 conn(0x7f3fa40cd870 0x7f3fa40cc8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:00.593 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.586+0000 7f3fa89fb700 1 -- 172.21.15.67:0/991865702 >> v1:172.21.15.72:6789/0 conn(0x7f3fa40b1c00 legacy=0x7f3fa40b01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:00.594 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.586+0000 7f3fa89fb700 1 -- 172.21.15.67:0/991865702 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fa4161fb0 con 0x7f3fa40af740 2023-10-14T10:37:00.594 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.586+0000 7f3fa89fb700 1 -- 172.21.15.67:0/991865702 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3180043755 0 0) 0x7f3f94001670 con 0x7f3fa40af740 2023-10-14T10:37:00.594 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.586+0000 7f3fa89fb700 1 -- 172.21.15.67:0/991865702 >> v1:172.21.15.67:6789/0 conn(0x7f3fa40af740 legacy=0x7f3fa40b14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:00.594 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.586+0000 7f3fa89fb700 1 --2- 172.21.15.67:0/991865702 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f3f90003a00 0x7f3f90003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:00.594 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.586+0000 7f3fa89fb700 1 --2- 172.21.15.67:0/991865702 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3f90004850 0x7f3f90004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:00.595 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.586+0000 7f3fa89fb700 1 --2- 172.21.15.67:0/991865702 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3f90005210 0x7f3f9000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:00.595 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.586+0000 7f3fa89fb700 1 -- 172.21.15.67:0/991865702 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa4161cd0 con 0x7f3f90005210 2023-10-14T10:37:00.595 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.586+0000 7f3fa89fb700 1 -- 172.21.15.67:0/991865702 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f9000c0b0 con 0x7f3f90004850 2023-10-14T10:37:00.595 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.586+0000 7f3fa89fb700 1 -- 172.21.15.67:0/991865702 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f9000c270 con 0x7f3f90003a00 2023-10-14T10:37:00.596 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.586+0000 7f3faa1fe700 1 --2- 172.21.15.67:0/991865702 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3f90005210 0x7f3f9000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:00.596 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.586+0000 7f3fa91fc700 1 --2- 172.21.15.67:0/991865702 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3f90004850 0x7f3f90004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:00.596 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.586+0000 7f3fa99fd700 1 --2- 172.21.15.67:0/991865702 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f3f90003a00 0x7f3f90003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:00.596 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.586+0000 7f3faa1fe700 1 -- 172.21.15.67:0/991865702 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f3f90003a00 msgr2=0x7f3f90003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:37:00.597 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.586+0000 7f3faa1fe700 1 --2- 172.21.15.67:0/991865702 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f3f90003a00 0x7f3f90003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:00.597 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.586+0000 7f3faa1fe700 1 -- 172.21.15.67:0/991865702 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3f90004850 msgr2=0x7f3f90004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:37:00.597 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.586+0000 7f3faa1fe700 1 --2- 172.21.15.67:0/991865702 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3f90004850 0x7f3f90004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:00.598 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.586+0000 7f3faa1fe700 1 -- 172.21.15.67:0/991865702 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f900035b0 con 0x7f3f90005210 2023-10-14T10:37:00.598 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.586+0000 7f3faa1fe700 1 -- 172.21.15.67:0/991865702 reap_dead start 2023-10-14T10:37:00.598 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.587+0000 7f3faa1fe700 1 --2- 172.21.15.67:0/991865702 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3f90005210 0x7f3f9000ba10 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f3fa40cab30 tx=0x7f3f90005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15e04c3a99d85080 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:00.598 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.587+0000 7f3fa89fb700 1 -- 172.21.15.67:0/991865702 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f9c001db0 con 0x7f3f90005210 2023-10-14T10:37:00.599 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.587+0000 7f3fa89fb700 1 -- 172.21.15.67:0/991865702 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f9c00cc90 con 0x7f3f90005210 2023-10-14T10:37:00.599 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.587+0000 7f3fa89fb700 1 -- 172.21.15.67:0/991865702 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f9c003a00 con 0x7f3f90005210 2023-10-14T10:37:00.599 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.587+0000 7f3fabc61700 1 -- 172.21.15.67:0/991865702 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3f90005210 msgr2=0x7f3f9000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:00.599 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.587+0000 7f3fabc61700 1 --2- 172.21.15.67:0/991865702 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3f90005210 0x7f3f9000ba10 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f3fa40cab30 tx=0x7f3f90005880 comp rx=0 tx=0).stop 2023-10-14T10:37:00.599 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.587+0000 7f3fabc61700 1 -- 172.21.15.67:0/991865702 shutdown_connections 2023-10-14T10:37:00.600 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.587+0000 7f3fabc61700 1 --2- 172.21.15.67:0/991865702 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3f90005210 0x7f3f9000ba10 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:00.600 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.587+0000 7f3fabc61700 1 -- 172.21.15.67:0/991865702 >> 172.21.15.67:0/991865702 conn(0x7f3fa4099a30 msgr2=0x7f3fa40d0530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:37:00.600 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.587+0000 7f3fabc61700 1 -- 172.21.15.67:0/991865702 shutdown_connections 2023-10-14T10:37:00.600 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.587+0000 7f3fabc61700 1 -- 172.21.15.67:0/991865702 wait complete. 2023-10-14T10:37:00.600 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.589+0000 7f3fabc61700 1 Processor -- start 2023-10-14T10:37:00.601 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.589+0000 7f3fabc61700 1 -- start start 2023-10-14T10:37:00.604 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.589+0000 7f3fabc61700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3f90005210 0x7f3fa4150730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:00.604 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.589+0000 7f3fabc61700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f3fa40c5460 0x7f3fa40d0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:00.604 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.589+0000 7f3fabc61700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3fa40c58e0 0x7f3fa40d0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:00.604 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.589+0000 7f3fabc61700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa40b30a0 con 0x7f3f90005210 2023-10-14T10:37:00.604 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.589+0000 7f3fabc61700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa41601e0 con 0x7f3fa40c58e0 2023-10-14T10:37:00.605 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.589+0000 7f3fabc61700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa40b32f0 con 0x7f3fa40c5460 2023-10-14T10:37:00.605 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.590+0000 7f3fa99fd700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3f90005210 0x7f3fa4150730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:00.605 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.590+0000 7f3fa99fd700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3f90005210 0x7f3fa4150730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45770/0 (socket says 172.21.15.67:45770) 2023-10-14T10:37:00.605 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.590+0000 7f3fa99fd700 1 -- 172.21.15.67:0/3769775051 learned_addr learned my addr 172.21.15.67:0/3769775051 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:37:00.605 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.590+0000 7f3faa1fe700 1 --2- 172.21.15.67:0/3769775051 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3fa40c58e0 0x7f3fa40d0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:00.606 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.590+0000 7f3fa91fc700 1 --2- 172.21.15.67:0/3769775051 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f3fa40c5460 0x7f3fa40d0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:00.606 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.590+0000 7f3fa99fd700 1 -- 172.21.15.67:0/3769775051 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f3fa40c5460 msgr2=0x7f3fa40d0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:00.606 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.590+0000 7f3fa99fd700 1 --2- 172.21.15.67:0/3769775051 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f3fa40c5460 0x7f3fa40d0420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:00.606 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.590+0000 7f3fa99fd700 1 -- 172.21.15.67:0/3769775051 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3fa40c58e0 msgr2=0x7f3fa40d0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:00.607 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.590+0000 7f3fa99fd700 1 --2- 172.21.15.67:0/3769775051 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3fa40c58e0 0x7f3fa40d0960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:00.607 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.590+0000 7f3fa99fd700 1 -- 172.21.15.67:0/3769775051 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fa40b35d0 con 0x7f3f90005210 2023-10-14T10:37:00.607 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.590+0000 7f3fa91fc700 1 --2- 172.21.15.67:0/3769775051 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f3fa40c5460 0x7f3fa40d0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-14T10:37:00.607 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.590+0000 7f3faa1fe700 1 --2- 172.21.15.67:0/3769775051 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3fa40c58e0 0x7f3fa40d0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-14T10:37:00.607 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.590+0000 7f3fa99fd700 1 --2- 172.21.15.67:0/3769775051 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3f90005210 0x7f3fa4150730 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f3fa0009fd0 tx=0x7f3fa0011d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:00.607 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.590+0000 7f3f9a7fc700 1 -- 172.21.15.67:0/3769775051 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa0009610 con 0x7f3f90005210 2023-10-14T10:37:00.608 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.590+0000 7f3f9a7fc700 1 -- 172.21.15.67:0/3769775051 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fa0005c30 con 0x7f3f90005210 2023-10-14T10:37:00.608 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.590+0000 7f3fabc61700 1 -- 172.21.15.67:0/3769775051 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fa40b38b0 con 0x7f3f90005210 2023-10-14T10:37:00.608 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.590+0000 7f3f9a7fc700 1 -- 172.21.15.67:0/3769775051 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa000f730 con 0x7f3f90005210 2023-10-14T10:37:00.608 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.590+0000 7f3fabc61700 1 -- 172.21.15.67:0/3769775051 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fa40b3dd0 con 0x7f3f90005210 2023-10-14T10:37:00.609 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.591+0000 7f3fabc61700 1 -- 172.21.15.67:0/3769775051 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fa40b3c20 con 0x7f3f90005210 2023-10-14T10:37:00.609 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.592+0000 7f3f9a7fc700 1 -- 172.21.15.67:0/3769775051 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f3fa000f890 con 0x7f3f90005210 2023-10-14T10:37:00.609 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.592+0000 7f3f9a7fc700 1 --2- 172.21.15.67:0/3769775051 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f3f8806af00 0x7f3f8806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:00.609 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.592+0000 7f3f9a7fc700 1 -- 172.21.15.67:0/3769775051 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5729+0+0 (secure 0 0 0) 0x7f3fa0089d90 con 0x7f3f90005210 2023-10-14T10:37:00.609 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.592+0000 7f3fa91fc700 1 --2- 172.21.15.67:0/3769775051 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f3f8806af00 0x7f3f8806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:00.609 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.593+0000 7f3fa91fc700 1 --2- 172.21.15.67:0/3769775051 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f3f8806af00 0x7f3f8806d3c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3fa40cf010 tx=0x7f3f94009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:00.610 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.597+0000 7f3f9a7fc700 1 -- 172.21.15.67:0/3769775051 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3fa0054eb0 con 0x7f3f90005210 2023-10-14T10:37:00.801 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.799+0000 7f3fabc61700 1 -- 172.21.15.67:0/3769775051 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3fa404ebb0 con 0x7f3f90005210 2023-10-14T10:37:00.802 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.800+0000 7f3f9a7fc700 1 -- 172.21.15.67:0/3769775051 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+16554 (secure 0 0 0) 0x7f3fa00584d0 con 0x7f3f90005210 2023-10-14T10:37:00.803 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-14T10:37:00.803 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":32,"fsid":"37f9ab46-3082-4b71-8b24-08d39897b008","created":"2023-10-14T10:35:46.067554+0000","modified":"2023-10-14T10:36:59.462557+0000","last_up_change":"2023-10-14T10:35:56.668142+0000","last_in_change":"2023-10-14T10:35:48.551490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-14T10:35:57.809566+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"testcephfs1_metadata","create_time":"2023-10-14T10:36:57.591719+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"testcephfs1"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"testcephfs1_data","create_time":"2023-10-14T10:36:58.436708+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"testcephfs1"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2b59a07e-11ef-440c-b766-9f64ac885838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6801","nonce":1287255499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6803","nonce":1287255499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6807","nonce":1287255499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6805","nonce":1287255499}]},"public_addr":"172.21.15.67:6801/1287255499","cluster_addr":"172.21.15.67:6803/1287255499","heartbeat_back_addr":"172.21.15.67:6807/1287255499","heartbeat_front_addr":"172.21.15.67:6805/1287255499","state":["exists","up"]},{"osd":1,"uuid":"6f813bd6-fa0b-487d-a204-431eb1276a60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6818","nonce":2435538787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6822","nonce":2435538787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6830","nonce":2435538787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6826","nonce":2435538787}]},"public_addr":"172.21.15.67:6818/2435538787","cluster_addr":"172.21.15.67:6822/2435538787","heartbeat_back_addr":"172.21.15.67:6830/2435538787","heartbeat_front_addr":"172.21.15.67:6826/2435538787","state":["exists","up"]},{"osd":2,"uuid":"6af42c6a-1b59-4669-9c04-b6876788446d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6809","nonce":2827491791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6811","nonce":2827491791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6815","nonce":2827491791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6813","nonce":2827491791}]},"public_addr":"172.21.15.67:6809/2827491791","cluster_addr":"172.21.15.67:6811/2827491791","heartbeat_back_addr":"172.21.15.67:6815/2827491791","heartbeat_front_addr":"172.21.15.67:6813/2827491791","state":["exists","up"]},{"osd":3,"uuid":"1d889d76-a646-499d-bfff-ed5bae5434ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6819","nonce":1848001176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6821","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6823","nonce":1848001176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6831","nonce":1848001176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6825","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6827","nonce":1848001176}]},"public_addr":"172.21.15.67:6819/1848001176","cluster_addr":"172.21.15.67:6823/1848001176","heartbeat_back_addr":"172.21.15.67:6831/1848001176","heartbeat_front_addr":"172.21.15.67:6827/1848001176","state":["exists","up"]},{"osd":4,"uuid":"c3271dff-b437-4fdd-b13b-6e49a35473af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6804","nonce":1891230785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6808","nonce":1891230785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6815","nonce":1891230785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6811","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6813","nonce":1891230785}]},"public_addr":"172.21.15.72:6804/1891230785","cluster_addr":"172.21.15.72:6808/1891230785","heartbeat_back_addr":"172.21.15.72:6815/1891230785","heartbeat_front_addr":"172.21.15.72:6813/1891230785","state":["exists","up"]},{"osd":5,"uuid":"0337bb18-4b68-4eff-b528-a667275e6926","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6801","nonce":569686991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6803","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6805","nonce":569686991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6812","nonce":569686991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6807","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6809","nonce":569686991}]},"public_addr":"172.21.15.72:6801/569686991","cluster_addr":"172.21.15.72:6805/569686991","heartbeat_back_addr":"172.21.15.72:6812/569686991","heartbeat_front_addr":"172.21.15.72:6809/569686991","state":["exists","up"]},{"osd":6,"uuid":"8e92d9cf-0c8c-46eb-89a7-db2023ef1509","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6817","nonce":3767685934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6819","nonce":3767685934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6823","nonce":3767685934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6821","nonce":3767685934}]},"public_addr":"172.21.15.72:6817/3767685934","cluster_addr":"172.21.15.72:6819/3767685934","heartbeat_back_addr":"172.21.15.72:6823/3767685934","heartbeat_front_addr":"172.21.15.72:6821/3767685934","state":["exists","up"]},{"osd":7,"uuid":"3fd6aa7d-50a9-45c9-9561-5902a044a119","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6825","nonce":3567465926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6827","nonce":3567465926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6831","nonce":3567465926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6829","nonce":3567465926}]},"public_addr":"172.21.15.72:6825/3567465926","cluster_addr":"172.21.15.72:6827/3567465926","heartbeat_back_addr":"172.21.15.72:6831/3567465926","heartbeat_front_addr":"172.21.15.72:6829/3567465926","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.511466+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.795107+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.858674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.728895+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.794842+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.750064+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.732122+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.765185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-14T10:37:00.805 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.802+0000 7f3fabc61700 1 -- 172.21.15.67:0/3769775051 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f3f8806af00 msgr2=0x7f3f8806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:00.805 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.802+0000 7f3fabc61700 1 --2- 172.21.15.67:0/3769775051 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f3f8806af00 0x7f3f8806d3c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3fa40cf010 tx=0x7f3f94009040 comp rx=0 tx=0).stop 2023-10-14T10:37:00.805 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.802+0000 7f3fabc61700 1 -- 172.21.15.67:0/3769775051 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3f90005210 msgr2=0x7f3fa4150730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:00.806 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.802+0000 7f3fabc61700 1 --2- 172.21.15.67:0/3769775051 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3f90005210 0x7f3fa4150730 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f3fa0009fd0 tx=0x7f3fa0011d30 comp rx=0 tx=0).stop 2023-10-14T10:37:00.806 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.803+0000 7f3fabc61700 1 -- 172.21.15.67:0/3769775051 shutdown_connections 2023-10-14T10:37:00.806 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.803+0000 7f3fabc61700 1 --2- 172.21.15.67:0/3769775051 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f3f8806af00 0x7f3f8806d3c0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:00.806 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.803+0000 7f3fabc61700 1 --2- 172.21.15.67:0/3769775051 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3fa40c58e0 0x7f3fa40d0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:00.806 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.803+0000 7f3fabc61700 1 --2- 172.21.15.67:0/3769775051 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3f90005210 0x7f3fa4150730 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:00.807 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.803+0000 7f3fabc61700 1 --2- 172.21.15.67:0/3769775051 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f3fa40c5460 0x7f3fa40d0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:00.807 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.803+0000 7f3fabc61700 1 -- 172.21.15.67:0/3769775051 >> 172.21.15.67:0/3769775051 conn(0x7f3fa4099a30 msgr2=0x7f3fa406ebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:37:00.807 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.803+0000 7f3fabc61700 1 -- 172.21.15.67:0/3769775051 shutdown_connections 2023-10-14T10:37:00.807 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.803+0000 7f3fabc61700 1 -- 172.21.15.67:0/3769775051 wait complete. 2023-10-14T10:37:00.819 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set testcephfs1 standby_count_wanted 0 2023-10-14T10:37:00.947 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.945+0000 7f73c472c700 1 Processor -- start 2023-10-14T10:37:00.947 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.945+0000 7f73c472c700 1 -- start start 2023-10-14T10:37:00.949 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.946+0000 7f73c472c700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f73bc06af50 0x7f73bc0cede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:00.950 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.946+0000 7f73c472c700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f73bc0b28b0 0x7f73bc0b3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:00.950 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.946+0000 7f73c472c700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73bc0b3530 con 0x7f73bc15f130 2023-10-14T10:37:00.950 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.946+0000 7f73c472c700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73bc0cde00 con 0x7f73bc06af50 2023-10-14T10:37:00.950 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.946+0000 7f73c472c700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73bc0cd180 con 0x7f73bc0b28b0 2023-10-14T10:37:00.951 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.946+0000 7f73c1cc7700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f73bc06af50 0x7f73bc0cede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:00.951 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.946+0000 7f73c1cc7700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f73bc06af50 0x7f73bc0cede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45780/0 (socket says 172.21.15.67:45780) 2023-10-14T10:37:00.951 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.946+0000 7f73c1cc7700 1 -- 172.21.15.67:0/90702695 learned_addr learned my addr 172.21.15.67:0/90702695 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:37:00.951 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.946+0000 7f73c2cc9700 1 --2- 172.21.15.67:0/90702695 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f73bc0b28b0 0x7f73bc0b3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:00.951 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.946+0000 7f73c1cc7700 1 -- 172.21.15.67:0/90702695 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f73bc0b28b0 msgr2=0x7f73bc0b3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:37:00.952 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.946+0000 7f73c1cc7700 1 --2- 172.21.15.67:0/90702695 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f73bc0b28b0 0x7f73bc0b3960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:00.952 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.946+0000 7f73c1cc7700 1 -- 172.21.15.67:0/90702695 >> v1:172.21.15.67:6789/0 conn(0x7f73bc15f130 legacy=0x7f73bc06a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:00.952 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.946+0000 7f73c14c6700 1 -- 172.21.15.67:0/90702695 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4128124450 0 0) 0x7f73bc0b3530 con 0x7f73bc15f130 2023-10-14T10:37:00.952 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.946+0000 7f73c1cc7700 1 -- 172.21.15.67:0/90702695 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73bc0cd460 con 0x7f73bc06af50 2023-10-14T10:37:00.952 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.946+0000 7f73c1cc7700 1 --2- 172.21.15.67:0/90702695 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f73bc06af50 0x7f73bc0cede0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f73ac006fd0 tx=0x7f73ac009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e13142574d363d7 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:00.953 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.947+0000 7f73c14c6700 1 -- 172.21.15.67:0/90702695 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73ac005e30 con 0x7f73bc06af50 2023-10-14T10:37:00.953 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.947+0000 7f73c14c6700 1 -- 172.21.15.67:0/90702695 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73ac004030 con 0x7f73bc06af50 2023-10-14T10:37:00.953 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.947+0000 7f73c14c6700 1 -- 172.21.15.67:0/90702695 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73ac0043e0 con 0x7f73bc06af50 2023-10-14T10:37:00.953 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.947+0000 7f73c472c700 1 -- 172.21.15.67:0/90702695 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f73bc06af50 msgr2=0x7f73bc0cede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:00.954 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.947+0000 7f73c472c700 1 --2- 172.21.15.67:0/90702695 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f73bc06af50 0x7f73bc0cede0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f73ac006fd0 tx=0x7f73ac009bd0 comp rx=0 tx=0).stop 2023-10-14T10:37:00.954 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.947+0000 7f73c472c700 1 -- 172.21.15.67:0/90702695 shutdown_connections 2023-10-14T10:37:00.954 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.947+0000 7f73c472c700 1 --2- 172.21.15.67:0/90702695 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f73bc0b28b0 0x7f73bc0b3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:00.954 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.947+0000 7f73c472c700 1 --2- 172.21.15.67:0/90702695 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f73bc06af50 0x7f73bc0cede0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:00.954 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.947+0000 7f73c472c700 1 -- 172.21.15.67:0/90702695 >> 172.21.15.67:0/90702695 conn(0x7f73bc06dae0 msgr2=0x7f73bc06def0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:37:00.955 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.947+0000 7f73c472c700 1 -- 172.21.15.67:0/90702695 shutdown_connections 2023-10-14T10:37:00.955 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.947+0000 7f73c472c700 1 -- 172.21.15.67:0/90702695 wait complete. 2023-10-14T10:37:00.957 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.948+0000 7f73c472c700 1 Processor -- start 2023-10-14T10:37:00.958 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.948+0000 7f73c472c700 1 -- start start 2023-10-14T10:37:00.958 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.949+0000 7f73c472c700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f73bc06af50 0x7f73bc15c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:00.958 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.949+0000 7f73c472c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f73bc0b28b0 0x7f73bc0c61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:00.958 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.949+0000 7f73c472c700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f73bc15f130 0x7f73bc0c6710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:00.959 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.949+0000 7f73c472c700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73bc0cc980 con 0x7f73bc0b28b0 2023-10-14T10:37:00.959 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.949+0000 7f73c472c700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73bc0ad490 con 0x7f73bc15f130 2023-10-14T10:37:00.959 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.949+0000 7f73c472c700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73bc0ad770 con 0x7f73bc06af50 2023-10-14T10:37:00.959 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.949+0000 7f73c1cc7700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f73bc0b28b0 0x7f73bc0c61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:00.959 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.949+0000 7f73c1cc7700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f73bc0b28b0 0x7f73bc0c61d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45786/0 (socket says 172.21.15.67:45786) 2023-10-14T10:37:00.960 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.949+0000 7f73c1cc7700 1 -- 172.21.15.67:0/1123949158 learned_addr learned my addr 172.21.15.67:0/1123949158 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:37:00.960 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.950+0000 7f73c2cc9700 1 --2- 172.21.15.67:0/1123949158 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f73bc15f130 0x7f73bc0c6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:00.960 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.950+0000 7f73c24c8700 1 --2- 172.21.15.67:0/1123949158 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f73bc06af50 0x7f73bc15c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:00.960 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.950+0000 7f73c1cc7700 1 -- 172.21.15.67:0/1123949158 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f73bc06af50 msgr2=0x7f73bc15c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:00.960 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.950+0000 7f73c1cc7700 1 --2- 172.21.15.67:0/1123949158 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f73bc06af50 0x7f73bc15c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:00.961 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.950+0000 7f73c1cc7700 1 -- 172.21.15.67:0/1123949158 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f73bc15f130 msgr2=0x7f73bc0c6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:00.961 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.950+0000 7f73c1cc7700 1 --2- 172.21.15.67:0/1123949158 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f73bc15f130 0x7f73bc0c6710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:00.961 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.950+0000 7f73c1cc7700 1 -- 172.21.15.67:0/1123949158 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73ac009500 con 0x7f73bc0b28b0 2023-10-14T10:37:00.961 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.950+0000 7f73c2cc9700 1 --2- 172.21.15.67:0/1123949158 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f73bc15f130 0x7f73bc0c6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-14T10:37:00.961 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.950+0000 7f73c24c8700 1 --2- 172.21.15.67:0/1123949158 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f73bc06af50 0x7f73bc15c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-14T10:37:00.962 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.950+0000 7f73c1cc7700 1 --2- 172.21.15.67:0/1123949158 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f73bc0b28b0 0x7f73bc0c61d0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f73ac009130 tx=0x7f73ac004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:00.962 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.950+0000 7f73b2ffd700 1 -- 172.21.15.67:0/1123949158 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73ac003d90 con 0x7f73bc0b28b0 2023-10-14T10:37:00.962 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.950+0000 7f73b2ffd700 1 -- 172.21.15.67:0/1123949158 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73ac011420 con 0x7f73bc0b28b0 2023-10-14T10:37:00.962 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.950+0000 7f73b2ffd700 1 -- 172.21.15.67:0/1123949158 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73ac0117d0 con 0x7f73bc0b28b0 2023-10-14T10:37:00.963 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.950+0000 7f73c472c700 1 -- 172.21.15.67:0/1123949158 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73bc0ada50 con 0x7f73bc0b28b0 2023-10-14T10:37:00.963 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.950+0000 7f73c472c700 1 -- 172.21.15.67:0/1123949158 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73bc0add30 con 0x7f73bc0b28b0 2023-10-14T10:37:00.963 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.951+0000 7f73c472c700 1 -- 172.21.15.67:0/1123949158 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73bc0c8ac0 con 0x7f73bc0b28b0 2023-10-14T10:37:00.963 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.951+0000 7f73b2ffd700 1 -- 172.21.15.67:0/1123949158 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f73ac011930 con 0x7f73bc0b28b0 2023-10-14T10:37:00.963 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.952+0000 7f73b2ffd700 1 --2- 172.21.15.67:0/1123949158 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f73a006af00 0x7f73a006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:00.963 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.952+0000 7f73b2ffd700 1 -- 172.21.15.67:0/1123949158 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5729+0+0 (secure 0 0 0) 0x7f73ac08a7c0 con 0x7f73bc0b28b0 2023-10-14T10:37:00.964 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.952+0000 7f73c24c8700 1 --2- 172.21.15.67:0/1123949158 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f73a006af00 0x7f73a006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:00.964 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.952+0000 7f73c24c8700 1 --2- 172.21.15.67:0/1123949158 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f73a006af00 0x7f73a006d3c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f73b8000f60 tx=0x7f73b8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:00.964 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:00.956+0000 7f73b2ffd700 1 -- 172.21.15.67:0/1123949158 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f73bc0c8ac0 con 0x7f73bc0b28b0 2023-10-14T10:37:01.207 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:01.205+0000 7f73c472c700 1 -- 172.21.15.67:0/1123949158 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "testcephfs1", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f73bc0c8ea0 con 0x7f73bc0b28b0 2023-10-14T10:37:02.079 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:02.077+0000 7f73b2ffd700 1 -- 172.21.15.67:0/1123949158 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "testcephfs1", "var": "standby_count_wanted", "val": "0"}]=0 v15) v1 ==== 123+0+0 (secure 0 0 0) 0x7f73bc0c8ea0 con 0x7f73bc0b28b0 2023-10-14T10:37:02.082 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:02.080+0000 7f73c472c700 1 -- 172.21.15.67:0/1123949158 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f73a006af00 msgr2=0x7f73a006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:02.083 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:02.080+0000 7f73c472c700 1 --2- 172.21.15.67:0/1123949158 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f73a006af00 0x7f73a006d3c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f73b8000f60 tx=0x7f73b8009040 comp rx=0 tx=0).stop 2023-10-14T10:37:02.083 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:02.080+0000 7f73c472c700 1 -- 172.21.15.67:0/1123949158 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f73bc0b28b0 msgr2=0x7f73bc0c61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:02.083 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:02.080+0000 7f73c472c700 1 --2- 172.21.15.67:0/1123949158 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f73bc0b28b0 0x7f73bc0c61d0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f73ac009130 tx=0x7f73ac004750 comp rx=0 tx=0).stop 2023-10-14T10:37:02.083 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:02.080+0000 7f73c472c700 1 -- 172.21.15.67:0/1123949158 shutdown_connections 2023-10-14T10:37:02.083 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:02.080+0000 7f73c472c700 1 --2- 172.21.15.67:0/1123949158 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f73a006af00 0x7f73a006d3c0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:02.084 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:02.080+0000 7f73c472c700 1 --2- 172.21.15.67:0/1123949158 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f73bc15f130 0x7f73bc0c6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:02.084 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:02.080+0000 7f73c472c700 1 --2- 172.21.15.67:0/1123949158 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f73bc06af50 0x7f73bc15c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:02.084 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:02.080+0000 7f73c472c700 1 --2- 172.21.15.67:0/1123949158 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f73bc0b28b0 0x7f73bc0c61d0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:02.084 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:02.080+0000 7f73c472c700 1 -- 172.21.15.67:0/1123949158 >> 172.21.15.67:0/1123949158 conn(0x7f73bc06dae0 msgr2=0x7f73bc0aff30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:37:02.085 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:02.080+0000 7f73c472c700 1 -- 172.21.15.67:0/1123949158 shutdown_connections 2023-10-14T10:37:02.085 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:02.080+0000 7f73c472c700 1 -- 172.21.15.67:0/1123949158 wait complete. 2023-10-14T10:37:02.097 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-14T10:37:02.098 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set testcephfs1 allow_standby_replay false 2023-10-14T10:37:02.234 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:02.232+0000 7f0d02870700 1 Processor -- start 2023-10-14T10:37:02.235 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:02.232+0000 7f0d02870700 1 -- start start 2023-10-14T10:37:02.237 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:02.233+0000 7f0d02870700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f0cfc15ab10 0x7f0cfc15f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:02.238 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:02.233+0000 7f0d02870700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f0cfc0b1f50 0x7f0cfc0cf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:02.238 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:02.233+0000 7f0d02870700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f0cfc0b2400 0x7f0cfc0cfc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:02.238 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:02.233+0000 7f0d02870700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cfc15e7a0 con 0x7f0cfc15ab10 2023-10-14T10:37:02.238 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:02.233+0000 7f0d02870700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cfc15f730 con 0x7f0cfc0b1f50 2023-10-14T10:37:02.239 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:02.233+0000 7f0d02870700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cfc0b29a0 con 0x7f0cfc0b2400 2023-10-14T10:37:02.239 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:02.233+0000 7f0cfbfff700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f0cfc15ab10 0x7f0cfc15f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:02.239 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:02.233+0000 7f0cfbfff700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f0cfc15ab10 0x7f0cfc15f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45792/0 (socket says 172.21.15.67:45792) 2023-10-14T10:37:02.239 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:02.233+0000 7f0cfbfff700 1 -- 172.21.15.67:0/95250577 learned_addr learned my addr 172.21.15.67:0/95250577 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:37:02.240 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:02.233+0000 7f0cfb7fe700 1 --2- 172.21.15.67:0/95250577 >> v2:172.21.15.72:3300/0 conn(0x7f0cfc0b1f50 0x7f0cfc0cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:02.240 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:02.233+0000 7f0d00e0d700 1 --2- 172.21.15.67:0/95250577 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f0cfc0b2400 0x7f0cfc0cfc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:02.240 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:02.233+0000 7f0cfbfff700 1 -- 172.21.15.67:0/95250577 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f0cfc0b2400 msgr2=0x7f0cfc0cfc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:37:02.240 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:02.233+0000 7f0cfbfff700 1 --2- 172.21.15.67:0/95250577 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f0cfc0b2400 0x7f0cfc0cfc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:02.240 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:02.233+0000 7f0cfbfff700 1 -- 172.21.15.67:0/95250577 >> v2:172.21.15.72:3300/0 conn(0x7f0cfc0b1f50 msgr2=0x7f0cfc0cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:37:02.241 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:02.233+0000 7f0cfbfff700 1 --2- 172.21.15.67:0/95250577 >> v2:172.21.15.72:3300/0 conn(0x7f0cfc0b1f50 0x7f0cfc0cf720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:02.241 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:02.233+0000 7f0cfbfff700 1 -- 172.21.15.67:0/95250577 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cfc0b2c80 con 0x7f0cfc15ab10 2023-10-14T10:37:02.241 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:02.233+0000 7f0d00e0d700 1 --2- 172.21.15.67:0/95250577 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f0cfc0b2400 0x7f0cfc0cfc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-14T10:37:02.241 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:02.234+0000 7f0cfbfff700 1 --2- 172.21.15.67:0/95250577 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0cfc15ab10 0x7f0cfc15f1f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc06d470 tx=0x7f0ce4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35fac022eace2dc9 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:02.241 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:02.234+0000 7f0cfaffd700 1 -- 172.21.15.67:0/95250577 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce400b870 con 0x7f0cfc15ab10 2023-10-14T10:37:02.242 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:02.234+0000 7f0cfaffd700 1 -- 172.21.15.67:0/95250577 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ce400b9d0 con 0x7f0cfc15ab10 2023-10-14T10:37:02.242 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:02.234+0000 7f0cfaffd700 1 -- 172.21.15.67:0/95250577 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce400bd60 con 0x7f0cfc15ab10 2023-10-14T10:37:02.242 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:02.234+0000 7f0d02870700 1 -- 172.21.15.67:0/95250577 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0cfc15ab10 msgr2=0x7f0cfc15f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:02.242 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:02.234+0000 7f0d02870700 1 --2- 172.21.15.67:0/95250577 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0cfc15ab10 0x7f0cfc15f1f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc06d470 tx=0x7f0ce4009bd0 comp rx=0 tx=0).stop 2023-10-14T10:37:02.242 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:02.234+0000 7f0d02870700 1 -- 172.21.15.67:0/95250577 shutdown_connections 2023-10-14T10:37:02.243 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:02.234+0000 7f0d02870700 1 --2- 172.21.15.67:0/95250577 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f0cfc0b2400 0x7f0cfc0cfc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:02.243 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:02.234+0000 7f0d02870700 1 --2- 172.21.15.67:0/95250577 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0cfc15ab10 0x7f0cfc15f1f0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:02.243 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:02.234+0000 7f0d02870700 1 --2- 172.21.15.67:0/95250577 >> v2:172.21.15.72:3300/0 conn(0x7f0cfc0b1f50 0x7f0cfc0cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:02.243 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:02.234+0000 7f0d02870700 1 -- 172.21.15.67:0/95250577 >> 172.21.15.67:0/95250577 conn(0x7f0cfc06dae0 msgr2=0x7f0cfc06def0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:37:02.244 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:02.234+0000 7f0d02870700 1 -- 172.21.15.67:0/95250577 shutdown_connections 2023-10-14T10:37:02.244 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:02.234+0000 7f0d02870700 1 -- 172.21.15.67:0/95250577 wait complete. 2023-10-14T10:37:02.244 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:02.236+0000 7f0d02870700 1 Processor -- start 2023-10-14T10:37:02.244 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:02.236+0000 7f0d02870700 1 -- start start 2023-10-14T10:37:02.247 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:02.236+0000 7f0d02870700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f0cfc0b1f50 0x7f0cfc15c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:02.247 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:02.236+0000 7f0d02870700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0cfc0b2400 0x7f0cfc15aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:02.247 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:02.236+0000 7f0d02870700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0cfc15ab10 0x7f0cfc15d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:02.247 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:02.237+0000 7f0d02870700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cfc15c170 con 0x7f0cfc15ab10 2023-10-14T10:37:02.248 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:02.237+0000 7f0d02870700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cfc15de10 con 0x7f0cfc0b2400 2023-10-14T10:37:02.248 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:02.237+0000 7f0d02870700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cfc15b560 con 0x7f0cfc0b1f50 2023-10-14T10:37:02.248 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:02.237+0000 7f0d00e0d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0cfc15ab10 0x7f0cfc15d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:02.248 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:02.237+0000 7f0d00e0d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0cfc15ab10 0x7f0cfc15d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45800/0 (socket says 172.21.15.67:45800) 2023-10-14T10:37:02.248 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:02.237+0000 7f0d00e0d700 1 -- 172.21.15.67:0/2363259883 learned_addr learned my addr 172.21.15.67:0/2363259883 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:37:02.249 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:02.237+0000 7f0cfbfff700 1 --2- 172.21.15.67:0/2363259883 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f0cfc0b1f50 0x7f0cfc15c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:02.249 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:02.237+0000 7f0cfb7fe700 1 --2- 172.21.15.67:0/2363259883 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0cfc0b2400 0x7f0cfc15aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:02.249 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:02.237+0000 7f0d00e0d700 1 -- 172.21.15.67:0/2363259883 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f0cfc0b1f50 msgr2=0x7f0cfc15c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:02.249 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:02.237+0000 7f0d00e0d700 1 --2- 172.21.15.67:0/2363259883 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f0cfc0b1f50 0x7f0cfc15c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:02.249 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:02.237+0000 7f0d00e0d700 1 -- 172.21.15.67:0/2363259883 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0cfc0b2400 msgr2=0x7f0cfc15aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:02.250 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:02.237+0000 7f0d00e0d700 1 --2- 172.21.15.67:0/2363259883 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0cfc0b2400 0x7f0cfc15aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:02.250 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:02.237+0000 7f0d00e0d700 1 -- 172.21.15.67:0/2363259883 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ce4009550 con 0x7f0cfc15ab10 2023-10-14T10:37:02.250 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:02.237+0000 7f0d00e0d700 1 --2- 172.21.15.67:0/2363259883 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0cfc15ab10 0x7f0cfc15d120 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f0cf400cd50 tx=0x7f0cf400a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:02.250 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:02.237+0000 7f0cf8ff9700 1 -- 172.21.15.67:0/2363259883 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf400cea0 con 0x7f0cfc15ab10 2023-10-14T10:37:02.250 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:02.237+0000 7f0cf8ff9700 1 -- 172.21.15.67:0/2363259883 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cf400f480 con 0x7f0cfc15ab10 2023-10-14T10:37:02.251 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:02.237+0000 7f0d02870700 1 -- 172.21.15.67:0/2363259883 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cfc15b840 con 0x7f0cfc15ab10 2023-10-14T10:37:02.251 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:02.237+0000 7f0cf8ff9700 1 -- 172.21.15.67:0/2363259883 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf400f830 con 0x7f0cfc15ab10 2023-10-14T10:37:02.251 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:02.237+0000 7f0d02870700 1 -- 172.21.15.67:0/2363259883 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cfc15be00 con 0x7f0cfc15ab10 2023-10-14T10:37:02.251 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:02.238+0000 7f0d02870700 1 -- 172.21.15.67:0/2363259883 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cfc15bfc0 con 0x7f0cfc15ab10 2023-10-14T10:37:02.251 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:02.239+0000 7f0cf8ff9700 1 -- 172.21.15.67:0/2363259883 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f0cf400b040 con 0x7f0cfc15ab10 2023-10-14T10:37:02.252 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:02.240+0000 7f0cf8ff9700 1 --2- 172.21.15.67:0/2363259883 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f0cdc06af00 0x7f0cdc06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:02.252 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:02.240+0000 7f0cf8ff9700 1 -- 172.21.15.67:0/2363259883 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5729+0+0 (secure 0 0 0) 0x7f0cf4089fe0 con 0x7f0cfc15ab10 2023-10-14T10:37:02.252 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:02.240+0000 7f0cfbfff700 1 --2- 172.21.15.67:0/2363259883 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f0cdc06af00 0x7f0cdc06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:02.252 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:02.240+0000 7f0cfbfff700 1 --2- 172.21.15.67:0/2363259883 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f0cdc06af00 0x7f0cdc06d3c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0ce4000c00 tx=0x7f0ce4008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:02.252 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:02.244+0000 7f0cf8ff9700 1 -- 172.21.15.67:0/2363259883 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0cf40550e0 con 0x7f0cfc15ab10 2023-10-14T10:37:02.494 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:02.492+0000 7f0d02870700 1 -- 172.21.15.67:0/2363259883 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "testcephfs1", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f0cfc067d20 con 0x7f0cfc15ab10 2023-10-14T10:37:03.082 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.080+0000 7f0cf8ff9700 1 -- 172.21.15.67:0/2363259883 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "testcephfs1", "var": "allow_standby_replay", "val": "false"}]=0 v16) v1 ==== 127+0+0 (secure 0 0 0) 0x7f0cf4058700 con 0x7f0cfc15ab10 2023-10-14T10:37:03.085 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.083+0000 7f0d02870700 1 -- 172.21.15.67:0/2363259883 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f0cdc06af00 msgr2=0x7f0cdc06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:03.085 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.083+0000 7f0d02870700 1 --2- 172.21.15.67:0/2363259883 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f0cdc06af00 0x7f0cdc06d3c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0ce4000c00 tx=0x7f0ce4008030 comp rx=0 tx=0).stop 2023-10-14T10:37:03.086 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.083+0000 7f0d02870700 1 -- 172.21.15.67:0/2363259883 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0cfc15ab10 msgr2=0x7f0cfc15d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:03.086 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.083+0000 7f0d02870700 1 --2- 172.21.15.67:0/2363259883 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0cfc15ab10 0x7f0cfc15d120 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f0cf400cd50 tx=0x7f0cf400a560 comp rx=0 tx=0).stop 2023-10-14T10:37:03.086 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.083+0000 7f0d02870700 1 -- 172.21.15.67:0/2363259883 shutdown_connections 2023-10-14T10:37:03.086 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.083+0000 7f0d02870700 1 --2- 172.21.15.67:0/2363259883 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f0cdc06af00 0x7f0cdc06d3c0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:03.087 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.083+0000 7f0d02870700 1 --2- 172.21.15.67:0/2363259883 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0cfc15ab10 0x7f0cfc15d120 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:03.087 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.083+0000 7f0d02870700 1 --2- 172.21.15.67:0/2363259883 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f0cfc0b1f50 0x7f0cfc15c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:03.087 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.083+0000 7f0d02870700 1 --2- 172.21.15.67:0/2363259883 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0cfc0b2400 0x7f0cfc15aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:03.087 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.083+0000 7f0d02870700 1 -- 172.21.15.67:0/2363259883 >> 172.21.15.67:0/2363259883 conn(0x7f0cfc06dae0 msgr2=0x7f0cfc0b5040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:37:03.087 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.083+0000 7f0d02870700 1 -- 172.21.15.67:0/2363259883 shutdown_connections 2023-10-14T10:37:03.088 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.083+0000 7f0d02870700 1 -- 172.21.15.67:0/2363259883 wait complete. 2023-10-14T10:37:03.099 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'testcephfs1' 2023-10-14T10:37:03.099 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-14T10:37:03.227 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.225+0000 7f665124a700 1 Processor -- start 2023-10-14T10:37:03.228 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.225+0000 7f665124a700 1 -- start start 2023-10-14T10:37:03.230 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.226+0000 7f665124a700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f664c0d0c70 0x7f664c0cdfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:03.230 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.226+0000 7f665124a700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f664c0c5110 0x7f664c0ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:03.231 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.226+0000 7f665124a700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f664c0d23e0 con 0x7f664c15b740 2023-10-14T10:37:03.231 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.226+0000 7f665124a700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f664c15bb20 con 0x7f664c0d0c70 2023-10-14T10:37:03.231 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.226+0000 7f665124a700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f664c0cf190 con 0x7f664c0c5110 2023-10-14T10:37:03.231 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.226+0000 7f664a59c700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f664c0d0c70 0x7f664c0cdfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:03.231 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.226+0000 7f664b59e700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f664c0c5110 0x7f664c0ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:03.232 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.226+0000 7f664ad9d700 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f664c15b740 0x7f664c0d1cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.67:33582/0 (socket says 172.21.15.67:33582) 2023-10-14T10:37:03.232 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.227+0000 7f664b59e700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f664c0c5110 0x7f664c0ce510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3301/0 says I am v2:172.21.15.67:33860/0 (socket says 172.21.15.67:33860) 2023-10-14T10:37:03.232 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.227+0000 7f664ad9d700 1 -- 172.21.15.67:0/487257701 learned_addr learned my addr 172.21.15.67:0/487257701 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:37:03.232 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.227+0000 7f664a59c700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f664c0d0c70 0x7f664c0cdfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.67:40598/0 (socket says 172.21.15.67:40598) 2023-10-14T10:37:03.232 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.227+0000 7f664a59c700 1 -- 172.21.15.67:0/487257701 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f664c0c5110 msgr2=0x7f664c0ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:37:03.233 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.227+0000 7f664a59c700 1 --2- 172.21.15.67:0/487257701 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f664c0c5110 0x7f664c0ce510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:03.233 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.227+0000 7f664a59c700 1 -- 172.21.15.67:0/487257701 >> v1:172.21.15.72:6789/0 conn(0x7f664c15b740 legacy=0x7f664c0d1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:03.233 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.227+0000 7f664a59c700 1 -- 172.21.15.67:0/487257701 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f664c0cbd40 con 0x7f664c0d0c70 2023-10-14T10:37:03.233 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.227+0000 7f6649d9b700 1 -- 172.21.15.67:0/487257701 <== mon.1 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1224054295 0 0) 0x7f6640001110 con 0x7f664c15b740 2023-10-14T10:37:03.233 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.227+0000 7f664a59c700 1 --2- 172.21.15.67:0/487257701 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f664c0d0c70 0x7f664c0cdfd0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f663c009840 tx=0x7f663c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13fe4f78ea2e3a23 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:03.234 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.227+0000 7f6649d9b700 1 -- 172.21.15.67:0/487257701 <== mon.1 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f663c011830 con 0x7f664c0d0c70 2023-10-14T10:37:03.234 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.227+0000 7f6649d9b700 1 -- 172.21.15.67:0/487257701 <== mon.1 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f663c011990 con 0x7f664c0d0c70 2023-10-14T10:37:03.234 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.227+0000 7f6649d9b700 1 -- 172.21.15.67:0/487257701 <== mon.1 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f663c011d20 con 0x7f664c0d0c70 2023-10-14T10:37:03.234 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.228+0000 7f665124a700 1 -- 172.21.15.67:0/487257701 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f664c0d0c70 msgr2=0x7f664c0cdfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:03.234 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.228+0000 7f665124a700 1 --2- 172.21.15.67:0/487257701 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f664c0d0c70 0x7f664c0cdfd0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f663c009840 tx=0x7f663c009b50 comp rx=0 tx=0).stop 2023-10-14T10:37:03.235 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.228+0000 7f665124a700 1 -- 172.21.15.67:0/487257701 shutdown_connections 2023-10-14T10:37:03.235 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.228+0000 7f665124a700 1 --2- 172.21.15.67:0/487257701 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f664c0c5110 0x7f664c0ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:03.235 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.228+0000 7f665124a700 1 --2- 172.21.15.67:0/487257701 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f664c0d0c70 0x7f664c0cdfd0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:03.235 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.228+0000 7f665124a700 1 -- 172.21.15.67:0/487257701 >> 172.21.15.67:0/487257701 conn(0x7f664c0665a0 msgr2=0x7f664c066980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:37:03.235 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.228+0000 7f665124a700 1 -- 172.21.15.67:0/487257701 shutdown_connections 2023-10-14T10:37:03.235 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.228+0000 7f665124a700 1 -- 172.21.15.67:0/487257701 wait complete. 2023-10-14T10:37:03.238 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.229+0000 7f665124a700 1 Processor -- start 2023-10-14T10:37:03.238 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.229+0000 7f665124a700 1 -- start start 2023-10-14T10:37:03.239 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.230+0000 7f665124a700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f664c0c5110 0x7f664c1f3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:03.239 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.230+0000 7f665124a700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f664c0d0c70 0x7f664c1f0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:03.239 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.230+0000 7f665124a700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f664c15b740 0x7f664c1f1340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:03.239 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.230+0000 7f665124a700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f664c0b5a20 con 0x7f664c0d0c70 2023-10-14T10:37:03.239 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.230+0000 7f665124a700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f664c1563d0 con 0x7f664c15b740 2023-10-14T10:37:03.240 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.230+0000 7f665124a700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f664c0b4da0 con 0x7f664c0c5110 2023-10-14T10:37:03.240 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.230+0000 7f664a59c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f664c0d0c70 0x7f664c1f0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:03.240 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.230+0000 7f664a59c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f664c0d0c70 0x7f664c1f0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45814/0 (socket says 172.21.15.67:45814) 2023-10-14T10:37:03.240 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.230+0000 7f664a59c700 1 -- 172.21.15.67:0/1794069245 learned_addr learned my addr 172.21.15.67:0/1794069245 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:37:03.240 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.230+0000 7f664b59e700 1 --2- 172.21.15.67:0/1794069245 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f664c15b740 0x7f664c1f1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:03.241 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.230+0000 7f664ad9d700 1 --2- 172.21.15.67:0/1794069245 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f664c0c5110 0x7f664c1f3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:03.241 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.231+0000 7f664a59c700 1 -- 172.21.15.67:0/1794069245 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f664c0c5110 msgr2=0x7f664c1f3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:03.241 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.231+0000 7f664a59c700 1 --2- 172.21.15.67:0/1794069245 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f664c0c5110 0x7f664c1f3bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:03.241 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.231+0000 7f664a59c700 1 -- 172.21.15.67:0/1794069245 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f664c15b740 msgr2=0x7f664c1f1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:03.241 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.231+0000 7f664a59c700 1 --2- 172.21.15.67:0/1794069245 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f664c15b740 0x7f664c1f1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:03.242 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.231+0000 7f664a59c700 1 -- 172.21.15.67:0/1794069245 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f663c0094a0 con 0x7f664c0d0c70 2023-10-14T10:37:03.242 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.231+0000 7f664ad9d700 1 --2- 172.21.15.67:0/1794069245 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f664c0c5110 0x7f664c1f3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-14T10:37:03.242 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.231+0000 7f664a59c700 1 --2- 172.21.15.67:0/1794069245 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f664c0d0c70 0x7f664c1f0e00 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f663c009fd0 tx=0x7f663c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:03.242 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.231+0000 7f66337fe700 1 -- 172.21.15.67:0/1794069245 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f663c0036a0 con 0x7f664c0d0c70 2023-10-14T10:37:03.242 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.231+0000 7f66337fe700 1 -- 172.21.15.67:0/1794069245 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f663c01f070 con 0x7f664c0d0c70 2023-10-14T10:37:03.243 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.231+0000 7f665124a700 1 -- 172.21.15.67:0/1794069245 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f664c0b5080 con 0x7f664c0d0c70 2023-10-14T10:37:03.243 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.231+0000 7f66337fe700 1 -- 172.21.15.67:0/1794069245 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f663c003a20 con 0x7f664c0d0c70 2023-10-14T10:37:03.243 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.231+0000 7f665124a700 1 -- 172.21.15.67:0/1794069245 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f664c0b5640 con 0x7f664c0d0c70 2023-10-14T10:37:03.243 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.232+0000 7f665124a700 1 -- 172.21.15.67:0/1794069245 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f664c0b5b60 con 0x7f664c0d0c70 2023-10-14T10:37:03.243 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.232+0000 7f66337fe700 1 -- 172.21.15.67:0/1794069245 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f663c003b80 con 0x7f664c0d0c70 2023-10-14T10:37:03.244 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.233+0000 7f66337fe700 1 --2- 172.21.15.67:0/1794069245 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f663806af00 0x7f663806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:03.244 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.233+0000 7f66337fe700 1 -- 172.21.15.67:0/1794069245 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5729+0+0 (secure 0 0 0) 0x7f663c021070 con 0x7f664c0d0c70 2023-10-14T10:37:03.244 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.233+0000 7f664ad9d700 1 --2- 172.21.15.67:0/1794069245 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f663806af00 0x7f663806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:03.244 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.234+0000 7f664ad9d700 1 --2- 172.21.15.67:0/1794069245 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f663806af00 0x7f663806d3c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6640000f60 tx=0x7f6640007d50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:03.245 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.237+0000 7f66337fe700 1 -- 172.21.15.67:0/1794069245 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f663c056020 con 0x7f664c0d0c70 2023-10-14T10:37:03.484 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.482+0000 7f665124a700 1 -- 172.21.15.67:0/1794069245 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f664c1f0c30 con 0x7f664c0d0c70 2023-10-14T10:37:03.484 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.482+0000 7f66337fe700 1 -- 172.21.15.67:0/1794069245 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4918 (secure 0 0 0) 0x7f663c015090 con 0x7f664c0d0c70 2023-10-14T10:37:03.485 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-14T10:37:03.485 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6836/3371338659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":3371338659},{"type":"v1","addr":"172.21.15.72:6836","nonce":3371338659}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4688,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/2815272197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6835","nonce":2815272197},{"type":"v1","addr":"172.21.15.72:6837","nonce":2815272197}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5094,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6834/3494180260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":3494180260},{"type":"v1","addr":"172.21.15.67:6834","nonce":3494180260}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-14T10:36:59.462859+0000","modified":"2023-10-14T10:37:02.493817+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5097},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5097":{"gid":5097,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.67:6835/3065963596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6833","nonce":3065963596},{"type":"v1","addr":"172.21.15.67:6835","nonce":3065963596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"testcephfs1","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-14T10:37:03.486 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.484+0000 7f665124a700 1 -- 172.21.15.67:0/1794069245 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f663806af00 msgr2=0x7f663806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:03.487 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.484+0000 7f665124a700 1 --2- 172.21.15.67:0/1794069245 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f663806af00 0x7f663806d3c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6640000f60 tx=0x7f6640007d50 comp rx=0 tx=0).stop 2023-10-14T10:37:03.487 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.484+0000 7f665124a700 1 -- 172.21.15.67:0/1794069245 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f664c0d0c70 msgr2=0x7f664c1f0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:03.487 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.484+0000 7f665124a700 1 --2- 172.21.15.67:0/1794069245 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f664c0d0c70 0x7f664c1f0e00 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f663c009fd0 tx=0x7f663c004750 comp rx=0 tx=0).stop 2023-10-14T10:37:03.487 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.484+0000 7f665124a700 1 -- 172.21.15.67:0/1794069245 shutdown_connections 2023-10-14T10:37:03.488 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.484+0000 7f665124a700 1 --2- 172.21.15.67:0/1794069245 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f663806af00 0x7f663806d3c0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:03.488 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.484+0000 7f665124a700 1 --2- 172.21.15.67:0/1794069245 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f664c15b740 0x7f664c1f1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:03.488 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.484+0000 7f665124a700 1 --2- 172.21.15.67:0/1794069245 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f664c0c5110 0x7f664c1f3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:03.488 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.484+0000 7f665124a700 1 --2- 172.21.15.67:0/1794069245 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f664c0d0c70 0x7f664c1f0e00 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:03.489 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.484+0000 7f665124a700 1 -- 172.21.15.67:0/1794069245 >> 172.21.15.67:0/1794069245 conn(0x7f664c0665a0 msgr2=0x7f664c0c8720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:37:03.489 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.484+0000 7f665124a700 1 -- 172.21.15.67:0/1794069245 shutdown_connections 2023-10-14T10:37:03.489 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.484+0000 7f665124a700 1 -- 172.21.15.67:0/1794069245 wait complete. 2023-10-14T10:37:03.489 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 16 2023-10-14T10:37:03.499 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-14T10:37:03.624 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.622+0000 7fad1b838700 1 Processor -- start 2023-10-14T10:37:03.624 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.622+0000 7fad1b838700 1 -- start start 2023-10-14T10:37:03.627 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.623+0000 7fad1b838700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fad1415ab10 0x7fad1415f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:03.627 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.623+0000 7fad1b838700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fad140cfe30 0x7fad140cbf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:03.629 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.623+0000 7fad1b838700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad1415e7a0 con 0x7fad140b1f50 2023-10-14T10:37:03.629 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.623+0000 7fad1b838700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad1415f730 con 0x7fad140cfe30 2023-10-14T10:37:03.629 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.623+0000 7fad1b838700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad140b29a0 con 0x7fad1415ab10 2023-10-14T10:37:03.630 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.623+0000 7fad18dd3700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fad140b1f50 0x7fad140cf720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:53588/0 (socket says 172.21.15.67:53588) 2023-10-14T10:37:03.630 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.623+0000 7fad18dd3700 1 -- 172.21.15.67:0/1097259056 learned_addr learned my addr 172.21.15.67:0/1097259056 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:37:03.630 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.623+0000 7fad19dd5700 1 --2- 172.21.15.67:0/1097259056 >> v2:172.21.15.72:3300/0 conn(0x7fad140cfe30 0x7fad140cbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:03.630 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.623+0000 7fad195d4700 1 --2- 172.21.15.67:0/1097259056 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fad1415ab10 0x7fad1415f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:03.631 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.624+0000 7fad0bfff700 1 -- 172.21.15.67:0/1097259056 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 104002593 0 0) 0x7fad1415e7a0 con 0x7fad140b1f50 2023-10-14T10:37:03.631 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.624+0000 7fad0bfff700 1 -- 172.21.15.67:0/1097259056 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad000035b0 con 0x7fad140b1f50 2023-10-14T10:37:03.631 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.624+0000 7fad195d4700 1 -- 172.21.15.67:0/1097259056 >> v2:172.21.15.72:3300/0 conn(0x7fad140cfe30 msgr2=0x7fad140cbf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:37:03.631 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.624+0000 7fad195d4700 1 --2- 172.21.15.67:0/1097259056 >> v2:172.21.15.72:3300/0 conn(0x7fad140cfe30 0x7fad140cbf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:03.631 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.624+0000 7fad195d4700 1 -- 172.21.15.67:0/1097259056 >> v1:172.21.15.67:6789/0 conn(0x7fad140b1f50 legacy=0x7fad140cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:03.631 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.624+0000 7fad0bfff700 1 -- 172.21.15.67:0/1097259056 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1167193135 0 0) 0x7fad000035b0 con 0x7fad140b1f50 2023-10-14T10:37:03.632 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.624+0000 7fad195d4700 1 -- 172.21.15.67:0/1097259056 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad140b2c80 con 0x7fad1415ab10 2023-10-14T10:37:03.632 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.624+0000 7fad19dd5700 1 --2- 172.21.15.67:0/1097259056 >> v2:172.21.15.72:3300/0 conn(0x7fad140cfe30 0x7fad140cbf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-14T10:37:03.632 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.624+0000 7fad195d4700 1 --2- 172.21.15.67:0/1097259056 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fad1415ab10 0x7fad1415f1f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fad04009840 tx=0x7fad04009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2cf5f5495b9e352a server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:03.632 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.624+0000 7fad0bfff700 1 -- 172.21.15.67:0/1097259056 <== mon.2 v2:172.21.15.72:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad100017f0 con 0x7fad1415ab10 2023-10-14T10:37:03.632 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.624+0000 7fad0bfff700 1 -- 172.21.15.67:0/1097259056 <== mon.2 v2:172.21.15.72:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad04004030 con 0x7fad1415ab10 2023-10-14T10:37:03.633 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.624+0000 7fad0bfff700 1 -- 172.21.15.67:0/1097259056 <== mon.2 v2:172.21.15.72:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad040043c0 con 0x7fad1415ab10 2023-10-14T10:37:03.633 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.625+0000 7fad1b838700 1 -- 172.21.15.67:0/1097259056 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fad1415ab10 msgr2=0x7fad1415f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:03.633 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.625+0000 7fad1b838700 1 --2- 172.21.15.67:0/1097259056 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fad1415ab10 0x7fad1415f1f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fad04009840 tx=0x7fad04009b50 comp rx=0 tx=0).stop 2023-10-14T10:37:03.633 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.625+0000 7fad1b838700 1 -- 172.21.15.67:0/1097259056 shutdown_connections 2023-10-14T10:37:03.633 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.625+0000 7fad1b838700 1 --2- 172.21.15.67:0/1097259056 >> v2:172.21.15.72:3300/0 conn(0x7fad140cfe30 0x7fad140cbf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:03.634 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.625+0000 7fad1b838700 1 --2- 172.21.15.67:0/1097259056 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fad1415ab10 0x7fad1415f1f0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:03.634 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.625+0000 7fad1b838700 1 -- 172.21.15.67:0/1097259056 >> 172.21.15.67:0/1097259056 conn(0x7fad1406dae0 msgr2=0x7fad1406def0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:37:03.634 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.625+0000 7fad1b838700 1 -- 172.21.15.67:0/1097259056 shutdown_connections 2023-10-14T10:37:03.634 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.625+0000 7fad1b838700 1 -- 172.21.15.67:0/1097259056 wait complete. 2023-10-14T10:37:03.637 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.626+0000 7fad1b838700 1 Processor -- start 2023-10-14T10:37:03.637 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.626+0000 7fad1b838700 1 -- start start 2023-10-14T10:37:03.638 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.627+0000 7fad1b838700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fad140b1f50 0x7fad140b0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:03.638 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.627+0000 7fad1b838700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fad140cfe30 0x7fad140b4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:03.638 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.627+0000 7fad1b838700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fad1415ab10 0x7fad140b4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:03.638 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.627+0000 7fad1b838700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad140b32d0 con 0x7fad1415ab10 2023-10-14T10:37:03.638 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.627+0000 7fad1b838700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad140b51b0 con 0x7fad140b1f50 2023-10-14T10:37:03.639 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.627+0000 7fad1b838700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad140b5490 con 0x7fad140cfe30 2023-10-14T10:37:03.639 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.627+0000 7fad19dd5700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fad1415ab10 0x7fad140b4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:03.639 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.627+0000 7fad19dd5700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fad1415ab10 0x7fad140b4b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45826/0 (socket says 172.21.15.67:45826) 2023-10-14T10:37:03.639 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.627+0000 7fad19dd5700 1 -- 172.21.15.67:0/539148859 learned_addr learned my addr 172.21.15.67:0/539148859 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:37:03.639 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.627+0000 7fad18dd3700 1 --2- 172.21.15.67:0/539148859 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fad140cfe30 0x7fad140b4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:03.640 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.627+0000 7fad195d4700 1 --2- 172.21.15.67:0/539148859 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fad140b1f50 0x7fad140b0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:03.640 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.627+0000 7fad19dd5700 1 -- 172.21.15.67:0/539148859 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fad140cfe30 msgr2=0x7fad140b4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:03.640 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.627+0000 7fad19dd5700 1 --2- 172.21.15.67:0/539148859 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fad140cfe30 0x7fad140b4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:03.640 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.627+0000 7fad19dd5700 1 -- 172.21.15.67:0/539148859 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fad140b1f50 msgr2=0x7fad140b0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:03.640 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.627+0000 7fad19dd5700 1 --2- 172.21.15.67:0/539148859 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fad140b1f50 0x7fad140b0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:03.641 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.627+0000 7fad19dd5700 1 -- 172.21.15.67:0/539148859 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad0c009a70 con 0x7fad1415ab10 2023-10-14T10:37:03.641 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.627+0000 7fad18dd3700 1 --2- 172.21.15.67:0/539148859 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fad140cfe30 0x7fad140b4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-14T10:37:03.641 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.628+0000 7fad19dd5700 1 --2- 172.21.15.67:0/539148859 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fad1415ab10 0x7fad140b4b80 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fad0c00efc0 tx=0x7fad0c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:03.641 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.628+0000 7fad09ffb700 1 -- 172.21.15.67:0/539148859 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0c00ce00 con 0x7fad1415ab10 2023-10-14T10:37:03.642 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.628+0000 7fad09ffb700 1 -- 172.21.15.67:0/539148859 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad0c00f040 con 0x7fad1415ab10 2023-10-14T10:37:03.642 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.628+0000 7fad1b838700 1 -- 172.21.15.67:0/539148859 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad040094a0 con 0x7fad1415ab10 2023-10-14T10:37:03.642 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.628+0000 7fad09ffb700 1 -- 172.21.15.67:0/539148859 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0c0146c0 con 0x7fad1415ab10 2023-10-14T10:37:03.642 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.628+0000 7fad1b838700 1 -- 172.21.15.67:0/539148859 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad140b5a50 con 0x7fad1415ab10 2023-10-14T10:37:03.642 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.629+0000 7fad1b838700 1 -- 172.21.15.67:0/539148859 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad14067d20 con 0x7fad1415ab10 2023-10-14T10:37:03.643 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.630+0000 7fad09ffb700 1 -- 172.21.15.67:0/539148859 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fad0c014820 con 0x7fad1415ab10 2023-10-14T10:37:03.643 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.630+0000 7fad09ffb700 1 --2- 172.21.15.67:0/539148859 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7facf806af00 0x7facf806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:03.643 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.630+0000 7fad09ffb700 1 -- 172.21.15.67:0/539148859 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5729+0+0 (secure 0 0 0) 0x7fad0c08a340 con 0x7fad1415ab10 2023-10-14T10:37:03.643 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.630+0000 7fad195d4700 1 --2- 172.21.15.67:0/539148859 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7facf806af00 0x7facf806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:03.643 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.631+0000 7fad195d4700 1 --2- 172.21.15.67:0/539148859 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7facf806af00 0x7facf806d3c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fad0400a010 tx=0x7fad0401b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:03.644 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.634+0000 7fad09ffb700 1 -- 172.21.15.67:0/539148859 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fad0c0553d0 con 0x7fad1415ab10 2023-10-14T10:37:03.837 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.835+0000 7fad1b838700 1 -- 172.21.15.67:0/539148859 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fad140afc10 con 0x7fad1415ab10 2023-10-14T10:37:03.838 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.836+0000 7fad09ffb700 1 -- 172.21.15.67:0/539148859 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16541 (secure 0 0 0) 0x7fad0c0589f0 con 0x7fad1415ab10 2023-10-14T10:37:03.838 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-14T10:37:03.838 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":33,"fsid":"37f9ab46-3082-4b71-8b24-08d39897b008","created":"2023-10-14T10:35:46.067554+0000","modified":"2023-10-14T10:37:01.056657+0000","last_up_change":"2023-10-14T10:35:56.668142+0000","last_in_change":"2023-10-14T10:35:48.551490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-14T10:35:57.809566+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"testcephfs1_metadata","create_time":"2023-10-14T10:36:57.591719+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"testcephfs1"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"testcephfs1_data","create_time":"2023-10-14T10:36:58.436708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"testcephfs1"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2b59a07e-11ef-440c-b766-9f64ac885838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6801","nonce":1287255499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6803","nonce":1287255499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6807","nonce":1287255499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6805","nonce":1287255499}]},"public_addr":"172.21.15.67:6801/1287255499","cluster_addr":"172.21.15.67:6803/1287255499","heartbeat_back_addr":"172.21.15.67:6807/1287255499","heartbeat_front_addr":"172.21.15.67:6805/1287255499","state":["exists","up"]},{"osd":1,"uuid":"6f813bd6-fa0b-487d-a204-431eb1276a60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6818","nonce":2435538787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6822","nonce":2435538787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6830","nonce":2435538787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6826","nonce":2435538787}]},"public_addr":"172.21.15.67:6818/2435538787","cluster_addr":"172.21.15.67:6822/2435538787","heartbeat_back_addr":"172.21.15.67:6830/2435538787","heartbeat_front_addr":"172.21.15.67:6826/2435538787","state":["exists","up"]},{"osd":2,"uuid":"6af42c6a-1b59-4669-9c04-b6876788446d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6809","nonce":2827491791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6811","nonce":2827491791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6815","nonce":2827491791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6813","nonce":2827491791}]},"public_addr":"172.21.15.67:6809/2827491791","cluster_addr":"172.21.15.67:6811/2827491791","heartbeat_back_addr":"172.21.15.67:6815/2827491791","heartbeat_front_addr":"172.21.15.67:6813/2827491791","state":["exists","up"]},{"osd":3,"uuid":"1d889d76-a646-499d-bfff-ed5bae5434ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6819","nonce":1848001176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6821","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6823","nonce":1848001176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6831","nonce":1848001176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6825","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6827","nonce":1848001176}]},"public_addr":"172.21.15.67:6819/1848001176","cluster_addr":"172.21.15.67:6823/1848001176","heartbeat_back_addr":"172.21.15.67:6831/1848001176","heartbeat_front_addr":"172.21.15.67:6827/1848001176","state":["exists","up"]},{"osd":4,"uuid":"c3271dff-b437-4fdd-b13b-6e49a35473af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6804","nonce":1891230785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6808","nonce":1891230785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6815","nonce":1891230785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6811","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6813","nonce":1891230785}]},"public_addr":"172.21.15.72:6804/1891230785","cluster_addr":"172.21.15.72:6808/1891230785","heartbeat_back_addr":"172.21.15.72:6815/1891230785","heartbeat_front_addr":"172.21.15.72:6813/1891230785","state":["exists","up"]},{"osd":5,"uuid":"0337bb18-4b68-4eff-b528-a667275e6926","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6801","nonce":569686991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6803","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6805","nonce":569686991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6812","nonce":569686991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6807","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6809","nonce":569686991}]},"public_addr":"172.21.15.72:6801/569686991","cluster_addr":"172.21.15.72:6805/569686991","heartbeat_back_addr":"172.21.15.72:6812/569686991","heartbeat_front_addr":"172.21.15.72:6809/569686991","state":["exists","up"]},{"osd":6,"uuid":"8e92d9cf-0c8c-46eb-89a7-db2023ef1509","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6817","nonce":3767685934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6819","nonce":3767685934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6823","nonce":3767685934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6821","nonce":3767685934}]},"public_addr":"172.21.15.72:6817/3767685934","cluster_addr":"172.21.15.72:6819/3767685934","heartbeat_back_addr":"172.21.15.72:6823/3767685934","heartbeat_front_addr":"172.21.15.72:6821/3767685934","state":["exists","up"]},{"osd":7,"uuid":"3fd6aa7d-50a9-45c9-9561-5902a044a119","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6825","nonce":3567465926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6827","nonce":3567465926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6831","nonce":3567465926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6829","nonce":3567465926}]},"public_addr":"172.21.15.72:6825/3567465926","cluster_addr":"172.21.15.72:6827/3567465926","heartbeat_back_addr":"172.21.15.72:6831/3567465926","heartbeat_front_addr":"172.21.15.72:6829/3567465926","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.511466+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.795107+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.858674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.728895+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.794842+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.750064+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.732122+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.765185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-14T10:37:03.840 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.838+0000 7fad1b838700 1 -- 172.21.15.67:0/539148859 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7facf806af00 msgr2=0x7facf806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:03.840 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.838+0000 7fad1b838700 1 --2- 172.21.15.67:0/539148859 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7facf806af00 0x7facf806d3c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fad0400a010 tx=0x7fad0401b040 comp rx=0 tx=0).stop 2023-10-14T10:37:03.840 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.838+0000 7fad1b838700 1 -- 172.21.15.67:0/539148859 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fad1415ab10 msgr2=0x7fad140b4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:03.841 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.838+0000 7fad1b838700 1 --2- 172.21.15.67:0/539148859 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fad1415ab10 0x7fad140b4b80 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fad0c00efc0 tx=0x7fad0c00c3a0 comp rx=0 tx=0).stop 2023-10-14T10:37:03.841 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.838+0000 7fad1b838700 1 -- 172.21.15.67:0/539148859 shutdown_connections 2023-10-14T10:37:03.841 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.838+0000 7fad1b838700 1 --2- 172.21.15.67:0/539148859 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7facf806af00 0x7facf806d3c0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:03.841 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.838+0000 7fad1b838700 1 --2- 172.21.15.67:0/539148859 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fad1415ab10 0x7fad140b4b80 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:03.841 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.838+0000 7fad1b838700 1 --2- 172.21.15.67:0/539148859 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fad140b1f50 0x7fad140b0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:03.842 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.838+0000 7fad1b838700 1 --2- 172.21.15.67:0/539148859 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fad140cfe30 0x7fad140b4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:03.842 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.838+0000 7fad1b838700 1 -- 172.21.15.67:0/539148859 >> 172.21.15.67:0/539148859 conn(0x7fad1406dae0 msgr2=0x7fad140c3730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:37:03.842 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.838+0000 7fad1b838700 1 -- 172.21.15.67:0/539148859 shutdown_connections 2023-10-14T10:37:03.842 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.838+0000 7fad1b838700 1 -- 172.21.15.67:0/539148859 wait complete. 2023-10-14T10:37:03.852 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-14T10:37:03.853 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T10:37:03.978 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.976+0000 7fcf6ccec700 1 Processor -- start 2023-10-14T10:37:03.979 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.976+0000 7fcf6ccec700 1 -- start start 2023-10-14T10:37:03.981 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.977+0000 7fcf6ccec700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fcf6815ab10 0x7fcf6815f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:03.981 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.977+0000 7fcf6ccec700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fcf680b1f50 0x7fcf680cf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:03.981 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.977+0000 7fcf6ccec700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf6815e7a0 con 0x7fcf680b2400 2023-10-14T10:37:03.981 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.977+0000 7fcf6ccec700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf6815f730 con 0x7fcf680b1f50 2023-10-14T10:37:03.982 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.977+0000 7fcf6ccec700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf680b29a0 con 0x7fcf6815ab10 2023-10-14T10:37:03.982 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.977+0000 7fcf65d9b700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fcf680b1f50 0x7fcf680cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:03.982 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.977+0000 7fcf65d9b700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fcf680b1f50 0x7fcf680cf720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45834/0 (socket says 172.21.15.67:45834) 2023-10-14T10:37:03.982 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.977+0000 7fcf65d9b700 1 -- 172.21.15.67:0/475630149 learned_addr learned my addr 172.21.15.67:0/475630149 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:37:03.982 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.977+0000 7fcf6659c700 1 --2- 172.21.15.67:0/475630149 >> v2:172.21.15.72:3300/0 conn(0x7fcf6815ab10 0x7fcf6815f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:03.983 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.978+0000 7fcf65d9b700 1 -- 172.21.15.67:0/475630149 >> v2:172.21.15.72:3300/0 conn(0x7fcf6815ab10 msgr2=0x7fcf6815f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:37:03.983 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.978+0000 7fcf65d9b700 1 --2- 172.21.15.67:0/475630149 >> v2:172.21.15.72:3300/0 conn(0x7fcf6815ab10 0x7fcf6815f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:03.983 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.978+0000 7fcf65d9b700 1 -- 172.21.15.67:0/475630149 >> v1:172.21.15.72:6789/0 conn(0x7fcf680b2400 legacy=0x7fcf680cbf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:03.983 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.978+0000 7fcf65d9b700 1 -- 172.21.15.67:0/475630149 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf680b2c80 con 0x7fcf680b1f50 2023-10-14T10:37:03.983 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.978+0000 7fcf65d9b700 1 --2- 172.21.15.67:0/475630149 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf680b1f50 0x7fcf680cf720 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fcf680d0430 tx=0x7fcf58009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61d8cd7a39d27fe0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:03.984 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.978+0000 7fcf6559a700 1 -- 172.21.15.67:0/475630149 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf58005e30 con 0x7fcf680b1f50 2023-10-14T10:37:03.984 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.978+0000 7fcf6559a700 1 -- 172.21.15.67:0/475630149 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf58004030 con 0x7fcf680b1f50 2023-10-14T10:37:03.984 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.978+0000 7fcf6559a700 1 -- 172.21.15.67:0/475630149 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf580043e0 con 0x7fcf680b1f50 2023-10-14T10:37:03.984 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.978+0000 7fcf6ccec700 1 -- 172.21.15.67:0/475630149 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf680b1f50 msgr2=0x7fcf680cf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:03.984 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.978+0000 7fcf6ccec700 1 --2- 172.21.15.67:0/475630149 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf680b1f50 0x7fcf680cf720 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fcf680d0430 tx=0x7fcf58009bd0 comp rx=0 tx=0).stop 2023-10-14T10:37:03.985 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.979+0000 7fcf6ccec700 1 -- 172.21.15.67:0/475630149 shutdown_connections 2023-10-14T10:37:03.985 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.979+0000 7fcf6ccec700 1 --2- 172.21.15.67:0/475630149 >> v2:172.21.15.72:3300/0 conn(0x7fcf6815ab10 0x7fcf6815f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:03.985 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.979+0000 7fcf6ccec700 1 --2- 172.21.15.67:0/475630149 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf680b1f50 0x7fcf680cf720 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:03.985 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.979+0000 7fcf6ccec700 1 -- 172.21.15.67:0/475630149 >> 172.21.15.67:0/475630149 conn(0x7fcf6806dae0 msgr2=0x7fcf6806def0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:37:03.986 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.979+0000 7fcf6ccec700 1 -- 172.21.15.67:0/475630149 shutdown_connections 2023-10-14T10:37:03.986 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.979+0000 7fcf6ccec700 1 -- 172.21.15.67:0/475630149 wait complete. 2023-10-14T10:37:03.989 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.980+0000 7fcf6ccec700 1 Processor -- start 2023-10-14T10:37:03.989 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.980+0000 7fcf6ccec700 1 -- start start 2023-10-14T10:37:03.989 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.981+0000 7fcf6ccec700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fcf680b1f50 0x7fcf68153c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:03.989 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.981+0000 7fcf6ccec700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf680b2400 0x7fcf68152da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:03.990 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.981+0000 7fcf6ccec700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcf6815ab10 0x7fcf681532e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:03.990 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.981+0000 7fcf6ccec700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf6815dc20 con 0x7fcf680b2400 2023-10-14T10:37:03.990 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.981+0000 7fcf6ccec700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf680c4b80 con 0x7fcf6815ab10 2023-10-14T10:37:03.990 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.981+0000 7fcf6ccec700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf680c59c0 con 0x7fcf680b1f50 2023-10-14T10:37:03.991 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.981+0000 7fcf65d9b700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf680b2400 0x7fcf68152da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:03.991 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.981+0000 7fcf65d9b700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf680b2400 0x7fcf68152da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45840/0 (socket says 172.21.15.67:45840) 2023-10-14T10:37:03.991 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.981+0000 7fcf65d9b700 1 -- 172.21.15.67:0/2486673263 learned_addr learned my addr 172.21.15.67:0/2486673263 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:37:03.991 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.981+0000 7fcf66d9d700 1 --2- 172.21.15.67:0/2486673263 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcf6815ab10 0x7fcf681532e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:03.991 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.981+0000 7fcf6659c700 1 --2- 172.21.15.67:0/2486673263 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fcf680b1f50 0x7fcf68153c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:03.991 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.981+0000 7fcf65d9b700 1 -- 172.21.15.67:0/2486673263 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fcf680b1f50 msgr2=0x7fcf68153c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:03.992 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.981+0000 7fcf65d9b700 1 --2- 172.21.15.67:0/2486673263 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fcf680b1f50 0x7fcf68153c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:03.992 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.981+0000 7fcf65d9b700 1 -- 172.21.15.67:0/2486673263 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcf6815ab10 msgr2=0x7fcf681532e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:03.992 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.981+0000 7fcf65d9b700 1 --2- 172.21.15.67:0/2486673263 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcf6815ab10 0x7fcf681532e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:03.992 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.981+0000 7fcf65d9b700 1 -- 172.21.15.67:0/2486673263 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf58009500 con 0x7fcf680b2400 2023-10-14T10:37:03.992 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.981+0000 7fcf65d9b700 1 --2- 172.21.15.67:0/2486673263 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf680b2400 0x7fcf68152da0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fcf6806d4f0 tx=0x7fcf58004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:03.993 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.982+0000 7fcf56ffd700 1 -- 172.21.15.67:0/2486673263 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf58003c30 con 0x7fcf680b2400 2023-10-14T10:37:03.993 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.982+0000 7fcf56ffd700 1 -- 172.21.15.67:0/2486673263 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf58003d90 con 0x7fcf680b2400 2023-10-14T10:37:03.993 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.982+0000 7fcf6ccec700 1 -- 172.21.15.67:0/2486673263 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf680c4db0 con 0x7fcf680b2400 2023-10-14T10:37:03.993 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.982+0000 7fcf56ffd700 1 -- 172.21.15.67:0/2486673263 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf5801c640 con 0x7fcf680b2400 2023-10-14T10:37:03.993 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.982+0000 7fcf6ccec700 1 -- 172.21.15.67:0/2486673263 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf680c5370 con 0x7fcf680b2400 2023-10-14T10:37:03.994 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.983+0000 7fcf6ccec700 1 -- 172.21.15.67:0/2486673263 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf680c5b30 con 0x7fcf680b2400 2023-10-14T10:37:03.994 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.983+0000 7fcf56ffd700 1 -- 172.21.15.67:0/2486673263 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fcf5801c7a0 con 0x7fcf680b2400 2023-10-14T10:37:03.994 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.983+0000 7fcf56ffd700 1 --2- 172.21.15.67:0/2486673263 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fcf4806af00 0x7fcf4806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:03.994 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.984+0000 7fcf56ffd700 1 -- 172.21.15.67:0/2486673263 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5729+0+0 (secure 0 0 0) 0x7fcf5808a490 con 0x7fcf680b2400 2023-10-14T10:37:03.994 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.984+0000 7fcf6659c700 1 --2- 172.21.15.67:0/2486673263 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fcf4806af00 0x7fcf4806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:03.995 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.984+0000 7fcf6659c700 1 --2- 172.21.15.67:0/2486673263 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fcf4806af00 0x7fcf4806d3c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c009be0 tx=0x7fcf5c009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:03.995 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:03.988+0000 7fcf56ffd700 1 -- 172.21.15.67:0/2486673263 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcf580555b0 con 0x7fcf680b2400 2023-10-14T10:37:04.037 INFO:tasks.ceph.mds.c.smithi067.stderr:2023-10-14T10:37:04.035+0000 7f0b3bfab700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-14T10:37:04.187 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.185+0000 7fcf6ccec700 1 -- 172.21.15.67:0/2486673263 --> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcf6815ecd0 con 0x7fcf4806af00 2023-10-14T10:37:04.196 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.194+0000 7fcf56ffd700 1 -- 172.21.15.67:0/2486673263 <== mgr.4100 v2:172.21.15.72:6832/703488382 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289591 (secure 0 0 0) 0x7fcf5c011860 con 0x7fcf4806af00 2023-10-14T10:37:04.200 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-14T10:37:04.203 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.198+0000 7fcf6ccec700 1 -- 172.21.15.67:0/2486673263 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fcf4806af00 msgr2=0x7fcf4806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:04.203 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.198+0000 7fcf6ccec700 1 --2- 172.21.15.67:0/2486673263 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fcf4806af00 0x7fcf4806d3c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c009be0 tx=0x7fcf5c009210 comp rx=0 tx=0).stop 2023-10-14T10:37:04.203 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.198+0000 7fcf6ccec700 1 -- 172.21.15.67:0/2486673263 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf680b2400 msgr2=0x7fcf68152da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:04.203 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.198+0000 7fcf6ccec700 1 --2- 172.21.15.67:0/2486673263 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf680b2400 0x7fcf68152da0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fcf6806d4f0 tx=0x7fcf58004750 comp rx=0 tx=0).stop 2023-10-14T10:37:04.203 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.198+0000 7fcf6ccec700 1 -- 172.21.15.67:0/2486673263 shutdown_connections 2023-10-14T10:37:04.204 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.198+0000 7fcf6ccec700 1 --2- 172.21.15.67:0/2486673263 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fcf4806af00 0x7fcf4806d3c0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:04.204 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.198+0000 7fcf6ccec700 1 --2- 172.21.15.67:0/2486673263 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcf6815ab10 0x7fcf681532e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:04.204 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.198+0000 7fcf6ccec700 1 --2- 172.21.15.67:0/2486673263 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fcf680b1f50 0x7fcf68153c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:04.204 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.198+0000 7fcf6ccec700 1 --2- 172.21.15.67:0/2486673263 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf680b2400 0x7fcf68152da0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:04.204 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.198+0000 7fcf6ccec700 1 -- 172.21.15.67:0/2486673263 >> 172.21.15.67:0/2486673263 conn(0x7fcf6806dae0 msgr2=0x7fcf6806dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:37:04.205 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.198+0000 7fcf6ccec700 1 -- 172.21.15.67:0/2486673263 shutdown_connections 2023-10-14T10:37:04.205 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.198+0000 7fcf6ccec700 1 -- 172.21.15.67:0/2486673263 wait complete. 2023-10-14T10:37:04.205 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2023-10-14T10:37:04.212 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":60,"stamp":"2023-10-14T10:37:03.769569+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":122,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220232,"kb_used_data":4044,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754488,"statfs":{"total":773094113280,"available":772868595712,"internally_reserved":0,"allocated":4141056,"data_stored":2291019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12742,"internal_metadata":221236794},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.726849"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.787560+0000","last_change":"2023-10-14T10:37:00.075102+0000","last_active":"2023-10-14T10:37:01.787560+0000","last_peered":"2023-10-14T10:37:01.787560+0000","last_clean":"2023-10-14T10:37:01.787560+0000","last_became_active":"2023-10-14T10:37:00.074851+0000","last_became_peered":"2023-10-14T10:37:00.074851+0000","last_unstale":"2023-10-14T10:37:01.787560+0000","last_undegraded":"2023-10-14T10:37:01.787560+0000","last_fullsized":"2023-10-14T10:37:01.787560+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T21:39:35.587728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788402+0000","last_change":"2023-10-14T10:37:00.068871+0000","last_active":"2023-10-14T10:37:01.788402+0000","last_peered":"2023-10-14T10:37:01.788402+0000","last_clean":"2023-10-14T10:37:01.788402+0000","last_became_active":"2023-10-14T10:37:00.068586+0000","last_became_peered":"2023-10-14T10:37:00.068586+0000","last_unstale":"2023-10-14T10:37:01.788402+0000","last_undegraded":"2023-10-14T10:37:01.788402+0000","last_fullsized":"2023-10-14T10:37:01.788402+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T12:47:24.725885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.062215+0000","last_change":"2023-10-14T10:37:00.080988+0000","last_active":"2023-10-14T10:37:01.062215+0000","last_peered":"2023-10-14T10:37:01.062215+0000","last_clean":"2023-10-14T10:37:01.062215+0000","last_became_active":"2023-10-14T10:37:00.080476+0000","last_became_peered":"2023-10-14T10:37:00.080476+0000","last_unstale":"2023-10-14T10:37:01.062215+0000","last_undegraded":"2023-10-14T10:37:01.062215+0000","last_fullsized":"2023-10-14T10:37:01.062215+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T10:48:03.043383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.059874+0000","last_change":"2023-10-14T10:37:00.077598+0000","last_active":"2023-10-14T10:37:01.059874+0000","last_peered":"2023-10-14T10:37:01.059874+0000","last_clean":"2023-10-14T10:37:01.059874+0000","last_became_active":"2023-10-14T10:37:00.077322+0000","last_became_peered":"2023-10-14T10:37:00.077322+0000","last_unstale":"2023-10-14T10:37:01.059874+0000","last_undegraded":"2023-10-14T10:37:01.059874+0000","last_fullsized":"2023-10-14T10:37:01.059874+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:28:04.536948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788360+0000","last_change":"2023-10-14T10:37:00.068342+0000","last_active":"2023-10-14T10:37:01.788360+0000","last_peered":"2023-10-14T10:37:01.788360+0000","last_clean":"2023-10-14T10:37:01.788360+0000","last_became_active":"2023-10-14T10:37:00.068055+0000","last_became_peered":"2023-10-14T10:37:00.068055+0000","last_unstale":"2023-10-14T10:37:01.788360+0000","last_undegraded":"2023-10-14T10:37:01.788360+0000","last_fullsized":"2023-10-14T10:37:01.788360+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T12:52:16.372161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788917+0000","last_change":"2023-10-14T10:37:00.072629+0000","last_active":"2023-10-14T10:37:01.788917+0000","last_peered":"2023-10-14T10:37:01.788917+0000","last_clean":"2023-10-14T10:37:01.788917+0000","last_became_active":"2023-10-14T10:37:00.072345+0000","last_became_peered":"2023-10-14T10:37:00.072345+0000","last_unstale":"2023-10-14T10:37:01.788917+0000","last_undegraded":"2023-10-14T10:37:01.788917+0000","last_fullsized":"2023-10-14T10:37:01.788917+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T17:09:21.565688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.061635+0000","last_change":"2023-10-14T10:37:00.073019+0000","last_active":"2023-10-14T10:37:01.061635+0000","last_peered":"2023-10-14T10:37:01.061635+0000","last_clean":"2023-10-14T10:37:01.061635+0000","last_became_active":"2023-10-14T10:37:00.072368+0000","last_became_peered":"2023-10-14T10:37:00.072368+0000","last_unstale":"2023-10-14T10:37:01.061635+0000","last_undegraded":"2023-10-14T10:37:01.061635+0000","last_fullsized":"2023-10-14T10:37:01.061635+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:31:16.127752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.062210+0000","last_change":"2023-10-14T10:37:00.074091+0000","last_active":"2023-10-14T10:37:01.062210+0000","last_peered":"2023-10-14T10:37:01.062210+0000","last_clean":"2023-10-14T10:37:01.062210+0000","last_became_active":"2023-10-14T10:37:00.073792+0000","last_became_peered":"2023-10-14T10:37:00.073792+0000","last_unstale":"2023-10-14T10:37:01.062210+0000","last_undegraded":"2023-10-14T10:37:01.062210+0000","last_fullsized":"2023-10-14T10:37:01.062210+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T17:27:50.406084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.489588+0000","last_change":"2023-10-14T10:37:00.081745+0000","last_active":"2023-10-14T10:37:01.489588+0000","last_peered":"2023-10-14T10:37:01.489588+0000","last_clean":"2023-10-14T10:37:01.489588+0000","last_became_active":"2023-10-14T10:37:00.081451+0000","last_became_peered":"2023-10-14T10:37:00.081451+0000","last_unstale":"2023-10-14T10:37:01.489588+0000","last_undegraded":"2023-10-14T10:37:01.489588+0000","last_fullsized":"2023-10-14T10:37:01.489588+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:18:45.297555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.060294+0000","last_change":"2023-10-14T10:37:00.071138+0000","last_active":"2023-10-14T10:37:01.060294+0000","last_peered":"2023-10-14T10:37:01.060294+0000","last_clean":"2023-10-14T10:37:01.060294+0000","last_became_active":"2023-10-14T10:37:00.070667+0000","last_became_peered":"2023-10-14T10:37:00.070667+0000","last_unstale":"2023-10-14T10:37:01.060294+0000","last_undegraded":"2023-10-14T10:37:01.060294+0000","last_fullsized":"2023-10-14T10:37:01.060294+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:52:37.894720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.062228+0000","last_change":"2023-10-14T10:37:00.073322+0000","last_active":"2023-10-14T10:37:01.062228+0000","last_peered":"2023-10-14T10:37:01.062228+0000","last_clean":"2023-10-14T10:37:01.062228+0000","last_became_active":"2023-10-14T10:37:00.072639+0000","last_became_peered":"2023-10-14T10:37:00.072639+0000","last_unstale":"2023-10-14T10:37:01.062228+0000","last_undegraded":"2023-10-14T10:37:01.062228+0000","last_fullsized":"2023-10-14T10:37:01.062228+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T18:57:50.136605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.061858+0000","last_change":"2023-10-14T10:37:00.069681+0000","last_active":"2023-10-14T10:37:01.061858+0000","last_peered":"2023-10-14T10:37:01.061858+0000","last_clean":"2023-10-14T10:37:01.061858+0000","last_became_active":"2023-10-14T10:37:00.069373+0000","last_became_peered":"2023-10-14T10:37:00.069373+0000","last_unstale":"2023-10-14T10:37:01.061858+0000","last_undegraded":"2023-10-14T10:37:01.061858+0000","last_fullsized":"2023-10-14T10:37:01.061858+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:31:16.127752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788687+0000","last_change":"2023-10-14T10:37:00.074917+0000","last_active":"2023-10-14T10:37:01.788687+0000","last_peered":"2023-10-14T10:37:01.788687+0000","last_clean":"2023-10-14T10:37:01.788687+0000","last_became_active":"2023-10-14T10:37:00.074473+0000","last_became_peered":"2023-10-14T10:37:00.074473+0000","last_unstale":"2023-10-14T10:37:01.788687+0000","last_undegraded":"2023-10-14T10:37:01.788687+0000","last_fullsized":"2023-10-14T10:37:01.788687+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T20:20:02.187767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788602+0000","last_change":"2023-10-14T10:37:00.073986+0000","last_active":"2023-10-14T10:37:01.788602+0000","last_peered":"2023-10-14T10:37:01.788602+0000","last_clean":"2023-10-14T10:37:01.788602+0000","last_became_active":"2023-10-14T10:37:00.073651+0000","last_became_peered":"2023-10-14T10:37:00.073651+0000","last_unstale":"2023-10-14T10:37:01.788602+0000","last_undegraded":"2023-10-14T10:37:01.788602+0000","last_fullsized":"2023-10-14T10:37:01.788602+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T10:48:03.043383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.061624+0000","last_change":"2023-10-14T10:37:00.072277+0000","last_active":"2023-10-14T10:37:01.061624+0000","last_peered":"2023-10-14T10:37:01.061624+0000","last_clean":"2023-10-14T10:37:01.061624+0000","last_became_active":"2023-10-14T10:37:00.071999+0000","last_became_peered":"2023-10-14T10:37:00.071999+0000","last_unstale":"2023-10-14T10:37:01.061624+0000","last_undegraded":"2023-10-14T10:37:01.061624+0000","last_fullsized":"2023-10-14T10:37:01.061624+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:18:45.297555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.062458+0000","last_change":"2023-10-14T10:37:00.072492+0000","last_active":"2023-10-14T10:37:01.062458+0000","last_peered":"2023-10-14T10:37:01.062458+0000","last_clean":"2023-10-14T10:37:01.062458+0000","last_became_active":"2023-10-14T10:37:00.071509+0000","last_became_peered":"2023-10-14T10:37:00.071509+0000","last_unstale":"2023-10-14T10:37:01.062458+0000","last_undegraded":"2023-10-14T10:37:01.062458+0000","last_fullsized":"2023-10-14T10:37:01.062458+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:51:00.167498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.062381+0000","last_change":"2023-10-14T10:37:00.072597+0000","last_active":"2023-10-14T10:37:01.062381+0000","last_peered":"2023-10-14T10:37:01.062381+0000","last_clean":"2023-10-14T10:37:01.062381+0000","last_became_active":"2023-10-14T10:37:00.071946+0000","last_became_peered":"2023-10-14T10:37:00.071946+0000","last_unstale":"2023-10-14T10:37:01.062381+0000","last_undegraded":"2023-10-14T10:37:01.062381+0000","last_fullsized":"2023-10-14T10:37:01.062381+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:06:28.391046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788267+0000","last_change":"2023-10-14T10:37:00.080581+0000","last_active":"2023-10-14T10:37:01.788267+0000","last_peered":"2023-10-14T10:37:01.788267+0000","last_clean":"2023-10-14T10:37:01.788267+0000","last_became_active":"2023-10-14T10:37:00.080288+0000","last_became_peered":"2023-10-14T10:37:00.080288+0000","last_unstale":"2023-10-14T10:37:01.788267+0000","last_undegraded":"2023-10-14T10:37:01.788267+0000","last_fullsized":"2023-10-14T10:37:01.788267+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T18:57:50.136605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.787886+0000","last_change":"2023-10-14T10:37:00.080484+0000","last_active":"2023-10-14T10:37:01.787886+0000","last_peered":"2023-10-14T10:37:01.787886+0000","last_clean":"2023-10-14T10:37:01.787886+0000","last_became_active":"2023-10-14T10:37:00.080186+0000","last_became_peered":"2023-10-14T10:37:00.080186+0000","last_unstale":"2023-10-14T10:37:01.787886+0000","last_undegraded":"2023-10-14T10:37:01.787886+0000","last_fullsized":"2023-10-14T10:37:01.787886+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:17:54.730152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788501+0000","last_change":"2023-10-14T10:37:00.072056+0000","last_active":"2023-10-14T10:37:01.788501+0000","last_peered":"2023-10-14T10:37:01.788501+0000","last_clean":"2023-10-14T10:37:01.788501+0000","last_became_active":"2023-10-14T10:37:00.071796+0000","last_became_peered":"2023-10-14T10:37:00.071796+0000","last_unstale":"2023-10-14T10:37:01.788501+0000","last_undegraded":"2023-10-14T10:37:01.788501+0000","last_fullsized":"2023-10-14T10:37:01.788501+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T20:39:07.030985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.489206+0000","last_change":"2023-10-14T10:37:00.080796+0000","last_active":"2023-10-14T10:37:01.489206+0000","last_peered":"2023-10-14T10:37:01.489206+0000","last_clean":"2023-10-14T10:37:01.489206+0000","last_became_active":"2023-10-14T10:37:00.080526+0000","last_became_peered":"2023-10-14T10:37:00.080526+0000","last_unstale":"2023-10-14T10:37:01.489206+0000","last_undegraded":"2023-10-14T10:37:01.489206+0000","last_fullsized":"2023-10-14T10:37:01.489206+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:45:25.303185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.489168+0000","last_change":"2023-10-14T10:37:00.080350+0000","last_active":"2023-10-14T10:37:01.489168+0000","last_peered":"2023-10-14T10:37:01.489168+0000","last_clean":"2023-10-14T10:37:01.489168+0000","last_became_active":"2023-10-14T10:37:00.080062+0000","last_became_peered":"2023-10-14T10:37:00.080062+0000","last_unstale":"2023-10-14T10:37:01.489168+0000","last_undegraded":"2023-10-14T10:37:01.489168+0000","last_fullsized":"2023-10-14T10:37:01.489168+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:10:46.473503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.787611+0000","last_change":"2023-10-14T10:37:00.071237+0000","last_active":"2023-10-14T10:37:01.787611+0000","last_peered":"2023-10-14T10:37:01.787611+0000","last_clean":"2023-10-14T10:37:01.787611+0000","last_became_active":"2023-10-14T10:37:00.070897+0000","last_became_peered":"2023-10-14T10:37:00.070897+0000","last_unstale":"2023-10-14T10:37:01.787611+0000","last_undegraded":"2023-10-14T10:37:01.787611+0000","last_fullsized":"2023-10-14T10:37:01.787611+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:59:07.208582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.061365+0000","last_change":"2023-10-14T10:37:00.069525+0000","last_active":"2023-10-14T10:37:01.061365+0000","last_peered":"2023-10-14T10:37:01.061365+0000","last_clean":"2023-10-14T10:37:01.061365+0000","last_became_active":"2023-10-14T10:37:00.069246+0000","last_became_peered":"2023-10-14T10:37:00.069246+0000","last_unstale":"2023-10-14T10:37:01.061365+0000","last_undegraded":"2023-10-14T10:37:01.061365+0000","last_fullsized":"2023-10-14T10:37:01.061365+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:52:37.894720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.060407+0000","last_change":"2023-10-14T10:37:00.070144+0000","last_active":"2023-10-14T10:37:01.060407+0000","last_peered":"2023-10-14T10:37:01.060407+0000","last_clean":"2023-10-14T10:37:01.060407+0000","last_became_active":"2023-10-14T10:37:00.069871+0000","last_became_peered":"2023-10-14T10:37:00.069871+0000","last_unstale":"2023-10-14T10:37:01.060407+0000","last_undegraded":"2023-10-14T10:37:01.060407+0000","last_fullsized":"2023-10-14T10:37:01.060407+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:22:32.154663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788109+0000","last_change":"2023-10-14T10:37:00.076474+0000","last_active":"2023-10-14T10:37:01.788109+0000","last_peered":"2023-10-14T10:37:01.788109+0000","last_clean":"2023-10-14T10:37:01.788109+0000","last_became_active":"2023-10-14T10:37:00.076169+0000","last_became_peered":"2023-10-14T10:37:00.076169+0000","last_unstale":"2023-10-14T10:37:01.788109+0000","last_undegraded":"2023-10-14T10:37:01.788109+0000","last_fullsized":"2023-10-14T10:37:01.788109+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T14:52:18.060480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.060360+0000","last_change":"2023-10-14T10:37:00.076568+0000","last_active":"2023-10-14T10:37:01.060360+0000","last_peered":"2023-10-14T10:37:01.060360+0000","last_clean":"2023-10-14T10:37:01.060360+0000","last_became_active":"2023-10-14T10:37:00.076263+0000","last_became_peered":"2023-10-14T10:37:00.076263+0000","last_unstale":"2023-10-14T10:37:01.060360+0000","last_undegraded":"2023-10-14T10:37:01.060360+0000","last_fullsized":"2023-10-14T10:37:01.060360+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:18:45.297555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.061994+0000","last_change":"2023-10-14T10:37:00.079439+0000","last_active":"2023-10-14T10:37:01.061994+0000","last_peered":"2023-10-14T10:37:01.061994+0000","last_clean":"2023-10-14T10:37:01.061994+0000","last_became_active":"2023-10-14T10:37:00.079178+0000","last_became_peered":"2023-10-14T10:37:00.079178+0000","last_unstale":"2023-10-14T10:37:01.061994+0000","last_undegraded":"2023-10-14T10:37:01.061994+0000","last_fullsized":"2023-10-14T10:37:01.061994+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:17:54.730152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.489206+0000","last_change":"2023-10-14T10:37:00.080675+0000","last_active":"2023-10-14T10:37:01.489206+0000","last_peered":"2023-10-14T10:37:01.489206+0000","last_clean":"2023-10-14T10:37:01.489206+0000","last_became_active":"2023-10-14T10:37:00.080171+0000","last_became_peered":"2023-10-14T10:37:00.080171+0000","last_unstale":"2023-10-14T10:37:01.489206+0000","last_undegraded":"2023-10-14T10:37:01.489206+0000","last_fullsized":"2023-10-14T10:37:01.489206+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:28:04.536948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.061814+0000","last_change":"2023-10-14T10:37:00.072155+0000","last_active":"2023-10-14T10:37:01.061814+0000","last_peered":"2023-10-14T10:37:01.061814+0000","last_clean":"2023-10-14T10:37:01.061814+0000","last_became_active":"2023-10-14T10:37:00.071882+0000","last_became_peered":"2023-10-14T10:37:00.071882+0000","last_unstale":"2023-10-14T10:37:01.061814+0000","last_undegraded":"2023-10-14T10:37:01.061814+0000","last_fullsized":"2023-10-14T10:37:01.061814+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T10:48:03.043383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.787458+0000","last_change":"2023-10-14T10:37:00.081816+0000","last_active":"2023-10-14T10:37:01.787458+0000","last_peered":"2023-10-14T10:37:01.787458+0000","last_clean":"2023-10-14T10:37:01.787458+0000","last_became_active":"2023-10-14T10:37:00.081312+0000","last_became_peered":"2023-10-14T10:37:00.081312+0000","last_unstale":"2023-10-14T10:37:01.787458+0000","last_undegraded":"2023-10-14T10:37:01.787458+0000","last_fullsized":"2023-10-14T10:37:01.787458+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:58:56.558710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.061373+0000","last_change":"2023-10-14T10:37:00.078914+0000","last_active":"2023-10-14T10:37:01.061373+0000","last_peered":"2023-10-14T10:37:01.061373+0000","last_clean":"2023-10-14T10:37:01.061373+0000","last_became_active":"2023-10-14T10:37:00.078441+0000","last_became_peered":"2023-10-14T10:37:00.078441+0000","last_unstale":"2023-10-14T10:37:01.061373+0000","last_undegraded":"2023-10-14T10:37:01.061373+0000","last_fullsized":"2023-10-14T10:37:01.061373+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:22:32.154663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.787754+0000","last_change":"2023-10-14T10:37:00.071246+0000","last_active":"2023-10-14T10:37:01.787754+0000","last_peered":"2023-10-14T10:37:01.787754+0000","last_clean":"2023-10-14T10:37:01.787754+0000","last_became_active":"2023-10-14T10:37:00.070945+0000","last_became_peered":"2023-10-14T10:37:00.070945+0000","last_unstale":"2023-10-14T10:37:01.787754+0000","last_undegraded":"2023-10-14T10:37:01.787754+0000","last_fullsized":"2023-10-14T10:37:01.787754+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:30:57.784792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.787645+0000","last_change":"2023-10-14T10:36:59.066772+0000","last_active":"2023-10-14T10:37:01.787645+0000","last_peered":"2023-10-14T10:37:01.787645+0000","last_clean":"2023-10-14T10:37:01.787645+0000","last_became_active":"2023-10-14T10:36:59.066488+0000","last_became_peered":"2023-10-14T10:36:59.066488+0000","last_unstale":"2023-10-14T10:37:01.787645+0000","last_undegraded":"2023-10-14T10:37:01.787645+0000","last_fullsized":"2023-10-14T10:37:01.787645+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:39:01.994329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.787569+0000","last_change":"2023-10-14T10:37:00.077997+0000","last_active":"2023-10-14T10:37:01.787569+0000","last_peered":"2023-10-14T10:37:01.787569+0000","last_clean":"2023-10-14T10:37:01.787569+0000","last_became_active":"2023-10-14T10:37:00.077734+0000","last_became_peered":"2023-10-14T10:37:00.077734+0000","last_unstale":"2023-10-14T10:37:01.787569+0000","last_undegraded":"2023-10-14T10:37:01.787569+0000","last_fullsized":"2023-10-14T10:37:01.787569+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:28:45.821377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.787856+0000","last_change":"2023-10-14T10:36:59.070768+0000","last_active":"2023-10-14T10:37:01.787856+0000","last_peered":"2023-10-14T10:37:01.787856+0000","last_clean":"2023-10-14T10:37:01.787856+0000","last_became_active":"2023-10-14T10:36:59.070463+0000","last_became_peered":"2023-10-14T10:36:59.070463+0000","last_unstale":"2023-10-14T10:37:01.787856+0000","last_undegraded":"2023-10-14T10:37:01.787856+0000","last_fullsized":"2023-10-14T10:37:01.787856+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:08:05.335325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.489357+0000","last_change":"2023-10-14T10:37:00.080768+0000","last_active":"2023-10-14T10:37:01.489357+0000","last_peered":"2023-10-14T10:37:01.489357+0000","last_clean":"2023-10-14T10:37:01.489357+0000","last_became_active":"2023-10-14T10:37:00.080160+0000","last_became_peered":"2023-10-14T10:37:00.080160+0000","last_unstale":"2023-10-14T10:37:01.489357+0000","last_undegraded":"2023-10-14T10:37:01.489357+0000","last_fullsized":"2023-10-14T10:37:01.489357+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T20:27:28.798883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788006+0000","last_change":"2023-10-14T10:36:59.067900+0000","last_active":"2023-10-14T10:37:01.788006+0000","last_peered":"2023-10-14T10:37:01.788006+0000","last_clean":"2023-10-14T10:37:01.788006+0000","last_became_active":"2023-10-14T10:36:59.067616+0000","last_became_peered":"2023-10-14T10:36:59.067616+0000","last_unstale":"2023-10-14T10:37:01.788006+0000","last_undegraded":"2023-10-14T10:37:01.788006+0000","last_fullsized":"2023-10-14T10:37:01.788006+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:28:44.815010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.060586+0000","last_change":"2023-10-14T10:37:00.079200+0000","last_active":"2023-10-14T10:37:01.060586+0000","last_peered":"2023-10-14T10:37:01.060586+0000","last_clean":"2023-10-14T10:37:01.060586+0000","last_became_active":"2023-10-14T10:37:00.078829+0000","last_became_peered":"2023-10-14T10:37:00.078829+0000","last_unstale":"2023-10-14T10:37:01.060586+0000","last_undegraded":"2023-10-14T10:37:01.060586+0000","last_fullsized":"2023-10-14T10:37:01.060586+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:31:16.127752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788507+0000","last_change":"2023-10-14T10:36:59.065882+0000","last_active":"2023-10-14T10:37:01.788507+0000","last_peered":"2023-10-14T10:37:01.788507+0000","last_clean":"2023-10-14T10:37:01.788507+0000","last_became_active":"2023-10-14T10:36:59.065608+0000","last_became_peered":"2023-10-14T10:36:59.065608+0000","last_unstale":"2023-10-14T10:37:01.788507+0000","last_undegraded":"2023-10-14T10:37:01.788507+0000","last_fullsized":"2023-10-14T10:37:01.788507+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T21:18:46.547776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.062326+0000","last_change":"2023-10-14T10:37:00.077115+0000","last_active":"2023-10-14T10:37:01.062326+0000","last_peered":"2023-10-14T10:37:01.062326+0000","last_clean":"2023-10-14T10:37:01.062326+0000","last_became_active":"2023-10-14T10:37:00.076840+0000","last_became_peered":"2023-10-14T10:37:00.076840+0000","last_unstale":"2023-10-14T10:37:01.062326+0000","last_undegraded":"2023-10-14T10:37:01.062326+0000","last_fullsized":"2023-10-14T10:37:01.062326+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T12:47:24.725885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.489315+0000","last_change":"2023-10-14T10:36:59.069533+0000","last_active":"2023-10-14T10:37:01.489315+0000","last_peered":"2023-10-14T10:37:01.489315+0000","last_clean":"2023-10-14T10:37:01.489315+0000","last_became_active":"2023-10-14T10:36:59.069192+0000","last_became_peered":"2023-10-14T10:36:59.069192+0000","last_unstale":"2023-10-14T10:37:01.489315+0000","last_undegraded":"2023-10-14T10:37:01.489315+0000","last_fullsized":"2023-10-14T10:37:01.489315+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T21:39:34.581362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.061634+0000","last_change":"2023-10-14T10:37:00.068786+0000","last_active":"2023-10-14T10:37:01.061634+0000","last_peered":"2023-10-14T10:37:01.061634+0000","last_clean":"2023-10-14T10:37:01.061634+0000","last_became_active":"2023-10-14T10:37:00.068517+0000","last_became_peered":"2023-10-14T10:37:00.068517+0000","last_unstale":"2023-10-14T10:37:01.061634+0000","last_undegraded":"2023-10-14T10:37:01.061634+0000","last_fullsized":"2023-10-14T10:37:01.061634+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:22:32.154663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.489303+0000","last_change":"2023-10-14T10:36:59.069445+0000","last_active":"2023-10-14T10:37:01.489303+0000","last_peered":"2023-10-14T10:37:01.489303+0000","last_clean":"2023-10-14T10:37:01.489303+0000","last_became_active":"2023-10-14T10:36:59.069151+0000","last_became_peered":"2023-10-14T10:36:59.069151+0000","last_unstale":"2023-10-14T10:37:01.489303+0000","last_undegraded":"2023-10-14T10:37:01.489303+0000","last_fullsized":"2023-10-14T10:37:01.489303+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:54:15.011628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.488953+0000","last_change":"2023-10-14T10:37:00.076984+0000","last_active":"2023-10-14T10:37:01.488953+0000","last_peered":"2023-10-14T10:37:01.488953+0000","last_clean":"2023-10-14T10:37:01.488953+0000","last_became_active":"2023-10-14T10:37:00.076706+0000","last_became_peered":"2023-10-14T10:37:00.076706+0000","last_unstale":"2023-10-14T10:37:01.488953+0000","last_undegraded":"2023-10-14T10:37:01.488953+0000","last_fullsized":"2023-10-14T10:37:01.488953+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:30:57.784792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.787709+0000","last_change":"2023-10-14T10:36:59.062324+0000","last_active":"2023-10-14T10:37:01.787709+0000","last_peered":"2023-10-14T10:37:01.787709+0000","last_clean":"2023-10-14T10:37:01.787709+0000","last_became_active":"2023-10-14T10:36:59.062075+0000","last_became_peered":"2023-10-14T10:36:59.062075+0000","last_unstale":"2023-10-14T10:37:01.787709+0000","last_undegraded":"2023-10-14T10:37:01.787709+0000","last_fullsized":"2023-10-14T10:37:01.787709+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:45:50.919718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.059848+0000","last_change":"2023-10-14T10:37:00.067486+0000","last_active":"2023-10-14T10:37:01.059848+0000","last_peered":"2023-10-14T10:37:01.059848+0000","last_clean":"2023-10-14T10:37:01.059848+0000","last_became_active":"2023-10-14T10:37:00.067187+0000","last_became_peered":"2023-10-14T10:37:00.067187+0000","last_unstale":"2023-10-14T10:37:01.059848+0000","last_undegraded":"2023-10-14T10:37:01.059848+0000","last_fullsized":"2023-10-14T10:37:01.059848+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:10:46.473503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.488998+0000","last_change":"2023-10-14T10:36:59.066330+0000","last_active":"2023-10-14T10:37:01.488998+0000","last_peered":"2023-10-14T10:37:01.488998+0000","last_clean":"2023-10-14T10:37:01.488998+0000","last_became_active":"2023-10-14T10:36:59.064990+0000","last_became_peered":"2023-10-14T10:36:59.064990+0000","last_unstale":"2023-10-14T10:37:01.488998+0000","last_undegraded":"2023-10-14T10:37:01.488998+0000","last_fullsized":"2023-10-14T10:37:01.488998+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T12:15:16.823069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.061974+0000","last_change":"2023-10-14T10:37:00.077465+0000","last_active":"2023-10-14T10:37:01.061974+0000","last_peered":"2023-10-14T10:37:01.061974+0000","last_clean":"2023-10-14T10:37:01.061974+0000","last_became_active":"2023-10-14T10:37:00.077152+0000","last_became_peered":"2023-10-14T10:37:00.077152+0000","last_unstale":"2023-10-14T10:37:01.061974+0000","last_undegraded":"2023-10-14T10:37:01.061974+0000","last_fullsized":"2023-10-14T10:37:01.061974+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:51:00.167498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788575+0000","last_change":"2023-10-14T10:36:59.061903+0000","last_active":"2023-10-14T10:37:01.788575+0000","last_peered":"2023-10-14T10:37:01.788575+0000","last_clean":"2023-10-14T10:37:01.788575+0000","last_became_active":"2023-10-14T10:36:59.061597+0000","last_became_peered":"2023-10-14T10:36:59.061597+0000","last_unstale":"2023-10-14T10:37:01.788575+0000","last_undegraded":"2023-10-14T10:37:01.788575+0000","last_fullsized":"2023-10-14T10:37:01.788575+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:22:31.148297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788727+0000","last_change":"2023-10-14T10:37:00.076213+0000","last_active":"2023-10-14T10:37:01.788727+0000","last_peered":"2023-10-14T10:37:01.788727+0000","last_clean":"2023-10-14T10:37:01.788727+0000","last_became_active":"2023-10-14T10:37:00.075889+0000","last_became_peered":"2023-10-14T10:37:00.075889+0000","last_unstale":"2023-10-14T10:37:01.788727+0000","last_undegraded":"2023-10-14T10:37:01.788727+0000","last_fullsized":"2023-10-14T10:37:01.788727+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T14:19:22.310866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.787968+0000","last_change":"2023-10-14T10:36:59.070183+0000","last_active":"2023-10-14T10:37:01.787968+0000","last_peered":"2023-10-14T10:37:01.787968+0000","last_clean":"2023-10-14T10:37:01.787968+0000","last_became_active":"2023-10-14T10:36:59.069731+0000","last_became_peered":"2023-10-14T10:36:59.069731+0000","last_unstale":"2023-10-14T10:37:01.787968+0000","last_undegraded":"2023-10-14T10:37:01.787968+0000","last_fullsized":"2023-10-14T10:37:01.787968+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:58:55.552343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788210+0000","last_change":"2023-10-14T10:37:00.072592+0000","last_active":"2023-10-14T10:37:01.788210+0000","last_peered":"2023-10-14T10:37:01.788210+0000","last_clean":"2023-10-14T10:37:01.788210+0000","last_became_active":"2023-10-14T10:37:00.072288+0000","last_became_peered":"2023-10-14T10:37:00.072288+0000","last_unstale":"2023-10-14T10:37:01.788210+0000","last_undegraded":"2023-10-14T10:37:01.788210+0000","last_fullsized":"2023-10-14T10:37:01.788210+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T17:27:50.406084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.062406+0000","last_change":"2023-10-14T10:36:59.062120+0000","last_active":"2023-10-14T10:37:01.062406+0000","last_peered":"2023-10-14T10:37:01.062406+0000","last_clean":"2023-10-14T10:37:01.062406+0000","last_became_active":"2023-10-14T10:36:59.061700+0000","last_became_peered":"2023-10-14T10:36:59.061700+0000","last_unstale":"2023-10-14T10:37:01.062406+0000","last_undegraded":"2023-10-14T10:37:01.062406+0000","last_fullsized":"2023-10-14T10:37:01.062406+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:45:24.296819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.062364+0000","last_change":"2023-10-14T10:37:00.074977+0000","last_active":"2023-10-14T10:37:01.062364+0000","last_peered":"2023-10-14T10:37:01.062364+0000","last_clean":"2023-10-14T10:37:01.062364+0000","last_became_active":"2023-10-14T10:37:00.074700+0000","last_became_peered":"2023-10-14T10:37:00.074700+0000","last_unstale":"2023-10-14T10:37:01.062364+0000","last_undegraded":"2023-10-14T10:37:01.062364+0000","last_fullsized":"2023-10-14T10:37:01.062364+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T18:57:50.136605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788323+0000","last_change":"2023-10-14T10:36:59.069400+0000","last_active":"2023-10-14T10:37:01.788323+0000","last_peered":"2023-10-14T10:37:01.788323+0000","last_clean":"2023-10-14T10:37:01.788323+0000","last_became_active":"2023-10-14T10:36:59.069043+0000","last_became_peered":"2023-10-14T10:36:59.069043+0000","last_unstale":"2023-10-14T10:37:01.788323+0000","last_undegraded":"2023-10-14T10:37:01.788323+0000","last_fullsized":"2023-10-14T10:37:01.788323+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:10:45.467136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.060040+0000","last_change":"2023-10-14T10:37:00.068613+0000","last_active":"2023-10-14T10:37:01.060040+0000","last_peered":"2023-10-14T10:37:01.060040+0000","last_clean":"2023-10-14T10:37:01.060040+0000","last_became_active":"2023-10-14T10:37:00.065345+0000","last_became_peered":"2023-10-14T10:37:00.065345+0000","last_unstale":"2023-10-14T10:37:01.060040+0000","last_undegraded":"2023-10-14T10:37:01.060040+0000","last_fullsized":"2023-10-14T10:37:01.060040+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:45:25.303185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.489387+0000","last_change":"2023-10-14T10:36:59.065846+0000","last_active":"2023-10-14T10:37:01.489387+0000","last_peered":"2023-10-14T10:37:01.489387+0000","last_clean":"2023-10-14T10:37:01.489387+0000","last_became_active":"2023-10-14T10:36:59.065569+0000","last_became_peered":"2023-10-14T10:36:59.065569+0000","last_unstale":"2023-10-14T10:37:01.489387+0000","last_undegraded":"2023-10-14T10:37:01.489387+0000","last_fullsized":"2023-10-14T10:37:01.489387+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:28:44.815010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788692+0000","last_change":"2023-10-14T10:37:00.079137+0000","last_active":"2023-10-14T10:37:01.788692+0000","last_peered":"2023-10-14T10:37:01.788692+0000","last_clean":"2023-10-14T10:37:01.788692+0000","last_became_active":"2023-10-14T10:37:00.078844+0000","last_became_peered":"2023-10-14T10:37:00.078844+0000","last_unstale":"2023-10-14T10:37:01.788692+0000","last_undegraded":"2023-10-14T10:37:01.788692+0000","last_fullsized":"2023-10-14T10:37:01.788692+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:30:51.914014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.059920+0000","last_change":"2023-10-14T10:36:59.067245+0000","last_active":"2023-10-14T10:37:01.059920+0000","last_peered":"2023-10-14T10:37:01.059920+0000","last_clean":"2023-10-14T10:37:01.059920+0000","last_became_active":"2023-10-14T10:36:59.066952+0000","last_became_peered":"2023-10-14T10:36:59.066952+0000","last_unstale":"2023-10-14T10:37:01.059920+0000","last_undegraded":"2023-10-14T10:37:01.059920+0000","last_fullsized":"2023-10-14T10:37:01.059920+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T12:08:55.620308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.062454+0000","last_change":"2023-10-14T10:37:00.078646+0000","last_active":"2023-10-14T10:37:01.062454+0000","last_peered":"2023-10-14T10:37:01.062454+0000","last_clean":"2023-10-14T10:37:01.062454+0000","last_became_active":"2023-10-14T10:37:00.078353+0000","last_became_peered":"2023-10-14T10:37:00.078353+0000","last_unstale":"2023-10-14T10:37:01.062454+0000","last_undegraded":"2023-10-14T10:37:01.062454+0000","last_fullsized":"2023-10-14T10:37:01.062454+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:30:51.914014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788938+0000","last_change":"2023-10-14T10:36:59.063118+0000","last_active":"2023-10-14T10:37:01.788938+0000","last_peered":"2023-10-14T10:37:01.788938+0000","last_clean":"2023-10-14T10:37:01.788938+0000","last_became_active":"2023-10-14T10:36:59.062380+0000","last_became_peered":"2023-10-14T10:36:59.062380+0000","last_unstale":"2023-10-14T10:37:01.788938+0000","last_undegraded":"2023-10-14T10:37:01.788938+0000","last_fullsized":"2023-10-14T10:37:01.788938+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T21:39:34.581362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.787507+0000","last_change":"2023-10-14T10:37:00.078077+0000","last_active":"2023-10-14T10:37:01.787507+0000","last_peered":"2023-10-14T10:37:01.787507+0000","last_clean":"2023-10-14T10:37:01.787507+0000","last_became_active":"2023-10-14T10:37:00.077806+0000","last_became_peered":"2023-10-14T10:37:00.077806+0000","last_unstale":"2023-10-14T10:37:01.787507+0000","last_undegraded":"2023-10-14T10:37:01.787507+0000","last_fullsized":"2023-10-14T10:37:01.787507+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:45:51.926085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788260+0000","last_change":"2023-10-14T10:36:59.068012+0000","last_active":"2023-10-14T10:37:01.788260+0000","last_peered":"2023-10-14T10:37:01.788260+0000","last_clean":"2023-10-14T10:37:01.788260+0000","last_became_active":"2023-10-14T10:36:59.067727+0000","last_became_peered":"2023-10-14T10:36:59.067727+0000","last_unstale":"2023-10-14T10:37:01.788260+0000","last_undegraded":"2023-10-14T10:37:01.788260+0000","last_fullsized":"2023-10-14T10:37:01.788260+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:44:49.671921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.489522+0000","last_change":"2023-10-14T10:37:00.076926+0000","last_active":"2023-10-14T10:37:01.489522+0000","last_peered":"2023-10-14T10:37:01.489522+0000","last_clean":"2023-10-14T10:37:01.489522+0000","last_became_active":"2023-10-14T10:37:00.076658+0000","last_became_peered":"2023-10-14T10:37:00.076658+0000","last_unstale":"2023-10-14T10:37:01.489522+0000","last_undegraded":"2023-10-14T10:37:01.489522+0000","last_fullsized":"2023-10-14T10:37:01.489522+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:52:37.894720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.060490+0000","last_change":"2023-10-14T10:36:59.071714+0000","last_active":"2023-10-14T10:37:01.060490+0000","last_peered":"2023-10-14T10:37:01.060490+0000","last_clean":"2023-10-14T10:37:01.060490+0000","last_became_active":"2023-10-14T10:36:59.071406+0000","last_became_peered":"2023-10-14T10:36:59.071406+0000","last_unstale":"2023-10-14T10:37:01.060490+0000","last_undegraded":"2023-10-14T10:37:01.060490+0000","last_fullsized":"2023-10-14T10:37:01.060490+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:59:06.202215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.062345+0000","last_change":"2023-10-14T10:37:00.080060+0000","last_active":"2023-10-14T10:37:01.062345+0000","last_peered":"2023-10-14T10:37:01.062345+0000","last_clean":"2023-10-14T10:37:01.062345+0000","last_became_active":"2023-10-14T10:37:00.079795+0000","last_became_peered":"2023-10-14T10:37:00.079795+0000","last_unstale":"2023-10-14T10:37:01.062345+0000","last_undegraded":"2023-10-14T10:37:01.062345+0000","last_fullsized":"2023-10-14T10:37:01.062345+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T17:27:50.406084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.489554+0000","last_change":"2023-10-14T10:36:59.065271+0000","last_active":"2023-10-14T10:37:01.489554+0000","last_peered":"2023-10-14T10:37:01.489554+0000","last_clean":"2023-10-14T10:37:01.489554+0000","last_became_active":"2023-10-14T10:36:59.064959+0000","last_became_peered":"2023-10-14T10:36:59.064959+0000","last_unstale":"2023-10-14T10:37:01.489554+0000","last_undegraded":"2023-10-14T10:37:01.489554+0000","last_fullsized":"2023-10-14T10:37:01.489554+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T21:56:55.137838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.060192+0000","last_change":"2023-10-14T10:37:00.077751+0000","last_active":"2023-10-14T10:37:01.060192+0000","last_peered":"2023-10-14T10:37:01.060192+0000","last_clean":"2023-10-14T10:37:01.060192+0000","last_became_active":"2023-10-14T10:37:00.077459+0000","last_became_peered":"2023-10-14T10:37:00.077459+0000","last_unstale":"2023-10-14T10:37:01.060192+0000","last_undegraded":"2023-10-14T10:37:01.060192+0000","last_fullsized":"2023-10-14T10:37:01.060192+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T20:27:28.798883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.489473+0000","last_change":"2023-10-14T10:36:59.058893+0000","last_active":"2023-10-14T10:37:01.489473+0000","last_peered":"2023-10-14T10:37:01.489473+0000","last_clean":"2023-10-14T10:37:01.489473+0000","last_became_active":"2023-10-14T10:36:59.058558+0000","last_became_peered":"2023-10-14T10:36:59.058558+0000","last_unstale":"2023-10-14T10:37:01.489473+0000","last_undegraded":"2023-10-14T10:37:01.489473+0000","last_fullsized":"2023-10-14T10:37:01.489473+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:45:50.919718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788693+0000","last_change":"2023-10-14T10:37:00.071398+0000","last_active":"2023-10-14T10:37:01.788693+0000","last_peered":"2023-10-14T10:37:01.788693+0000","last_clean":"2023-10-14T10:37:01.788693+0000","last_became_active":"2023-10-14T10:37:00.071167+0000","last_became_peered":"2023-10-14T10:37:00.071167+0000","last_unstale":"2023-10-14T10:37:01.788693+0000","last_undegraded":"2023-10-14T10:37:01.788693+0000","last_fullsized":"2023-10-14T10:37:01.788693+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:31:12.123956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.489473+0000","last_change":"2023-10-14T10:36:59.069420+0000","last_active":"2023-10-14T10:37:01.489473+0000","last_peered":"2023-10-14T10:37:01.489473+0000","last_clean":"2023-10-14T10:37:01.489473+0000","last_became_active":"2023-10-14T10:36:59.069110+0000","last_became_peered":"2023-10-14T10:36:59.069110+0000","last_unstale":"2023-10-14T10:37:01.489473+0000","last_undegraded":"2023-10-14T10:37:01.489473+0000","last_fullsized":"2023-10-14T10:37:01.489473+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:09:59.812757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788586+0000","last_change":"2023-10-14T10:37:00.077550+0000","last_active":"2023-10-14T10:37:01.788586+0000","last_peered":"2023-10-14T10:37:01.788586+0000","last_clean":"2023-10-14T10:37:01.788586+0000","last_became_active":"2023-10-14T10:37:00.077175+0000","last_became_peered":"2023-10-14T10:37:00.077175+0000","last_unstale":"2023-10-14T10:37:01.788586+0000","last_undegraded":"2023-10-14T10:37:01.788586+0000","last_fullsized":"2023-10-14T10:37:01.788586+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T14:51:19.968923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.787861+0000","last_change":"2023-10-14T10:36:59.067621+0000","last_active":"2023-10-14T10:37:01.787861+0000","last_peered":"2023-10-14T10:37:01.787861+0000","last_clean":"2023-10-14T10:37:01.787861+0000","last_became_active":"2023-10-14T10:36:59.067345+0000","last_became_peered":"2023-10-14T10:36:59.067345+0000","last_unstale":"2023-10-14T10:37:01.787861+0000","last_undegraded":"2023-10-14T10:37:01.787861+0000","last_fullsized":"2023-10-14T10:37:01.787861+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:54:15.011628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788501+0000","last_change":"2023-10-14T10:37:00.077744+0000","last_active":"2023-10-14T10:37:01.788501+0000","last_peered":"2023-10-14T10:37:01.788501+0000","last_clean":"2023-10-14T10:37:01.788501+0000","last_became_active":"2023-10-14T10:37:00.077465+0000","last_became_peered":"2023-10-14T10:37:00.077465+0000","last_unstale":"2023-10-14T10:37:01.788501+0000","last_undegraded":"2023-10-14T10:37:01.788501+0000","last_fullsized":"2023-10-14T10:37:01.788501+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T20:36:23.971821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.787432+0000","last_change":"2023-10-14T10:36:59.069707+0000","last_active":"2023-10-14T10:37:01.787432+0000","last_peered":"2023-10-14T10:37:01.787432+0000","last_clean":"2023-10-14T10:37:01.787432+0000","last_became_active":"2023-10-14T10:36:59.069269+0000","last_became_peered":"2023-10-14T10:36:59.069269+0000","last_unstale":"2023-10-14T10:37:01.787432+0000","last_undegraded":"2023-10-14T10:37:01.787432+0000","last_fullsized":"2023-10-14T10:37:01.787432+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T12:15:16.823069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788221+0000","last_change":"2023-10-14T10:37:00.068887+0000","last_active":"2023-10-14T10:37:01.788221+0000","last_peered":"2023-10-14T10:37:01.788221+0000","last_clean":"2023-10-14T10:37:01.788221+0000","last_became_active":"2023-10-14T10:37:00.068620+0000","last_became_peered":"2023-10-14T10:37:00.068620+0000","last_unstale":"2023-10-14T10:37:01.788221+0000","last_undegraded":"2023-10-14T10:37:01.788221+0000","last_fullsized":"2023-10-14T10:37:01.788221+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T18:04:59.338050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.489058+0000","last_change":"2023-10-14T10:36:59.061947+0000","last_active":"2023-10-14T10:37:01.489058+0000","last_peered":"2023-10-14T10:37:01.489058+0000","last_clean":"2023-10-14T10:37:01.489058+0000","last_became_active":"2023-10-14T10:36:59.061678+0000","last_became_peered":"2023-10-14T10:36:59.061678+0000","last_unstale":"2023-10-14T10:37:01.489058+0000","last_undegraded":"2023-10-14T10:37:01.489058+0000","last_fullsized":"2023-10-14T10:37:01.489058+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:39:01.994329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.061998+0000","last_change":"2023-10-14T10:37:00.073204+0000","last_active":"2023-10-14T10:37:01.061998+0000","last_peered":"2023-10-14T10:37:01.061998+0000","last_clean":"2023-10-14T10:37:01.061998+0000","last_became_active":"2023-10-14T10:37:00.071692+0000","last_became_peered":"2023-10-14T10:37:00.071692+0000","last_unstale":"2023-10-14T10:37:01.061998+0000","last_undegraded":"2023-10-14T10:37:01.061998+0000","last_fullsized":"2023-10-14T10:37:01.061998+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:17:54.730152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.061467+0000","last_change":"2023-10-14T10:36:59.063545+0000","last_active":"2023-10-14T10:37:01.061467+0000","last_peered":"2023-10-14T10:37:01.061467+0000","last_clean":"2023-10-14T10:37:01.061467+0000","last_became_active":"2023-10-14T10:36:59.063332+0000","last_became_peered":"2023-10-14T10:36:59.063332+0000","last_unstale":"2023-10-14T10:37:01.061467+0000","last_undegraded":"2023-10-14T10:37:01.061467+0000","last_fullsized":"2023-10-14T10:37:01.061467+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:10:45.467136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788558+0000","last_change":"2023-10-14T10:37:00.075249+0000","last_active":"2023-10-14T10:37:01.788558+0000","last_peered":"2023-10-14T10:37:01.788558+0000","last_clean":"2023-10-14T10:37:01.788558+0000","last_became_active":"2023-10-14T10:37:00.074859+0000","last_became_peered":"2023-10-14T10:37:00.074859+0000","last_unstale":"2023-10-14T10:37:01.788558+0000","last_undegraded":"2023-10-14T10:37:01.788558+0000","last_fullsized":"2023-10-14T10:37:01.788558+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:04:51.088268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.060369+0000","last_change":"2023-10-14T10:36:59.069418+0000","last_active":"2023-10-14T10:37:01.060369+0000","last_peered":"2023-10-14T10:37:01.060369+0000","last_clean":"2023-10-14T10:37:01.060369+0000","last_became_active":"2023-10-14T10:36:59.069037+0000","last_became_peered":"2023-10-14T10:36:59.069037+0000","last_unstale":"2023-10-14T10:37:01.060369+0000","last_undegraded":"2023-10-14T10:37:01.060369+0000","last_fullsized":"2023-10-14T10:37:01.060369+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:45:50.919718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.062253+0000","last_change":"2023-10-14T10:37:00.075731+0000","last_active":"2023-10-14T10:37:01.062253+0000","last_peered":"2023-10-14T10:37:01.062253+0000","last_clean":"2023-10-14T10:37:01.062253+0000","last_became_active":"2023-10-14T10:37:00.075491+0000","last_became_peered":"2023-10-14T10:37:00.075491+0000","last_unstale":"2023-10-14T10:37:01.062253+0000","last_undegraded":"2023-10-14T10:37:01.062253+0000","last_fullsized":"2023-10-14T10:37:01.062253+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T14:52:18.060480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788031+0000","last_change":"2023-10-14T10:36:59.066461+0000","last_active":"2023-10-14T10:37:01.788031+0000","last_peered":"2023-10-14T10:37:01.788031+0000","last_clean":"2023-10-14T10:37:01.788031+0000","last_became_active":"2023-10-14T10:36:59.066146+0000","last_became_peered":"2023-10-14T10:36:59.066146+0000","last_unstale":"2023-10-14T10:37:01.788031+0000","last_undegraded":"2023-10-14T10:37:01.788031+0000","last_fullsized":"2023-10-14T10:37:01.788031+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:45:50.919718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.787608+0000","last_change":"2023-10-14T10:37:00.082851+0000","last_active":"2023-10-14T10:37:01.787608+0000","last_peered":"2023-10-14T10:37:01.787608+0000","last_clean":"2023-10-14T10:37:01.787608+0000","last_became_active":"2023-10-14T10:37:00.082570+0000","last_became_peered":"2023-10-14T10:37:00.082570+0000","last_unstale":"2023-10-14T10:37:01.787608+0000","last_undegraded":"2023-10-14T10:37:01.787608+0000","last_fullsized":"2023-10-14T10:37:01.787608+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:28:45.821377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.059981+0000","last_change":"2023-10-14T10:36:59.061936+0000","last_active":"2023-10-14T10:37:01.059981+0000","last_peered":"2023-10-14T10:37:01.059981+0000","last_clean":"2023-10-14T10:37:01.059981+0000","last_became_active":"2023-10-14T10:36:59.061642+0000","last_became_peered":"2023-10-14T10:36:59.061642+0000","last_unstale":"2023-10-14T10:37:01.059981+0000","last_undegraded":"2023-10-14T10:37:01.059981+0000","last_fullsized":"2023-10-14T10:37:01.059981+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:54:15.011628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.787449+0000","last_change":"2023-10-14T10:37:00.071551+0000","last_active":"2023-10-14T10:37:01.787449+0000","last_peered":"2023-10-14T10:37:01.787449+0000","last_clean":"2023-10-14T10:37:01.787449+0000","last_became_active":"2023-10-14T10:37:00.071244+0000","last_became_peered":"2023-10-14T10:37:00.071244+0000","last_unstale":"2023-10-14T10:37:01.787449+0000","last_undegraded":"2023-10-14T10:37:01.787449+0000","last_fullsized":"2023-10-14T10:37:01.787449+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:08:06.341692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.787778+0000","last_change":"2023-10-14T10:36:59.066839+0000","last_active":"2023-10-14T10:37:01.787778+0000","last_peered":"2023-10-14T10:37:01.787778+0000","last_clean":"2023-10-14T10:37:01.787778+0000","last_became_active":"2023-10-14T10:36:59.066592+0000","last_became_peered":"2023-10-14T10:36:59.066592+0000","last_unstale":"2023-10-14T10:37:01.787778+0000","last_undegraded":"2023-10-14T10:37:01.787778+0000","last_fullsized":"2023-10-14T10:37:01.787778+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T12:15:16.823069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788013+0000","last_change":"2023-10-14T10:37:00.076076+0000","last_active":"2023-10-14T10:37:01.788013+0000","last_peered":"2023-10-14T10:37:01.788013+0000","last_clean":"2023-10-14T10:37:01.788013+0000","last_became_active":"2023-10-14T10:37:00.075794+0000","last_became_peered":"2023-10-14T10:37:00.075794+0000","last_unstale":"2023-10-14T10:37:01.788013+0000","last_undegraded":"2023-10-14T10:37:01.788013+0000","last_fullsized":"2023-10-14T10:37:01.788013+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:44:50.678288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.060021+0000","last_change":"2023-10-14T10:36:59.065626+0000","last_active":"2023-10-14T10:37:01.060021+0000","last_peered":"2023-10-14T10:37:01.060021+0000","last_clean":"2023-10-14T10:37:01.060021+0000","last_became_active":"2023-10-14T10:36:59.065163+0000","last_became_peered":"2023-10-14T10:36:59.065163+0000","last_unstale":"2023-10-14T10:37:01.060021+0000","last_undegraded":"2023-10-14T10:37:01.060021+0000","last_fullsized":"2023-10-14T10:37:01.060021+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:08:05.335325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788427+0000","last_change":"2023-10-14T10:37:00.078024+0000","last_active":"2023-10-14T10:37:01.788427+0000","last_peered":"2023-10-14T10:37:01.788427+0000","last_clean":"2023-10-14T10:37:01.788427+0000","last_became_active":"2023-10-14T10:37:00.077621+0000","last_became_peered":"2023-10-14T10:37:00.077621+0000","last_unstale":"2023-10-14T10:37:01.788427+0000","last_undegraded":"2023-10-14T10:37:01.788427+0000","last_fullsized":"2023-10-14T10:37:01.788427+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:06:28.391046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.489094+0000","last_change":"2023-10-14T10:36:59.064632+0000","last_active":"2023-10-14T10:37:01.489094+0000","last_peered":"2023-10-14T10:37:01.489094+0000","last_clean":"2023-10-14T10:37:01.489094+0000","last_became_active":"2023-10-14T10:36:59.064336+0000","last_became_peered":"2023-10-14T10:36:59.064336+0000","last_unstale":"2023-10-14T10:37:01.489094+0000","last_undegraded":"2023-10-14T10:37:01.489094+0000","last_fullsized":"2023-10-14T10:37:01.489094+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:08:05.335325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.488971+0000","last_change":"2023-10-14T10:37:00.081042+0000","last_active":"2023-10-14T10:37:01.488971+0000","last_peered":"2023-10-14T10:37:01.488971+0000","last_clean":"2023-10-14T10:37:01.488971+0000","last_became_active":"2023-10-14T10:37:00.080340+0000","last_became_peered":"2023-10-14T10:37:00.080340+0000","last_unstale":"2023-10-14T10:37:01.488971+0000","last_undegraded":"2023-10-14T10:37:01.488971+0000","last_fullsized":"2023-10-14T10:37:01.488971+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:44:50.678288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788029+0000","last_change":"2023-10-14T10:36:59.063568+0000","last_active":"2023-10-14T10:37:01.788029+0000","last_peered":"2023-10-14T10:37:01.788029+0000","last_clean":"2023-10-14T10:37:01.788029+0000","last_became_active":"2023-10-14T10:36:59.063130+0000","last_became_peered":"2023-10-14T10:36:59.063130+0000","last_unstale":"2023-10-14T10:37:01.788029+0000","last_undegraded":"2023-10-14T10:37:01.788029+0000","last_fullsized":"2023-10-14T10:37:01.788029+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:59:06.202215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.061998+0000","last_change":"2023-10-14T10:37:00.068870+0000","last_active":"2023-10-14T10:37:01.061998+0000","last_peered":"2023-10-14T10:37:01.061998+0000","last_clean":"2023-10-14T10:37:01.061998+0000","last_became_active":"2023-10-14T10:37:00.068556+0000","last_became_peered":"2023-10-14T10:37:00.068556+0000","last_unstale":"2023-10-14T10:37:01.061998+0000","last_undegraded":"2023-10-14T10:37:01.061998+0000","last_fullsized":"2023-10-14T10:37:01.061998+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T14:52:18.060480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.787484+0000","last_change":"2023-10-14T10:36:59.066703+0000","last_active":"2023-10-14T10:37:01.787484+0000","last_peered":"2023-10-14T10:37:01.787484+0000","last_clean":"2023-10-14T10:37:01.787484+0000","last_became_active":"2023-10-14T10:36:59.066424+0000","last_became_peered":"2023-10-14T10:36:59.066424+0000","last_unstale":"2023-10-14T10:37:01.787484+0000","last_undegraded":"2023-10-14T10:37:01.787484+0000","last_fullsized":"2023-10-14T10:37:01.787484+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:54:15.011628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":176,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-14T10:37:00.060617+0000","last_change":"2023-10-14T10:35:59.770163+0000","last_active":"2023-10-14T10:37:00.060617+0000","last_peered":"2023-10-14T10:37:00.060617+0000","last_clean":"2023-10-14T10:37:00.060617+0000","last_became_active":"2023-10-14T10:35:59.769199+0000","last_became_peered":"2023-10-14T10:35:59.769199+0000","last_unstale":"2023-10-14T10:37:00.060617+0000","last_undegraded":"2023-10-14T10:37:00.060617+0000","last_fullsized":"2023-10-14T10:37:00.060617+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:35:58.675572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:35:58.675572+0000","last_clean_scrub_stamp":"2023-10-14T10:35:58.675572+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:03:25.281187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542161,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27956,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343884,"statfs":{"total":96636764160,"available":96608137216,"internally_reserved":0,"allocated":954368,"data_stored":726269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 10:36:59 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":2.4649999999999999,"5min":2.4649999999999999,"15min":2.4649999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.50700000000000001}]},{"osd":1,"last update":"Sat Oct 14 10:36:59 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":2.4289999999999998,"5min":2.4289999999999998,"15min":2.4289999999999998},"last":0.43099999999999999}]},{"osd":2,"last update":"Sat Oct 14 10:36:59 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.54800000000000004}]},{"osd":3,"last update":"Sat Oct 14 10:36:59 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.405,"5min":1.405,"15min":1.405},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.56100000000000005}]},{"osd":4,"last update":"Sat Oct 14 10:36:59 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.57899999999999996}]},{"osd":5,"last update":"Sat Oct 14 10:36:59 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.59299999999999997}]},{"osd":6,"last update":"Sat Oct 14 10:36:59 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.623}]}]},{"osd":6,"up_from":14,"seq":60129542162,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27388,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344452,"statfs":{"total":96636764160,"available":96608718848,"internally_reserved":0,"allocated":376832,"data_stored":142752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 10:36:57 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.34599999999999997}]},{"osd":1,"last update":"Sat Oct 14 10:36:57 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.389,"5min":1.389,"15min":1.389},"last":0.60299999999999998}]},{"osd":2,"last update":"Sat Oct 14 10:36:57 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.436},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.53000000000000003}]},{"osd":3,"last update":"Sat Oct 14 10:36:57 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.56599999999999995}]},{"osd":4,"last update":"Sat Oct 14 10:36:57 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.65800000000000003}]},{"osd":5,"last update":"Sat Oct 14 10:36:57 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.61899999999999999}]},{"osd":7,"last update":"Sat Oct 14 10:36:57 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.63400000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542161,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":376,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":385024,"data_stored":143770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.254,"15min":1.254},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":16.855,"5min":16.855,"15min":16.855},"last":0.311},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":19.353000000000002,"5min":19.353000000000002,"15min":19.353000000000002},"last":0.371}]},{"osd":2,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":3.0670000000000002,"5min":3.0670000000000002,"15min":3.0670000000000002},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.35399999999999998}]},{"osd":3,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":10.005000000000001,"5min":10.005000000000001,"15min":10.005000000000001},"last":0.33500000000000002}]},{"osd":4,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":2.9500000000000002,"5min":2.9500000000000002,"15min":2.9500000000000002},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":5.7270000000000003,"5min":5.7270000000000003,"15min":5.7270000000000003},"last":0.59399999999999997}]},{"osd":5,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.129,"15min":1.129},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":9.8800000000000008,"5min":9.8800000000000008,"15min":9.8800000000000008},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":3.0350000000000001,"5min":3.0350000000000001,"15min":3.0350000000000001},"last":0.56499999999999995}]},{"osd":6,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":5.681,"5min":5.681,"15min":5.681},"last":0.72299999999999998},{"interface":"front","average":{"1min":1.173,"5min":1.173,"15min":1.173},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":9.9380000000000006,"5min":9.9380000000000006,"15min":9.9380000000000006},"last":0.78600000000000003}]},{"osd":7,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":9.9730000000000008,"5min":9.9730000000000008,"15min":9.9730000000000008},"last":0.626},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":3.0009999999999999,"5min":3.0009999999999999,"15min":3.0009999999999999},"last":0.73799999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542160,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27952,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343888,"statfs":{"total":96636764160,"available":96608141312,"internally_reserved":0,"allocated":950272,"data_stored":726179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.62,"5min":0.62,"15min":0.62},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.495,"5min":0.495,"15min":0.495},"last":0.34000000000000002}]},{"osd":2,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"last":0.23999999999999999}]},{"osd":3,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"last":0.39900000000000002}]},{"osd":4,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.58599999999999997}]},{"osd":5,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.60099999999999998}]},{"osd":6,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.54700000000000004}]},{"osd":7,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.56599999999999995}]}]},{"osd":2,"up_from":14,"seq":60129542161,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27376,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344464,"statfs":{"total":96636764160,"available":96608731136,"internally_reserved":0,"allocated":360448,"data_stored":135881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":1.429,"5min":1.429,"15min":1.429},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.35199999999999998}]},{"osd":1,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":1.5269999999999999,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":1.0780000000000001},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":1.1000000000000001}]},{"osd":3,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":4.0629999999999997,"5min":4.0629999999999997,"15min":4.0629999999999997},"last":4.0629999999999997},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.381,"5min":1.381,"15min":1.381},"last":0.46000000000000002}]},{"osd":4,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.55200000000000005}]},{"osd":5,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":3.8599999999999999,"5min":3.8599999999999999,"15min":3.8599999999999999},"last":3.8599999999999999},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.464,"5min":1.464,"15min":1.464},"last":0.52300000000000002}]},{"osd":6,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":3.9820000000000002,"5min":3.9820000000000002,"15min":3.9820000000000002},"last":3.9820000000000002}]},{"osd":7,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.62},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":3.9049999999999998,"5min":3.9049999999999998,"15min":3.9049999999999998},"last":3.9049999999999998}]}]},{"osd":3,"up_from":14,"seq":60129542161,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":376,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":385024,"data_stored":143770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.441},{"interface":"front","average":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"last":0.29099999999999998}]},{"osd":1,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"last":0.501}]},{"osd":2,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.437,"5min":0.437,"15min":0.437},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.55800000000000005}]},{"osd":4,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":6.0350000000000001,"5min":6.0350000000000001,"15min":6.0350000000000001},"last":0.66800000000000004}]},{"osd":5,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":6.0540000000000003,"5min":6.0540000000000003,"15min":6.0540000000000003},"last":0.59599999999999997}]},{"osd":6,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.61299999999999999}]},{"osd":7,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":6.0819999999999999,"5min":6.0819999999999999,"15min":6.0819999999999999},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.63300000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542160,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":364544,"data_stored":135915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 10:36:56 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.59499999999999997}]},{"osd":1,"last update":"Sat Oct 14 10:36:56 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.64600000000000002}]},{"osd":2,"last update":"Sat Oct 14 10:36:56 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.51100000000000001}]},{"osd":3,"last update":"Sat Oct 14 10:36:56 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.624},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.68000000000000005}]},{"osd":5,"last update":"Sat Oct 14 10:36:56 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.70199999999999996}]},{"osd":6,"last update":"Sat Oct 14 10:36:56 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.84099999999999997}]},{"osd":7,"last update":"Sat Oct 14 10:36:56 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.75800000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542161,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":364544,"data_stored":136483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.72399999999999998}]},{"osd":1,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.627},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.66600000000000004}]},{"osd":2,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.441}]},{"osd":3,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.495},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.68500000000000005}]},{"osd":4,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.60099999999999998}]},{"osd":6,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.70499999999999996}]},{"osd":7,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.745}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-14T10:37:04.219 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T10:37:04.376 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.374+0000 7f6910e39700 1 Processor -- start 2023-10-14T10:37:04.377 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.374+0000 7f6910e39700 1 -- start start 2023-10-14T10:37:04.379 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.375+0000 7f6910e39700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f690c15f130 0x7f690c0cede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:04.379 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.375+0000 7f6910e39700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f690c0b28b0 0x7f690c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:04.380 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.375+0000 7f6910e39700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f690c0b3530 con 0x7f690c06ad80 2023-10-14T10:37:04.380 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.375+0000 7f6910e39700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f690c0cde00 con 0x7f690c15f130 2023-10-14T10:37:04.380 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.375+0000 7f6910e39700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f690c06b1b0 con 0x7f690c0b28b0 2023-10-14T10:37:04.380 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.376+0000 7f690a59c700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f690c15f130 0x7f690c0cede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:04.381 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.376+0000 7f690a59c700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f690c15f130 0x7f690c0cede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45846/0 (socket says 172.21.15.67:45846) 2023-10-14T10:37:04.381 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.376+0000 7f690a59c700 1 -- 172.21.15.67:0/86775793 learned_addr learned my addr 172.21.15.67:0/86775793 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:37:04.381 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.376+0000 7f6909d9b700 1 --2- 172.21.15.67:0/86775793 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f690c0b28b0 0x7f690c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:04.381 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.376+0000 7f690a59c700 1 -- 172.21.15.67:0/86775793 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f690c0b28b0 msgr2=0x7f690c06a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:37:04.381 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.376+0000 7f690a59c700 1 --2- 172.21.15.67:0/86775793 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f690c0b28b0 0x7f690c06a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:04.382 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.376+0000 7f690a59c700 1 -- 172.21.15.67:0/86775793 >> v1:172.21.15.72:6789/0 conn(0x7f690c06ad80 legacy=0x7f690c071af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:04.382 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.376+0000 7f690a59c700 1 -- 172.21.15.67:0/86775793 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f690c0cd180 con 0x7f690c15f130 2023-10-14T10:37:04.382 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.376+0000 7f6909d9b700 1 --2- 172.21.15.67:0/86775793 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f690c0b28b0 0x7f690c06a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-14T10:37:04.382 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.376+0000 7f690a59c700 1 --2- 172.21.15.67:0/86775793 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f690c15f130 0x7f690c0cede0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f68fc0098f0 tx=0x7f68fc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7fd539c9106aec7 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:04.382 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.376+0000 7f690959a700 1 -- 172.21.15.67:0/86775793 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68fc00b8a0 con 0x7f690c15f130 2023-10-14T10:37:04.383 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.376+0000 7f690959a700 1 -- 172.21.15.67:0/86775793 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68fc00ba00 con 0x7f690c15f130 2023-10-14T10:37:04.383 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.376+0000 7f690959a700 1 -- 172.21.15.67:0/86775793 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68fc00bd90 con 0x7f690c15f130 2023-10-14T10:37:04.383 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.377+0000 7f6910e39700 1 -- 172.21.15.67:0/86775793 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f690c15f130 msgr2=0x7f690c0cede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:04.383 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.377+0000 7f6910e39700 1 --2- 172.21.15.67:0/86775793 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f690c15f130 0x7f690c0cede0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f68fc0098f0 tx=0x7f68fc009c00 comp rx=0 tx=0).stop 2023-10-14T10:37:04.383 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.377+0000 7f6910e39700 1 -- 172.21.15.67:0/86775793 shutdown_connections 2023-10-14T10:37:04.384 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.377+0000 7f6910e39700 1 --2- 172.21.15.67:0/86775793 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f690c15f130 0x7f690c0cede0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:04.384 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.377+0000 7f6910e39700 1 --2- 172.21.15.67:0/86775793 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f690c0b28b0 0x7f690c06a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:04.384 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.377+0000 7f6910e39700 1 -- 172.21.15.67:0/86775793 >> 172.21.15.67:0/86775793 conn(0x7f690c06dae0 msgr2=0x7f690c06def0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:37:04.384 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.377+0000 7f6910e39700 1 -- 172.21.15.67:0/86775793 shutdown_connections 2023-10-14T10:37:04.384 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.377+0000 7f6910e39700 1 -- 172.21.15.67:0/86775793 wait complete. 2023-10-14T10:37:04.387 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.378+0000 7f6910e39700 1 Processor -- start 2023-10-14T10:37:04.388 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.378+0000 7f6910e39700 1 -- start start 2023-10-14T10:37:04.388 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.379+0000 7f6910e39700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f690c06ad80 0x7f690c1f4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:04.388 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.379+0000 7f6910e39700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f690c0b28b0 0x7f690c152280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:04.388 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.379+0000 7f6910e39700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f690c15f130 0x7f690c15cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:04.388 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.379+0000 7f6910e39700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f690c1f54f0 con 0x7f690c06ad80 2023-10-14T10:37:04.389 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.379+0000 7f6910e39700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f690c158430 con 0x7f690c0b28b0 2023-10-14T10:37:04.389 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.379+0000 7f6910e39700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f690c158710 con 0x7f690c15f130 2023-10-14T10:37:04.389 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.379+0000 7f690a59c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f690c06ad80 0x7f690c1f4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:04.389 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.379+0000 7f690a59c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f690c06ad80 0x7f690c1f4fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45852/0 (socket says 172.21.15.67:45852) 2023-10-14T10:37:04.389 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.379+0000 7f690a59c700 1 -- 172.21.15.67:0/586025072 learned_addr learned my addr 172.21.15.67:0/586025072 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:37:04.390 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.379+0000 7f6909d9b700 1 --2- 172.21.15.67:0/586025072 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f690c0b28b0 0x7f690c152280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:04.390 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.379+0000 7f690ad9d700 1 --2- 172.21.15.67:0/586025072 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f690c15f130 0x7f690c15cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:04.390 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.380+0000 7f690a59c700 1 -- 172.21.15.67:0/586025072 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f690c15f130 msgr2=0x7f690c15cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:04.390 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.380+0000 7f690a59c700 1 --2- 172.21.15.67:0/586025072 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f690c15f130 0x7f690c15cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:04.390 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.380+0000 7f690a59c700 1 -- 172.21.15.67:0/586025072 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f690c0b28b0 msgr2=0x7f690c152280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:04.391 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.380+0000 7f690a59c700 1 --2- 172.21.15.67:0/586025072 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f690c0b28b0 0x7f690c152280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:04.391 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.380+0000 7f690a59c700 1 -- 172.21.15.67:0/586025072 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68fc009550 con 0x7f690c06ad80 2023-10-14T10:37:04.391 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.380+0000 7f690a59c700 1 --2- 172.21.15.67:0/586025072 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f690c06ad80 0x7f690c1f4fb0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f68fc009c40 tx=0x7f68fc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:04.391 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.380+0000 7f68faffd700 1 -- 172.21.15.67:0/586025072 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68fc0036a0 con 0x7f690c06ad80 2023-10-14T10:37:04.391 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.380+0000 7f68faffd700 1 -- 172.21.15.67:0/586025072 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68fc01f070 con 0x7f690c06ad80 2023-10-14T10:37:04.392 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.380+0000 7f6910e39700 1 -- 172.21.15.67:0/586025072 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f690c1589f0 con 0x7f690c06ad80 2023-10-14T10:37:04.392 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.380+0000 7f68faffd700 1 -- 172.21.15.67:0/586025072 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68fc003a60 con 0x7f690c06ad80 2023-10-14T10:37:04.392 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.380+0000 7f6910e39700 1 -- 172.21.15.67:0/586025072 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f690c158fb0 con 0x7f690c06ad80 2023-10-14T10:37:04.392 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.381+0000 7f6910e39700 1 -- 172.21.15.67:0/586025072 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f690c0cd720 con 0x7f690c06ad80 2023-10-14T10:37:04.392 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.381+0000 7f68faffd700 1 -- 172.21.15.67:0/586025072 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f68fc003bc0 con 0x7f690c06ad80 2023-10-14T10:37:04.393 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.382+0000 7f68faffd700 1 --2- 172.21.15.67:0/586025072 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f68ec06af00 0x7f68ec06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:04.393 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.382+0000 7f68faffd700 1 -- 172.21.15.67:0/586025072 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5729+0+0 (secure 0 0 0) 0x7f68fc021070 con 0x7f690c06ad80 2023-10-14T10:37:04.393 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.382+0000 7f6909d9b700 1 --2- 172.21.15.67:0/586025072 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f68ec06af00 0x7f68ec06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:04.393 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.382+0000 7f6909d9b700 1 --2- 172.21.15.67:0/586025072 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f68ec06af00 0x7f68ec06d3c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f690c0c64a0 tx=0x7f6900008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:04.394 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.386+0000 7f68faffd700 1 -- 172.21.15.67:0/586025072 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f690c0cd720 con 0x7f690c06ad80 2023-10-14T10:37:04.584 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.582+0000 7f6910e39700 1 -- 172.21.15.67:0/586025072 --> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f690c158880 con 0x7f68ec06af00 2023-10-14T10:37:04.592 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.590+0000 7f68faffd700 1 -- 172.21.15.67:0/586025072 <== mgr.4100 v2:172.21.15.72:6832/703488382 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289591 (secure 0 0 0) 0x7f690c158880 con 0x7f68ec06af00 2023-10-14T10:37:04.595 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-14T10:37:04.598 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.593+0000 7f6910e39700 1 -- 172.21.15.67:0/586025072 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f68ec06af00 msgr2=0x7f68ec06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:04.598 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.593+0000 7f6910e39700 1 --2- 172.21.15.67:0/586025072 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f68ec06af00 0x7f68ec06d3c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f690c0c64a0 tx=0x7f6900008040 comp rx=0 tx=0).stop 2023-10-14T10:37:04.598 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.593+0000 7f6910e39700 1 -- 172.21.15.67:0/586025072 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f690c06ad80 msgr2=0x7f690c1f4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:04.599 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.593+0000 7f6910e39700 1 --2- 172.21.15.67:0/586025072 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f690c06ad80 0x7f690c1f4fb0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f68fc009c40 tx=0x7f68fc004750 comp rx=0 tx=0).stop 2023-10-14T10:37:04.599 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.593+0000 7f6910e39700 1 -- 172.21.15.67:0/586025072 shutdown_connections 2023-10-14T10:37:04.599 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.593+0000 7f6910e39700 1 --2- 172.21.15.67:0/586025072 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f68ec06af00 0x7f68ec06d3c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:04.599 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.593+0000 7f6910e39700 1 --2- 172.21.15.67:0/586025072 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f690c15f130 0x7f690c15cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:04.599 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.593+0000 7f6910e39700 1 --2- 172.21.15.67:0/586025072 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f690c06ad80 0x7f690c1f4fb0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:04.600 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.593+0000 7f6910e39700 1 --2- 172.21.15.67:0/586025072 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f690c0b28b0 0x7f690c152280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:04.600 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.593+0000 7f6910e39700 1 -- 172.21.15.67:0/586025072 >> 172.21.15.67:0/586025072 conn(0x7f690c06dae0 msgr2=0x7f690c0b3af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:37:04.600 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.594+0000 7f6910e39700 1 -- 172.21.15.67:0/586025072 shutdown_connections 2023-10-14T10:37:04.600 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.594+0000 7f6910e39700 1 -- 172.21.15.67:0/586025072 wait complete. 2023-10-14T10:37:04.600 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2023-10-14T10:37:04.608 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":60,"stamp":"2023-10-14T10:37:03.769569+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":122,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220232,"kb_used_data":4044,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754488,"statfs":{"total":773094113280,"available":772868595712,"internally_reserved":0,"allocated":4141056,"data_stored":2291019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12742,"internal_metadata":221236794},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.726849"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.787560+0000","last_change":"2023-10-14T10:37:00.075102+0000","last_active":"2023-10-14T10:37:01.787560+0000","last_peered":"2023-10-14T10:37:01.787560+0000","last_clean":"2023-10-14T10:37:01.787560+0000","last_became_active":"2023-10-14T10:37:00.074851+0000","last_became_peered":"2023-10-14T10:37:00.074851+0000","last_unstale":"2023-10-14T10:37:01.787560+0000","last_undegraded":"2023-10-14T10:37:01.787560+0000","last_fullsized":"2023-10-14T10:37:01.787560+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T21:39:35.587728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788402+0000","last_change":"2023-10-14T10:37:00.068871+0000","last_active":"2023-10-14T10:37:01.788402+0000","last_peered":"2023-10-14T10:37:01.788402+0000","last_clean":"2023-10-14T10:37:01.788402+0000","last_became_active":"2023-10-14T10:37:00.068586+0000","last_became_peered":"2023-10-14T10:37:00.068586+0000","last_unstale":"2023-10-14T10:37:01.788402+0000","last_undegraded":"2023-10-14T10:37:01.788402+0000","last_fullsized":"2023-10-14T10:37:01.788402+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T12:47:24.725885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.062215+0000","last_change":"2023-10-14T10:37:00.080988+0000","last_active":"2023-10-14T10:37:01.062215+0000","last_peered":"2023-10-14T10:37:01.062215+0000","last_clean":"2023-10-14T10:37:01.062215+0000","last_became_active":"2023-10-14T10:37:00.080476+0000","last_became_peered":"2023-10-14T10:37:00.080476+0000","last_unstale":"2023-10-14T10:37:01.062215+0000","last_undegraded":"2023-10-14T10:37:01.062215+0000","last_fullsized":"2023-10-14T10:37:01.062215+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T10:48:03.043383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.059874+0000","last_change":"2023-10-14T10:37:00.077598+0000","last_active":"2023-10-14T10:37:01.059874+0000","last_peered":"2023-10-14T10:37:01.059874+0000","last_clean":"2023-10-14T10:37:01.059874+0000","last_became_active":"2023-10-14T10:37:00.077322+0000","last_became_peered":"2023-10-14T10:37:00.077322+0000","last_unstale":"2023-10-14T10:37:01.059874+0000","last_undegraded":"2023-10-14T10:37:01.059874+0000","last_fullsized":"2023-10-14T10:37:01.059874+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:28:04.536948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788360+0000","last_change":"2023-10-14T10:37:00.068342+0000","last_active":"2023-10-14T10:37:01.788360+0000","last_peered":"2023-10-14T10:37:01.788360+0000","last_clean":"2023-10-14T10:37:01.788360+0000","last_became_active":"2023-10-14T10:37:00.068055+0000","last_became_peered":"2023-10-14T10:37:00.068055+0000","last_unstale":"2023-10-14T10:37:01.788360+0000","last_undegraded":"2023-10-14T10:37:01.788360+0000","last_fullsized":"2023-10-14T10:37:01.788360+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T12:52:16.372161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788917+0000","last_change":"2023-10-14T10:37:00.072629+0000","last_active":"2023-10-14T10:37:01.788917+0000","last_peered":"2023-10-14T10:37:01.788917+0000","last_clean":"2023-10-14T10:37:01.788917+0000","last_became_active":"2023-10-14T10:37:00.072345+0000","last_became_peered":"2023-10-14T10:37:00.072345+0000","last_unstale":"2023-10-14T10:37:01.788917+0000","last_undegraded":"2023-10-14T10:37:01.788917+0000","last_fullsized":"2023-10-14T10:37:01.788917+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T17:09:21.565688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.061635+0000","last_change":"2023-10-14T10:37:00.073019+0000","last_active":"2023-10-14T10:37:01.061635+0000","last_peered":"2023-10-14T10:37:01.061635+0000","last_clean":"2023-10-14T10:37:01.061635+0000","last_became_active":"2023-10-14T10:37:00.072368+0000","last_became_peered":"2023-10-14T10:37:00.072368+0000","last_unstale":"2023-10-14T10:37:01.061635+0000","last_undegraded":"2023-10-14T10:37:01.061635+0000","last_fullsized":"2023-10-14T10:37:01.061635+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:31:16.127752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.062210+0000","last_change":"2023-10-14T10:37:00.074091+0000","last_active":"2023-10-14T10:37:01.062210+0000","last_peered":"2023-10-14T10:37:01.062210+0000","last_clean":"2023-10-14T10:37:01.062210+0000","last_became_active":"2023-10-14T10:37:00.073792+0000","last_became_peered":"2023-10-14T10:37:00.073792+0000","last_unstale":"2023-10-14T10:37:01.062210+0000","last_undegraded":"2023-10-14T10:37:01.062210+0000","last_fullsized":"2023-10-14T10:37:01.062210+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T17:27:50.406084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.489588+0000","last_change":"2023-10-14T10:37:00.081745+0000","last_active":"2023-10-14T10:37:01.489588+0000","last_peered":"2023-10-14T10:37:01.489588+0000","last_clean":"2023-10-14T10:37:01.489588+0000","last_became_active":"2023-10-14T10:37:00.081451+0000","last_became_peered":"2023-10-14T10:37:00.081451+0000","last_unstale":"2023-10-14T10:37:01.489588+0000","last_undegraded":"2023-10-14T10:37:01.489588+0000","last_fullsized":"2023-10-14T10:37:01.489588+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:18:45.297555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.060294+0000","last_change":"2023-10-14T10:37:00.071138+0000","last_active":"2023-10-14T10:37:01.060294+0000","last_peered":"2023-10-14T10:37:01.060294+0000","last_clean":"2023-10-14T10:37:01.060294+0000","last_became_active":"2023-10-14T10:37:00.070667+0000","last_became_peered":"2023-10-14T10:37:00.070667+0000","last_unstale":"2023-10-14T10:37:01.060294+0000","last_undegraded":"2023-10-14T10:37:01.060294+0000","last_fullsized":"2023-10-14T10:37:01.060294+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:52:37.894720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.062228+0000","last_change":"2023-10-14T10:37:00.073322+0000","last_active":"2023-10-14T10:37:01.062228+0000","last_peered":"2023-10-14T10:37:01.062228+0000","last_clean":"2023-10-14T10:37:01.062228+0000","last_became_active":"2023-10-14T10:37:00.072639+0000","last_became_peered":"2023-10-14T10:37:00.072639+0000","last_unstale":"2023-10-14T10:37:01.062228+0000","last_undegraded":"2023-10-14T10:37:01.062228+0000","last_fullsized":"2023-10-14T10:37:01.062228+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T18:57:50.136605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.061858+0000","last_change":"2023-10-14T10:37:00.069681+0000","last_active":"2023-10-14T10:37:01.061858+0000","last_peered":"2023-10-14T10:37:01.061858+0000","last_clean":"2023-10-14T10:37:01.061858+0000","last_became_active":"2023-10-14T10:37:00.069373+0000","last_became_peered":"2023-10-14T10:37:00.069373+0000","last_unstale":"2023-10-14T10:37:01.061858+0000","last_undegraded":"2023-10-14T10:37:01.061858+0000","last_fullsized":"2023-10-14T10:37:01.061858+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:31:16.127752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788687+0000","last_change":"2023-10-14T10:37:00.074917+0000","last_active":"2023-10-14T10:37:01.788687+0000","last_peered":"2023-10-14T10:37:01.788687+0000","last_clean":"2023-10-14T10:37:01.788687+0000","last_became_active":"2023-10-14T10:37:00.074473+0000","last_became_peered":"2023-10-14T10:37:00.074473+0000","last_unstale":"2023-10-14T10:37:01.788687+0000","last_undegraded":"2023-10-14T10:37:01.788687+0000","last_fullsized":"2023-10-14T10:37:01.788687+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T20:20:02.187767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788602+0000","last_change":"2023-10-14T10:37:00.073986+0000","last_active":"2023-10-14T10:37:01.788602+0000","last_peered":"2023-10-14T10:37:01.788602+0000","last_clean":"2023-10-14T10:37:01.788602+0000","last_became_active":"2023-10-14T10:37:00.073651+0000","last_became_peered":"2023-10-14T10:37:00.073651+0000","last_unstale":"2023-10-14T10:37:01.788602+0000","last_undegraded":"2023-10-14T10:37:01.788602+0000","last_fullsized":"2023-10-14T10:37:01.788602+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T10:48:03.043383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.061624+0000","last_change":"2023-10-14T10:37:00.072277+0000","last_active":"2023-10-14T10:37:01.061624+0000","last_peered":"2023-10-14T10:37:01.061624+0000","last_clean":"2023-10-14T10:37:01.061624+0000","last_became_active":"2023-10-14T10:37:00.071999+0000","last_became_peered":"2023-10-14T10:37:00.071999+0000","last_unstale":"2023-10-14T10:37:01.061624+0000","last_undegraded":"2023-10-14T10:37:01.061624+0000","last_fullsized":"2023-10-14T10:37:01.061624+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:18:45.297555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.062458+0000","last_change":"2023-10-14T10:37:00.072492+0000","last_active":"2023-10-14T10:37:01.062458+0000","last_peered":"2023-10-14T10:37:01.062458+0000","last_clean":"2023-10-14T10:37:01.062458+0000","last_became_active":"2023-10-14T10:37:00.071509+0000","last_became_peered":"2023-10-14T10:37:00.071509+0000","last_unstale":"2023-10-14T10:37:01.062458+0000","last_undegraded":"2023-10-14T10:37:01.062458+0000","last_fullsized":"2023-10-14T10:37:01.062458+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:51:00.167498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.062381+0000","last_change":"2023-10-14T10:37:00.072597+0000","last_active":"2023-10-14T10:37:01.062381+0000","last_peered":"2023-10-14T10:37:01.062381+0000","last_clean":"2023-10-14T10:37:01.062381+0000","last_became_active":"2023-10-14T10:37:00.071946+0000","last_became_peered":"2023-10-14T10:37:00.071946+0000","last_unstale":"2023-10-14T10:37:01.062381+0000","last_undegraded":"2023-10-14T10:37:01.062381+0000","last_fullsized":"2023-10-14T10:37:01.062381+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:06:28.391046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788267+0000","last_change":"2023-10-14T10:37:00.080581+0000","last_active":"2023-10-14T10:37:01.788267+0000","last_peered":"2023-10-14T10:37:01.788267+0000","last_clean":"2023-10-14T10:37:01.788267+0000","last_became_active":"2023-10-14T10:37:00.080288+0000","last_became_peered":"2023-10-14T10:37:00.080288+0000","last_unstale":"2023-10-14T10:37:01.788267+0000","last_undegraded":"2023-10-14T10:37:01.788267+0000","last_fullsized":"2023-10-14T10:37:01.788267+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T18:57:50.136605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.787886+0000","last_change":"2023-10-14T10:37:00.080484+0000","last_active":"2023-10-14T10:37:01.787886+0000","last_peered":"2023-10-14T10:37:01.787886+0000","last_clean":"2023-10-14T10:37:01.787886+0000","last_became_active":"2023-10-14T10:37:00.080186+0000","last_became_peered":"2023-10-14T10:37:00.080186+0000","last_unstale":"2023-10-14T10:37:01.787886+0000","last_undegraded":"2023-10-14T10:37:01.787886+0000","last_fullsized":"2023-10-14T10:37:01.787886+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:17:54.730152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788501+0000","last_change":"2023-10-14T10:37:00.072056+0000","last_active":"2023-10-14T10:37:01.788501+0000","last_peered":"2023-10-14T10:37:01.788501+0000","last_clean":"2023-10-14T10:37:01.788501+0000","last_became_active":"2023-10-14T10:37:00.071796+0000","last_became_peered":"2023-10-14T10:37:00.071796+0000","last_unstale":"2023-10-14T10:37:01.788501+0000","last_undegraded":"2023-10-14T10:37:01.788501+0000","last_fullsized":"2023-10-14T10:37:01.788501+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T20:39:07.030985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.489206+0000","last_change":"2023-10-14T10:37:00.080796+0000","last_active":"2023-10-14T10:37:01.489206+0000","last_peered":"2023-10-14T10:37:01.489206+0000","last_clean":"2023-10-14T10:37:01.489206+0000","last_became_active":"2023-10-14T10:37:00.080526+0000","last_became_peered":"2023-10-14T10:37:00.080526+0000","last_unstale":"2023-10-14T10:37:01.489206+0000","last_undegraded":"2023-10-14T10:37:01.489206+0000","last_fullsized":"2023-10-14T10:37:01.489206+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:45:25.303185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.489168+0000","last_change":"2023-10-14T10:37:00.080350+0000","last_active":"2023-10-14T10:37:01.489168+0000","last_peered":"2023-10-14T10:37:01.489168+0000","last_clean":"2023-10-14T10:37:01.489168+0000","last_became_active":"2023-10-14T10:37:00.080062+0000","last_became_peered":"2023-10-14T10:37:00.080062+0000","last_unstale":"2023-10-14T10:37:01.489168+0000","last_undegraded":"2023-10-14T10:37:01.489168+0000","last_fullsized":"2023-10-14T10:37:01.489168+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:10:46.473503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.787611+0000","last_change":"2023-10-14T10:37:00.071237+0000","last_active":"2023-10-14T10:37:01.787611+0000","last_peered":"2023-10-14T10:37:01.787611+0000","last_clean":"2023-10-14T10:37:01.787611+0000","last_became_active":"2023-10-14T10:37:00.070897+0000","last_became_peered":"2023-10-14T10:37:00.070897+0000","last_unstale":"2023-10-14T10:37:01.787611+0000","last_undegraded":"2023-10-14T10:37:01.787611+0000","last_fullsized":"2023-10-14T10:37:01.787611+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:59:07.208582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.061365+0000","last_change":"2023-10-14T10:37:00.069525+0000","last_active":"2023-10-14T10:37:01.061365+0000","last_peered":"2023-10-14T10:37:01.061365+0000","last_clean":"2023-10-14T10:37:01.061365+0000","last_became_active":"2023-10-14T10:37:00.069246+0000","last_became_peered":"2023-10-14T10:37:00.069246+0000","last_unstale":"2023-10-14T10:37:01.061365+0000","last_undegraded":"2023-10-14T10:37:01.061365+0000","last_fullsized":"2023-10-14T10:37:01.061365+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:52:37.894720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.060407+0000","last_change":"2023-10-14T10:37:00.070144+0000","last_active":"2023-10-14T10:37:01.060407+0000","last_peered":"2023-10-14T10:37:01.060407+0000","last_clean":"2023-10-14T10:37:01.060407+0000","last_became_active":"2023-10-14T10:37:00.069871+0000","last_became_peered":"2023-10-14T10:37:00.069871+0000","last_unstale":"2023-10-14T10:37:01.060407+0000","last_undegraded":"2023-10-14T10:37:01.060407+0000","last_fullsized":"2023-10-14T10:37:01.060407+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:22:32.154663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788109+0000","last_change":"2023-10-14T10:37:00.076474+0000","last_active":"2023-10-14T10:37:01.788109+0000","last_peered":"2023-10-14T10:37:01.788109+0000","last_clean":"2023-10-14T10:37:01.788109+0000","last_became_active":"2023-10-14T10:37:00.076169+0000","last_became_peered":"2023-10-14T10:37:00.076169+0000","last_unstale":"2023-10-14T10:37:01.788109+0000","last_undegraded":"2023-10-14T10:37:01.788109+0000","last_fullsized":"2023-10-14T10:37:01.788109+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T14:52:18.060480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.060360+0000","last_change":"2023-10-14T10:37:00.076568+0000","last_active":"2023-10-14T10:37:01.060360+0000","last_peered":"2023-10-14T10:37:01.060360+0000","last_clean":"2023-10-14T10:37:01.060360+0000","last_became_active":"2023-10-14T10:37:00.076263+0000","last_became_peered":"2023-10-14T10:37:00.076263+0000","last_unstale":"2023-10-14T10:37:01.060360+0000","last_undegraded":"2023-10-14T10:37:01.060360+0000","last_fullsized":"2023-10-14T10:37:01.060360+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:18:45.297555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.061994+0000","last_change":"2023-10-14T10:37:00.079439+0000","last_active":"2023-10-14T10:37:01.061994+0000","last_peered":"2023-10-14T10:37:01.061994+0000","last_clean":"2023-10-14T10:37:01.061994+0000","last_became_active":"2023-10-14T10:37:00.079178+0000","last_became_peered":"2023-10-14T10:37:00.079178+0000","last_unstale":"2023-10-14T10:37:01.061994+0000","last_undegraded":"2023-10-14T10:37:01.061994+0000","last_fullsized":"2023-10-14T10:37:01.061994+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:17:54.730152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.489206+0000","last_change":"2023-10-14T10:37:00.080675+0000","last_active":"2023-10-14T10:37:01.489206+0000","last_peered":"2023-10-14T10:37:01.489206+0000","last_clean":"2023-10-14T10:37:01.489206+0000","last_became_active":"2023-10-14T10:37:00.080171+0000","last_became_peered":"2023-10-14T10:37:00.080171+0000","last_unstale":"2023-10-14T10:37:01.489206+0000","last_undegraded":"2023-10-14T10:37:01.489206+0000","last_fullsized":"2023-10-14T10:37:01.489206+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:28:04.536948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.061814+0000","last_change":"2023-10-14T10:37:00.072155+0000","last_active":"2023-10-14T10:37:01.061814+0000","last_peered":"2023-10-14T10:37:01.061814+0000","last_clean":"2023-10-14T10:37:01.061814+0000","last_became_active":"2023-10-14T10:37:00.071882+0000","last_became_peered":"2023-10-14T10:37:00.071882+0000","last_unstale":"2023-10-14T10:37:01.061814+0000","last_undegraded":"2023-10-14T10:37:01.061814+0000","last_fullsized":"2023-10-14T10:37:01.061814+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T10:48:03.043383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.787458+0000","last_change":"2023-10-14T10:37:00.081816+0000","last_active":"2023-10-14T10:37:01.787458+0000","last_peered":"2023-10-14T10:37:01.787458+0000","last_clean":"2023-10-14T10:37:01.787458+0000","last_became_active":"2023-10-14T10:37:00.081312+0000","last_became_peered":"2023-10-14T10:37:00.081312+0000","last_unstale":"2023-10-14T10:37:01.787458+0000","last_undegraded":"2023-10-14T10:37:01.787458+0000","last_fullsized":"2023-10-14T10:37:01.787458+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:58:56.558710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.061373+0000","last_change":"2023-10-14T10:37:00.078914+0000","last_active":"2023-10-14T10:37:01.061373+0000","last_peered":"2023-10-14T10:37:01.061373+0000","last_clean":"2023-10-14T10:37:01.061373+0000","last_became_active":"2023-10-14T10:37:00.078441+0000","last_became_peered":"2023-10-14T10:37:00.078441+0000","last_unstale":"2023-10-14T10:37:01.061373+0000","last_undegraded":"2023-10-14T10:37:01.061373+0000","last_fullsized":"2023-10-14T10:37:01.061373+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:22:32.154663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.787754+0000","last_change":"2023-10-14T10:37:00.071246+0000","last_active":"2023-10-14T10:37:01.787754+0000","last_peered":"2023-10-14T10:37:01.787754+0000","last_clean":"2023-10-14T10:37:01.787754+0000","last_became_active":"2023-10-14T10:37:00.070945+0000","last_became_peered":"2023-10-14T10:37:00.070945+0000","last_unstale":"2023-10-14T10:37:01.787754+0000","last_undegraded":"2023-10-14T10:37:01.787754+0000","last_fullsized":"2023-10-14T10:37:01.787754+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:30:57.784792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.787645+0000","last_change":"2023-10-14T10:36:59.066772+0000","last_active":"2023-10-14T10:37:01.787645+0000","last_peered":"2023-10-14T10:37:01.787645+0000","last_clean":"2023-10-14T10:37:01.787645+0000","last_became_active":"2023-10-14T10:36:59.066488+0000","last_became_peered":"2023-10-14T10:36:59.066488+0000","last_unstale":"2023-10-14T10:37:01.787645+0000","last_undegraded":"2023-10-14T10:37:01.787645+0000","last_fullsized":"2023-10-14T10:37:01.787645+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:39:01.994329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.787569+0000","last_change":"2023-10-14T10:37:00.077997+0000","last_active":"2023-10-14T10:37:01.787569+0000","last_peered":"2023-10-14T10:37:01.787569+0000","last_clean":"2023-10-14T10:37:01.787569+0000","last_became_active":"2023-10-14T10:37:00.077734+0000","last_became_peered":"2023-10-14T10:37:00.077734+0000","last_unstale":"2023-10-14T10:37:01.787569+0000","last_undegraded":"2023-10-14T10:37:01.787569+0000","last_fullsized":"2023-10-14T10:37:01.787569+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:28:45.821377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.787856+0000","last_change":"2023-10-14T10:36:59.070768+0000","last_active":"2023-10-14T10:37:01.787856+0000","last_peered":"2023-10-14T10:37:01.787856+0000","last_clean":"2023-10-14T10:37:01.787856+0000","last_became_active":"2023-10-14T10:36:59.070463+0000","last_became_peered":"2023-10-14T10:36:59.070463+0000","last_unstale":"2023-10-14T10:37:01.787856+0000","last_undegraded":"2023-10-14T10:37:01.787856+0000","last_fullsized":"2023-10-14T10:37:01.787856+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:08:05.335325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.489357+0000","last_change":"2023-10-14T10:37:00.080768+0000","last_active":"2023-10-14T10:37:01.489357+0000","last_peered":"2023-10-14T10:37:01.489357+0000","last_clean":"2023-10-14T10:37:01.489357+0000","last_became_active":"2023-10-14T10:37:00.080160+0000","last_became_peered":"2023-10-14T10:37:00.080160+0000","last_unstale":"2023-10-14T10:37:01.489357+0000","last_undegraded":"2023-10-14T10:37:01.489357+0000","last_fullsized":"2023-10-14T10:37:01.489357+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T20:27:28.798883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788006+0000","last_change":"2023-10-14T10:36:59.067900+0000","last_active":"2023-10-14T10:37:01.788006+0000","last_peered":"2023-10-14T10:37:01.788006+0000","last_clean":"2023-10-14T10:37:01.788006+0000","last_became_active":"2023-10-14T10:36:59.067616+0000","last_became_peered":"2023-10-14T10:36:59.067616+0000","last_unstale":"2023-10-14T10:37:01.788006+0000","last_undegraded":"2023-10-14T10:37:01.788006+0000","last_fullsized":"2023-10-14T10:37:01.788006+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:28:44.815010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.060586+0000","last_change":"2023-10-14T10:37:00.079200+0000","last_active":"2023-10-14T10:37:01.060586+0000","last_peered":"2023-10-14T10:37:01.060586+0000","last_clean":"2023-10-14T10:37:01.060586+0000","last_became_active":"2023-10-14T10:37:00.078829+0000","last_became_peered":"2023-10-14T10:37:00.078829+0000","last_unstale":"2023-10-14T10:37:01.060586+0000","last_undegraded":"2023-10-14T10:37:01.060586+0000","last_fullsized":"2023-10-14T10:37:01.060586+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:31:16.127752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788507+0000","last_change":"2023-10-14T10:36:59.065882+0000","last_active":"2023-10-14T10:37:01.788507+0000","last_peered":"2023-10-14T10:37:01.788507+0000","last_clean":"2023-10-14T10:37:01.788507+0000","last_became_active":"2023-10-14T10:36:59.065608+0000","last_became_peered":"2023-10-14T10:36:59.065608+0000","last_unstale":"2023-10-14T10:37:01.788507+0000","last_undegraded":"2023-10-14T10:37:01.788507+0000","last_fullsized":"2023-10-14T10:37:01.788507+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T21:18:46.547776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.062326+0000","last_change":"2023-10-14T10:37:00.077115+0000","last_active":"2023-10-14T10:37:01.062326+0000","last_peered":"2023-10-14T10:37:01.062326+0000","last_clean":"2023-10-14T10:37:01.062326+0000","last_became_active":"2023-10-14T10:37:00.076840+0000","last_became_peered":"2023-10-14T10:37:00.076840+0000","last_unstale":"2023-10-14T10:37:01.062326+0000","last_undegraded":"2023-10-14T10:37:01.062326+0000","last_fullsized":"2023-10-14T10:37:01.062326+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T12:47:24.725885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.489315+0000","last_change":"2023-10-14T10:36:59.069533+0000","last_active":"2023-10-14T10:37:01.489315+0000","last_peered":"2023-10-14T10:37:01.489315+0000","last_clean":"2023-10-14T10:37:01.489315+0000","last_became_active":"2023-10-14T10:36:59.069192+0000","last_became_peered":"2023-10-14T10:36:59.069192+0000","last_unstale":"2023-10-14T10:37:01.489315+0000","last_undegraded":"2023-10-14T10:37:01.489315+0000","last_fullsized":"2023-10-14T10:37:01.489315+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T21:39:34.581362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.061634+0000","last_change":"2023-10-14T10:37:00.068786+0000","last_active":"2023-10-14T10:37:01.061634+0000","last_peered":"2023-10-14T10:37:01.061634+0000","last_clean":"2023-10-14T10:37:01.061634+0000","last_became_active":"2023-10-14T10:37:00.068517+0000","last_became_peered":"2023-10-14T10:37:00.068517+0000","last_unstale":"2023-10-14T10:37:01.061634+0000","last_undegraded":"2023-10-14T10:37:01.061634+0000","last_fullsized":"2023-10-14T10:37:01.061634+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:22:32.154663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.489303+0000","last_change":"2023-10-14T10:36:59.069445+0000","last_active":"2023-10-14T10:37:01.489303+0000","last_peered":"2023-10-14T10:37:01.489303+0000","last_clean":"2023-10-14T10:37:01.489303+0000","last_became_active":"2023-10-14T10:36:59.069151+0000","last_became_peered":"2023-10-14T10:36:59.069151+0000","last_unstale":"2023-10-14T10:37:01.489303+0000","last_undegraded":"2023-10-14T10:37:01.489303+0000","last_fullsized":"2023-10-14T10:37:01.489303+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:54:15.011628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.488953+0000","last_change":"2023-10-14T10:37:00.076984+0000","last_active":"2023-10-14T10:37:01.488953+0000","last_peered":"2023-10-14T10:37:01.488953+0000","last_clean":"2023-10-14T10:37:01.488953+0000","last_became_active":"2023-10-14T10:37:00.076706+0000","last_became_peered":"2023-10-14T10:37:00.076706+0000","last_unstale":"2023-10-14T10:37:01.488953+0000","last_undegraded":"2023-10-14T10:37:01.488953+0000","last_fullsized":"2023-10-14T10:37:01.488953+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:30:57.784792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.787709+0000","last_change":"2023-10-14T10:36:59.062324+0000","last_active":"2023-10-14T10:37:01.787709+0000","last_peered":"2023-10-14T10:37:01.787709+0000","last_clean":"2023-10-14T10:37:01.787709+0000","last_became_active":"2023-10-14T10:36:59.062075+0000","last_became_peered":"2023-10-14T10:36:59.062075+0000","last_unstale":"2023-10-14T10:37:01.787709+0000","last_undegraded":"2023-10-14T10:37:01.787709+0000","last_fullsized":"2023-10-14T10:37:01.787709+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:45:50.919718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.059848+0000","last_change":"2023-10-14T10:37:00.067486+0000","last_active":"2023-10-14T10:37:01.059848+0000","last_peered":"2023-10-14T10:37:01.059848+0000","last_clean":"2023-10-14T10:37:01.059848+0000","last_became_active":"2023-10-14T10:37:00.067187+0000","last_became_peered":"2023-10-14T10:37:00.067187+0000","last_unstale":"2023-10-14T10:37:01.059848+0000","last_undegraded":"2023-10-14T10:37:01.059848+0000","last_fullsized":"2023-10-14T10:37:01.059848+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:10:46.473503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.488998+0000","last_change":"2023-10-14T10:36:59.066330+0000","last_active":"2023-10-14T10:37:01.488998+0000","last_peered":"2023-10-14T10:37:01.488998+0000","last_clean":"2023-10-14T10:37:01.488998+0000","last_became_active":"2023-10-14T10:36:59.064990+0000","last_became_peered":"2023-10-14T10:36:59.064990+0000","last_unstale":"2023-10-14T10:37:01.488998+0000","last_undegraded":"2023-10-14T10:37:01.488998+0000","last_fullsized":"2023-10-14T10:37:01.488998+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T12:15:16.823069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.061974+0000","last_change":"2023-10-14T10:37:00.077465+0000","last_active":"2023-10-14T10:37:01.061974+0000","last_peered":"2023-10-14T10:37:01.061974+0000","last_clean":"2023-10-14T10:37:01.061974+0000","last_became_active":"2023-10-14T10:37:00.077152+0000","last_became_peered":"2023-10-14T10:37:00.077152+0000","last_unstale":"2023-10-14T10:37:01.061974+0000","last_undegraded":"2023-10-14T10:37:01.061974+0000","last_fullsized":"2023-10-14T10:37:01.061974+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:51:00.167498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788575+0000","last_change":"2023-10-14T10:36:59.061903+0000","last_active":"2023-10-14T10:37:01.788575+0000","last_peered":"2023-10-14T10:37:01.788575+0000","last_clean":"2023-10-14T10:37:01.788575+0000","last_became_active":"2023-10-14T10:36:59.061597+0000","last_became_peered":"2023-10-14T10:36:59.061597+0000","last_unstale":"2023-10-14T10:37:01.788575+0000","last_undegraded":"2023-10-14T10:37:01.788575+0000","last_fullsized":"2023-10-14T10:37:01.788575+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:22:31.148297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788727+0000","last_change":"2023-10-14T10:37:00.076213+0000","last_active":"2023-10-14T10:37:01.788727+0000","last_peered":"2023-10-14T10:37:01.788727+0000","last_clean":"2023-10-14T10:37:01.788727+0000","last_became_active":"2023-10-14T10:37:00.075889+0000","last_became_peered":"2023-10-14T10:37:00.075889+0000","last_unstale":"2023-10-14T10:37:01.788727+0000","last_undegraded":"2023-10-14T10:37:01.788727+0000","last_fullsized":"2023-10-14T10:37:01.788727+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T14:19:22.310866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.787968+0000","last_change":"2023-10-14T10:36:59.070183+0000","last_active":"2023-10-14T10:37:01.787968+0000","last_peered":"2023-10-14T10:37:01.787968+0000","last_clean":"2023-10-14T10:37:01.787968+0000","last_became_active":"2023-10-14T10:36:59.069731+0000","last_became_peered":"2023-10-14T10:36:59.069731+0000","last_unstale":"2023-10-14T10:37:01.787968+0000","last_undegraded":"2023-10-14T10:37:01.787968+0000","last_fullsized":"2023-10-14T10:37:01.787968+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:58:55.552343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788210+0000","last_change":"2023-10-14T10:37:00.072592+0000","last_active":"2023-10-14T10:37:01.788210+0000","last_peered":"2023-10-14T10:37:01.788210+0000","last_clean":"2023-10-14T10:37:01.788210+0000","last_became_active":"2023-10-14T10:37:00.072288+0000","last_became_peered":"2023-10-14T10:37:00.072288+0000","last_unstale":"2023-10-14T10:37:01.788210+0000","last_undegraded":"2023-10-14T10:37:01.788210+0000","last_fullsized":"2023-10-14T10:37:01.788210+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T17:27:50.406084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.062406+0000","last_change":"2023-10-14T10:36:59.062120+0000","last_active":"2023-10-14T10:37:01.062406+0000","last_peered":"2023-10-14T10:37:01.062406+0000","last_clean":"2023-10-14T10:37:01.062406+0000","last_became_active":"2023-10-14T10:36:59.061700+0000","last_became_peered":"2023-10-14T10:36:59.061700+0000","last_unstale":"2023-10-14T10:37:01.062406+0000","last_undegraded":"2023-10-14T10:37:01.062406+0000","last_fullsized":"2023-10-14T10:37:01.062406+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:45:24.296819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.062364+0000","last_change":"2023-10-14T10:37:00.074977+0000","last_active":"2023-10-14T10:37:01.062364+0000","last_peered":"2023-10-14T10:37:01.062364+0000","last_clean":"2023-10-14T10:37:01.062364+0000","last_became_active":"2023-10-14T10:37:00.074700+0000","last_became_peered":"2023-10-14T10:37:00.074700+0000","last_unstale":"2023-10-14T10:37:01.062364+0000","last_undegraded":"2023-10-14T10:37:01.062364+0000","last_fullsized":"2023-10-14T10:37:01.062364+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T18:57:50.136605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788323+0000","last_change":"2023-10-14T10:36:59.069400+0000","last_active":"2023-10-14T10:37:01.788323+0000","last_peered":"2023-10-14T10:37:01.788323+0000","last_clean":"2023-10-14T10:37:01.788323+0000","last_became_active":"2023-10-14T10:36:59.069043+0000","last_became_peered":"2023-10-14T10:36:59.069043+0000","last_unstale":"2023-10-14T10:37:01.788323+0000","last_undegraded":"2023-10-14T10:37:01.788323+0000","last_fullsized":"2023-10-14T10:37:01.788323+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:10:45.467136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.060040+0000","last_change":"2023-10-14T10:37:00.068613+0000","last_active":"2023-10-14T10:37:01.060040+0000","last_peered":"2023-10-14T10:37:01.060040+0000","last_clean":"2023-10-14T10:37:01.060040+0000","last_became_active":"2023-10-14T10:37:00.065345+0000","last_became_peered":"2023-10-14T10:37:00.065345+0000","last_unstale":"2023-10-14T10:37:01.060040+0000","last_undegraded":"2023-10-14T10:37:01.060040+0000","last_fullsized":"2023-10-14T10:37:01.060040+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:45:25.303185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.489387+0000","last_change":"2023-10-14T10:36:59.065846+0000","last_active":"2023-10-14T10:37:01.489387+0000","last_peered":"2023-10-14T10:37:01.489387+0000","last_clean":"2023-10-14T10:37:01.489387+0000","last_became_active":"2023-10-14T10:36:59.065569+0000","last_became_peered":"2023-10-14T10:36:59.065569+0000","last_unstale":"2023-10-14T10:37:01.489387+0000","last_undegraded":"2023-10-14T10:37:01.489387+0000","last_fullsized":"2023-10-14T10:37:01.489387+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:28:44.815010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788692+0000","last_change":"2023-10-14T10:37:00.079137+0000","last_active":"2023-10-14T10:37:01.788692+0000","last_peered":"2023-10-14T10:37:01.788692+0000","last_clean":"2023-10-14T10:37:01.788692+0000","last_became_active":"2023-10-14T10:37:00.078844+0000","last_became_peered":"2023-10-14T10:37:00.078844+0000","last_unstale":"2023-10-14T10:37:01.788692+0000","last_undegraded":"2023-10-14T10:37:01.788692+0000","last_fullsized":"2023-10-14T10:37:01.788692+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:30:51.914014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.059920+0000","last_change":"2023-10-14T10:36:59.067245+0000","last_active":"2023-10-14T10:37:01.059920+0000","last_peered":"2023-10-14T10:37:01.059920+0000","last_clean":"2023-10-14T10:37:01.059920+0000","last_became_active":"2023-10-14T10:36:59.066952+0000","last_became_peered":"2023-10-14T10:36:59.066952+0000","last_unstale":"2023-10-14T10:37:01.059920+0000","last_undegraded":"2023-10-14T10:37:01.059920+0000","last_fullsized":"2023-10-14T10:37:01.059920+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T12:08:55.620308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.062454+0000","last_change":"2023-10-14T10:37:00.078646+0000","last_active":"2023-10-14T10:37:01.062454+0000","last_peered":"2023-10-14T10:37:01.062454+0000","last_clean":"2023-10-14T10:37:01.062454+0000","last_became_active":"2023-10-14T10:37:00.078353+0000","last_became_peered":"2023-10-14T10:37:00.078353+0000","last_unstale":"2023-10-14T10:37:01.062454+0000","last_undegraded":"2023-10-14T10:37:01.062454+0000","last_fullsized":"2023-10-14T10:37:01.062454+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:30:51.914014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788938+0000","last_change":"2023-10-14T10:36:59.063118+0000","last_active":"2023-10-14T10:37:01.788938+0000","last_peered":"2023-10-14T10:37:01.788938+0000","last_clean":"2023-10-14T10:37:01.788938+0000","last_became_active":"2023-10-14T10:36:59.062380+0000","last_became_peered":"2023-10-14T10:36:59.062380+0000","last_unstale":"2023-10-14T10:37:01.788938+0000","last_undegraded":"2023-10-14T10:37:01.788938+0000","last_fullsized":"2023-10-14T10:37:01.788938+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T21:39:34.581362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.787507+0000","last_change":"2023-10-14T10:37:00.078077+0000","last_active":"2023-10-14T10:37:01.787507+0000","last_peered":"2023-10-14T10:37:01.787507+0000","last_clean":"2023-10-14T10:37:01.787507+0000","last_became_active":"2023-10-14T10:37:00.077806+0000","last_became_peered":"2023-10-14T10:37:00.077806+0000","last_unstale":"2023-10-14T10:37:01.787507+0000","last_undegraded":"2023-10-14T10:37:01.787507+0000","last_fullsized":"2023-10-14T10:37:01.787507+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:45:51.926085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788260+0000","last_change":"2023-10-14T10:36:59.068012+0000","last_active":"2023-10-14T10:37:01.788260+0000","last_peered":"2023-10-14T10:37:01.788260+0000","last_clean":"2023-10-14T10:37:01.788260+0000","last_became_active":"2023-10-14T10:36:59.067727+0000","last_became_peered":"2023-10-14T10:36:59.067727+0000","last_unstale":"2023-10-14T10:37:01.788260+0000","last_undegraded":"2023-10-14T10:37:01.788260+0000","last_fullsized":"2023-10-14T10:37:01.788260+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:44:49.671921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.489522+0000","last_change":"2023-10-14T10:37:00.076926+0000","last_active":"2023-10-14T10:37:01.489522+0000","last_peered":"2023-10-14T10:37:01.489522+0000","last_clean":"2023-10-14T10:37:01.489522+0000","last_became_active":"2023-10-14T10:37:00.076658+0000","last_became_peered":"2023-10-14T10:37:00.076658+0000","last_unstale":"2023-10-14T10:37:01.489522+0000","last_undegraded":"2023-10-14T10:37:01.489522+0000","last_fullsized":"2023-10-14T10:37:01.489522+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:52:37.894720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.060490+0000","last_change":"2023-10-14T10:36:59.071714+0000","last_active":"2023-10-14T10:37:01.060490+0000","last_peered":"2023-10-14T10:37:01.060490+0000","last_clean":"2023-10-14T10:37:01.060490+0000","last_became_active":"2023-10-14T10:36:59.071406+0000","last_became_peered":"2023-10-14T10:36:59.071406+0000","last_unstale":"2023-10-14T10:37:01.060490+0000","last_undegraded":"2023-10-14T10:37:01.060490+0000","last_fullsized":"2023-10-14T10:37:01.060490+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:59:06.202215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.062345+0000","last_change":"2023-10-14T10:37:00.080060+0000","last_active":"2023-10-14T10:37:01.062345+0000","last_peered":"2023-10-14T10:37:01.062345+0000","last_clean":"2023-10-14T10:37:01.062345+0000","last_became_active":"2023-10-14T10:37:00.079795+0000","last_became_peered":"2023-10-14T10:37:00.079795+0000","last_unstale":"2023-10-14T10:37:01.062345+0000","last_undegraded":"2023-10-14T10:37:01.062345+0000","last_fullsized":"2023-10-14T10:37:01.062345+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T17:27:50.406084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.489554+0000","last_change":"2023-10-14T10:36:59.065271+0000","last_active":"2023-10-14T10:37:01.489554+0000","last_peered":"2023-10-14T10:37:01.489554+0000","last_clean":"2023-10-14T10:37:01.489554+0000","last_became_active":"2023-10-14T10:36:59.064959+0000","last_became_peered":"2023-10-14T10:36:59.064959+0000","last_unstale":"2023-10-14T10:37:01.489554+0000","last_undegraded":"2023-10-14T10:37:01.489554+0000","last_fullsized":"2023-10-14T10:37:01.489554+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T21:56:55.137838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.060192+0000","last_change":"2023-10-14T10:37:00.077751+0000","last_active":"2023-10-14T10:37:01.060192+0000","last_peered":"2023-10-14T10:37:01.060192+0000","last_clean":"2023-10-14T10:37:01.060192+0000","last_became_active":"2023-10-14T10:37:00.077459+0000","last_became_peered":"2023-10-14T10:37:00.077459+0000","last_unstale":"2023-10-14T10:37:01.060192+0000","last_undegraded":"2023-10-14T10:37:01.060192+0000","last_fullsized":"2023-10-14T10:37:01.060192+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T20:27:28.798883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.489473+0000","last_change":"2023-10-14T10:36:59.058893+0000","last_active":"2023-10-14T10:37:01.489473+0000","last_peered":"2023-10-14T10:37:01.489473+0000","last_clean":"2023-10-14T10:37:01.489473+0000","last_became_active":"2023-10-14T10:36:59.058558+0000","last_became_peered":"2023-10-14T10:36:59.058558+0000","last_unstale":"2023-10-14T10:37:01.489473+0000","last_undegraded":"2023-10-14T10:37:01.489473+0000","last_fullsized":"2023-10-14T10:37:01.489473+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:45:50.919718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788693+0000","last_change":"2023-10-14T10:37:00.071398+0000","last_active":"2023-10-14T10:37:01.788693+0000","last_peered":"2023-10-14T10:37:01.788693+0000","last_clean":"2023-10-14T10:37:01.788693+0000","last_became_active":"2023-10-14T10:37:00.071167+0000","last_became_peered":"2023-10-14T10:37:00.071167+0000","last_unstale":"2023-10-14T10:37:01.788693+0000","last_undegraded":"2023-10-14T10:37:01.788693+0000","last_fullsized":"2023-10-14T10:37:01.788693+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:31:12.123956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.489473+0000","last_change":"2023-10-14T10:36:59.069420+0000","last_active":"2023-10-14T10:37:01.489473+0000","last_peered":"2023-10-14T10:37:01.489473+0000","last_clean":"2023-10-14T10:37:01.489473+0000","last_became_active":"2023-10-14T10:36:59.069110+0000","last_became_peered":"2023-10-14T10:36:59.069110+0000","last_unstale":"2023-10-14T10:37:01.489473+0000","last_undegraded":"2023-10-14T10:37:01.489473+0000","last_fullsized":"2023-10-14T10:37:01.489473+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:09:59.812757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788586+0000","last_change":"2023-10-14T10:37:00.077550+0000","last_active":"2023-10-14T10:37:01.788586+0000","last_peered":"2023-10-14T10:37:01.788586+0000","last_clean":"2023-10-14T10:37:01.788586+0000","last_became_active":"2023-10-14T10:37:00.077175+0000","last_became_peered":"2023-10-14T10:37:00.077175+0000","last_unstale":"2023-10-14T10:37:01.788586+0000","last_undegraded":"2023-10-14T10:37:01.788586+0000","last_fullsized":"2023-10-14T10:37:01.788586+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T14:51:19.968923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.787861+0000","last_change":"2023-10-14T10:36:59.067621+0000","last_active":"2023-10-14T10:37:01.787861+0000","last_peered":"2023-10-14T10:37:01.787861+0000","last_clean":"2023-10-14T10:37:01.787861+0000","last_became_active":"2023-10-14T10:36:59.067345+0000","last_became_peered":"2023-10-14T10:36:59.067345+0000","last_unstale":"2023-10-14T10:37:01.787861+0000","last_undegraded":"2023-10-14T10:37:01.787861+0000","last_fullsized":"2023-10-14T10:37:01.787861+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:54:15.011628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788501+0000","last_change":"2023-10-14T10:37:00.077744+0000","last_active":"2023-10-14T10:37:01.788501+0000","last_peered":"2023-10-14T10:37:01.788501+0000","last_clean":"2023-10-14T10:37:01.788501+0000","last_became_active":"2023-10-14T10:37:00.077465+0000","last_became_peered":"2023-10-14T10:37:00.077465+0000","last_unstale":"2023-10-14T10:37:01.788501+0000","last_undegraded":"2023-10-14T10:37:01.788501+0000","last_fullsized":"2023-10-14T10:37:01.788501+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T20:36:23.971821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.787432+0000","last_change":"2023-10-14T10:36:59.069707+0000","last_active":"2023-10-14T10:37:01.787432+0000","last_peered":"2023-10-14T10:37:01.787432+0000","last_clean":"2023-10-14T10:37:01.787432+0000","last_became_active":"2023-10-14T10:36:59.069269+0000","last_became_peered":"2023-10-14T10:36:59.069269+0000","last_unstale":"2023-10-14T10:37:01.787432+0000","last_undegraded":"2023-10-14T10:37:01.787432+0000","last_fullsized":"2023-10-14T10:37:01.787432+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T12:15:16.823069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788221+0000","last_change":"2023-10-14T10:37:00.068887+0000","last_active":"2023-10-14T10:37:01.788221+0000","last_peered":"2023-10-14T10:37:01.788221+0000","last_clean":"2023-10-14T10:37:01.788221+0000","last_became_active":"2023-10-14T10:37:00.068620+0000","last_became_peered":"2023-10-14T10:37:00.068620+0000","last_unstale":"2023-10-14T10:37:01.788221+0000","last_undegraded":"2023-10-14T10:37:01.788221+0000","last_fullsized":"2023-10-14T10:37:01.788221+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T18:04:59.338050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.489058+0000","last_change":"2023-10-14T10:36:59.061947+0000","last_active":"2023-10-14T10:37:01.489058+0000","last_peered":"2023-10-14T10:37:01.489058+0000","last_clean":"2023-10-14T10:37:01.489058+0000","last_became_active":"2023-10-14T10:36:59.061678+0000","last_became_peered":"2023-10-14T10:36:59.061678+0000","last_unstale":"2023-10-14T10:37:01.489058+0000","last_undegraded":"2023-10-14T10:37:01.489058+0000","last_fullsized":"2023-10-14T10:37:01.489058+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:39:01.994329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.061998+0000","last_change":"2023-10-14T10:37:00.073204+0000","last_active":"2023-10-14T10:37:01.061998+0000","last_peered":"2023-10-14T10:37:01.061998+0000","last_clean":"2023-10-14T10:37:01.061998+0000","last_became_active":"2023-10-14T10:37:00.071692+0000","last_became_peered":"2023-10-14T10:37:00.071692+0000","last_unstale":"2023-10-14T10:37:01.061998+0000","last_undegraded":"2023-10-14T10:37:01.061998+0000","last_fullsized":"2023-10-14T10:37:01.061998+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:17:54.730152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.061467+0000","last_change":"2023-10-14T10:36:59.063545+0000","last_active":"2023-10-14T10:37:01.061467+0000","last_peered":"2023-10-14T10:37:01.061467+0000","last_clean":"2023-10-14T10:37:01.061467+0000","last_became_active":"2023-10-14T10:36:59.063332+0000","last_became_peered":"2023-10-14T10:36:59.063332+0000","last_unstale":"2023-10-14T10:37:01.061467+0000","last_undegraded":"2023-10-14T10:37:01.061467+0000","last_fullsized":"2023-10-14T10:37:01.061467+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:10:45.467136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788558+0000","last_change":"2023-10-14T10:37:00.075249+0000","last_active":"2023-10-14T10:37:01.788558+0000","last_peered":"2023-10-14T10:37:01.788558+0000","last_clean":"2023-10-14T10:37:01.788558+0000","last_became_active":"2023-10-14T10:37:00.074859+0000","last_became_peered":"2023-10-14T10:37:00.074859+0000","last_unstale":"2023-10-14T10:37:01.788558+0000","last_undegraded":"2023-10-14T10:37:01.788558+0000","last_fullsized":"2023-10-14T10:37:01.788558+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:04:51.088268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.060369+0000","last_change":"2023-10-14T10:36:59.069418+0000","last_active":"2023-10-14T10:37:01.060369+0000","last_peered":"2023-10-14T10:37:01.060369+0000","last_clean":"2023-10-14T10:37:01.060369+0000","last_became_active":"2023-10-14T10:36:59.069037+0000","last_became_peered":"2023-10-14T10:36:59.069037+0000","last_unstale":"2023-10-14T10:37:01.060369+0000","last_undegraded":"2023-10-14T10:37:01.060369+0000","last_fullsized":"2023-10-14T10:37:01.060369+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:45:50.919718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.062253+0000","last_change":"2023-10-14T10:37:00.075731+0000","last_active":"2023-10-14T10:37:01.062253+0000","last_peered":"2023-10-14T10:37:01.062253+0000","last_clean":"2023-10-14T10:37:01.062253+0000","last_became_active":"2023-10-14T10:37:00.075491+0000","last_became_peered":"2023-10-14T10:37:00.075491+0000","last_unstale":"2023-10-14T10:37:01.062253+0000","last_undegraded":"2023-10-14T10:37:01.062253+0000","last_fullsized":"2023-10-14T10:37:01.062253+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T14:52:18.060480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788031+0000","last_change":"2023-10-14T10:36:59.066461+0000","last_active":"2023-10-14T10:37:01.788031+0000","last_peered":"2023-10-14T10:37:01.788031+0000","last_clean":"2023-10-14T10:37:01.788031+0000","last_became_active":"2023-10-14T10:36:59.066146+0000","last_became_peered":"2023-10-14T10:36:59.066146+0000","last_unstale":"2023-10-14T10:37:01.788031+0000","last_undegraded":"2023-10-14T10:37:01.788031+0000","last_fullsized":"2023-10-14T10:37:01.788031+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:45:50.919718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.787608+0000","last_change":"2023-10-14T10:37:00.082851+0000","last_active":"2023-10-14T10:37:01.787608+0000","last_peered":"2023-10-14T10:37:01.787608+0000","last_clean":"2023-10-14T10:37:01.787608+0000","last_became_active":"2023-10-14T10:37:00.082570+0000","last_became_peered":"2023-10-14T10:37:00.082570+0000","last_unstale":"2023-10-14T10:37:01.787608+0000","last_undegraded":"2023-10-14T10:37:01.787608+0000","last_fullsized":"2023-10-14T10:37:01.787608+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:28:45.821377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.059981+0000","last_change":"2023-10-14T10:36:59.061936+0000","last_active":"2023-10-14T10:37:01.059981+0000","last_peered":"2023-10-14T10:37:01.059981+0000","last_clean":"2023-10-14T10:37:01.059981+0000","last_became_active":"2023-10-14T10:36:59.061642+0000","last_became_peered":"2023-10-14T10:36:59.061642+0000","last_unstale":"2023-10-14T10:37:01.059981+0000","last_undegraded":"2023-10-14T10:37:01.059981+0000","last_fullsized":"2023-10-14T10:37:01.059981+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:54:15.011628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.787449+0000","last_change":"2023-10-14T10:37:00.071551+0000","last_active":"2023-10-14T10:37:01.787449+0000","last_peered":"2023-10-14T10:37:01.787449+0000","last_clean":"2023-10-14T10:37:01.787449+0000","last_became_active":"2023-10-14T10:37:00.071244+0000","last_became_peered":"2023-10-14T10:37:00.071244+0000","last_unstale":"2023-10-14T10:37:01.787449+0000","last_undegraded":"2023-10-14T10:37:01.787449+0000","last_fullsized":"2023-10-14T10:37:01.787449+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:08:06.341692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.787778+0000","last_change":"2023-10-14T10:36:59.066839+0000","last_active":"2023-10-14T10:37:01.787778+0000","last_peered":"2023-10-14T10:37:01.787778+0000","last_clean":"2023-10-14T10:37:01.787778+0000","last_became_active":"2023-10-14T10:36:59.066592+0000","last_became_peered":"2023-10-14T10:36:59.066592+0000","last_unstale":"2023-10-14T10:37:01.787778+0000","last_undegraded":"2023-10-14T10:37:01.787778+0000","last_fullsized":"2023-10-14T10:37:01.787778+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T12:15:16.823069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788013+0000","last_change":"2023-10-14T10:37:00.076076+0000","last_active":"2023-10-14T10:37:01.788013+0000","last_peered":"2023-10-14T10:37:01.788013+0000","last_clean":"2023-10-14T10:37:01.788013+0000","last_became_active":"2023-10-14T10:37:00.075794+0000","last_became_peered":"2023-10-14T10:37:00.075794+0000","last_unstale":"2023-10-14T10:37:01.788013+0000","last_undegraded":"2023-10-14T10:37:01.788013+0000","last_fullsized":"2023-10-14T10:37:01.788013+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:44:50.678288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.060021+0000","last_change":"2023-10-14T10:36:59.065626+0000","last_active":"2023-10-14T10:37:01.060021+0000","last_peered":"2023-10-14T10:37:01.060021+0000","last_clean":"2023-10-14T10:37:01.060021+0000","last_became_active":"2023-10-14T10:36:59.065163+0000","last_became_peered":"2023-10-14T10:36:59.065163+0000","last_unstale":"2023-10-14T10:37:01.060021+0000","last_undegraded":"2023-10-14T10:37:01.060021+0000","last_fullsized":"2023-10-14T10:37:01.060021+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:08:05.335325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788427+0000","last_change":"2023-10-14T10:37:00.078024+0000","last_active":"2023-10-14T10:37:01.788427+0000","last_peered":"2023-10-14T10:37:01.788427+0000","last_clean":"2023-10-14T10:37:01.788427+0000","last_became_active":"2023-10-14T10:37:00.077621+0000","last_became_peered":"2023-10-14T10:37:00.077621+0000","last_unstale":"2023-10-14T10:37:01.788427+0000","last_undegraded":"2023-10-14T10:37:01.788427+0000","last_fullsized":"2023-10-14T10:37:01.788427+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:06:28.391046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.489094+0000","last_change":"2023-10-14T10:36:59.064632+0000","last_active":"2023-10-14T10:37:01.489094+0000","last_peered":"2023-10-14T10:37:01.489094+0000","last_clean":"2023-10-14T10:37:01.489094+0000","last_became_active":"2023-10-14T10:36:59.064336+0000","last_became_peered":"2023-10-14T10:36:59.064336+0000","last_unstale":"2023-10-14T10:37:01.489094+0000","last_undegraded":"2023-10-14T10:37:01.489094+0000","last_fullsized":"2023-10-14T10:37:01.489094+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:08:05.335325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.488971+0000","last_change":"2023-10-14T10:37:00.081042+0000","last_active":"2023-10-14T10:37:01.488971+0000","last_peered":"2023-10-14T10:37:01.488971+0000","last_clean":"2023-10-14T10:37:01.488971+0000","last_became_active":"2023-10-14T10:37:00.080340+0000","last_became_peered":"2023-10-14T10:37:00.080340+0000","last_unstale":"2023-10-14T10:37:01.488971+0000","last_undegraded":"2023-10-14T10:37:01.488971+0000","last_fullsized":"2023-10-14T10:37:01.488971+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:44:50.678288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788029+0000","last_change":"2023-10-14T10:36:59.063568+0000","last_active":"2023-10-14T10:37:01.788029+0000","last_peered":"2023-10-14T10:37:01.788029+0000","last_clean":"2023-10-14T10:37:01.788029+0000","last_became_active":"2023-10-14T10:36:59.063130+0000","last_became_peered":"2023-10-14T10:36:59.063130+0000","last_unstale":"2023-10-14T10:37:01.788029+0000","last_undegraded":"2023-10-14T10:37:01.788029+0000","last_fullsized":"2023-10-14T10:37:01.788029+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:59:06.202215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.061998+0000","last_change":"2023-10-14T10:37:00.068870+0000","last_active":"2023-10-14T10:37:01.061998+0000","last_peered":"2023-10-14T10:37:01.061998+0000","last_clean":"2023-10-14T10:37:01.061998+0000","last_became_active":"2023-10-14T10:37:00.068556+0000","last_became_peered":"2023-10-14T10:37:00.068556+0000","last_unstale":"2023-10-14T10:37:01.061998+0000","last_undegraded":"2023-10-14T10:37:01.061998+0000","last_fullsized":"2023-10-14T10:37:01.061998+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T14:52:18.060480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.787484+0000","last_change":"2023-10-14T10:36:59.066703+0000","last_active":"2023-10-14T10:37:01.787484+0000","last_peered":"2023-10-14T10:37:01.787484+0000","last_clean":"2023-10-14T10:37:01.787484+0000","last_became_active":"2023-10-14T10:36:59.066424+0000","last_became_peered":"2023-10-14T10:36:59.066424+0000","last_unstale":"2023-10-14T10:37:01.787484+0000","last_undegraded":"2023-10-14T10:37:01.787484+0000","last_fullsized":"2023-10-14T10:37:01.787484+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:54:15.011628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":176,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-14T10:37:00.060617+0000","last_change":"2023-10-14T10:35:59.770163+0000","last_active":"2023-10-14T10:37:00.060617+0000","last_peered":"2023-10-14T10:37:00.060617+0000","last_clean":"2023-10-14T10:37:00.060617+0000","last_became_active":"2023-10-14T10:35:59.769199+0000","last_became_peered":"2023-10-14T10:35:59.769199+0000","last_unstale":"2023-10-14T10:37:00.060617+0000","last_undegraded":"2023-10-14T10:37:00.060617+0000","last_fullsized":"2023-10-14T10:37:00.060617+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:35:58.675572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:35:58.675572+0000","last_clean_scrub_stamp":"2023-10-14T10:35:58.675572+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:03:25.281187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542161,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27956,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343884,"statfs":{"total":96636764160,"available":96608137216,"internally_reserved":0,"allocated":954368,"data_stored":726269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 10:36:59 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":2.4649999999999999,"5min":2.4649999999999999,"15min":2.4649999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.50700000000000001}]},{"osd":1,"last update":"Sat Oct 14 10:36:59 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":2.4289999999999998,"5min":2.4289999999999998,"15min":2.4289999999999998},"last":0.43099999999999999}]},{"osd":2,"last update":"Sat Oct 14 10:36:59 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.54800000000000004}]},{"osd":3,"last update":"Sat Oct 14 10:36:59 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.405,"5min":1.405,"15min":1.405},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.56100000000000005}]},{"osd":4,"last update":"Sat Oct 14 10:36:59 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.57899999999999996}]},{"osd":5,"last update":"Sat Oct 14 10:36:59 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.59299999999999997}]},{"osd":6,"last update":"Sat Oct 14 10:36:59 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.623}]}]},{"osd":6,"up_from":14,"seq":60129542162,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27388,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344452,"statfs":{"total":96636764160,"available":96608718848,"internally_reserved":0,"allocated":376832,"data_stored":142752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 10:36:57 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.34599999999999997}]},{"osd":1,"last update":"Sat Oct 14 10:36:57 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.389,"5min":1.389,"15min":1.389},"last":0.60299999999999998}]},{"osd":2,"last update":"Sat Oct 14 10:36:57 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.436},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.53000000000000003}]},{"osd":3,"last update":"Sat Oct 14 10:36:57 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.56599999999999995}]},{"osd":4,"last update":"Sat Oct 14 10:36:57 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.65800000000000003}]},{"osd":5,"last update":"Sat Oct 14 10:36:57 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.61899999999999999}]},{"osd":7,"last update":"Sat Oct 14 10:36:57 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.63400000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542161,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":376,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":385024,"data_stored":143770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.254,"15min":1.254},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":16.855,"5min":16.855,"15min":16.855},"last":0.311},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":19.353000000000002,"5min":19.353000000000002,"15min":19.353000000000002},"last":0.371}]},{"osd":2,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":3.0670000000000002,"5min":3.0670000000000002,"15min":3.0670000000000002},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.35399999999999998}]},{"osd":3,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":10.005000000000001,"5min":10.005000000000001,"15min":10.005000000000001},"last":0.33500000000000002}]},{"osd":4,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":2.9500000000000002,"5min":2.9500000000000002,"15min":2.9500000000000002},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":5.7270000000000003,"5min":5.7270000000000003,"15min":5.7270000000000003},"last":0.59399999999999997}]},{"osd":5,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.129,"15min":1.129},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":9.8800000000000008,"5min":9.8800000000000008,"15min":9.8800000000000008},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":3.0350000000000001,"5min":3.0350000000000001,"15min":3.0350000000000001},"last":0.56499999999999995}]},{"osd":6,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":5.681,"5min":5.681,"15min":5.681},"last":0.72299999999999998},{"interface":"front","average":{"1min":1.173,"5min":1.173,"15min":1.173},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":9.9380000000000006,"5min":9.9380000000000006,"15min":9.9380000000000006},"last":0.78600000000000003}]},{"osd":7,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":9.9730000000000008,"5min":9.9730000000000008,"15min":9.9730000000000008},"last":0.626},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":3.0009999999999999,"5min":3.0009999999999999,"15min":3.0009999999999999},"last":0.73799999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542160,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27952,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343888,"statfs":{"total":96636764160,"available":96608141312,"internally_reserved":0,"allocated":950272,"data_stored":726179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.62,"5min":0.62,"15min":0.62},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.495,"5min":0.495,"15min":0.495},"last":0.34000000000000002}]},{"osd":2,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"last":0.23999999999999999}]},{"osd":3,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"last":0.39900000000000002}]},{"osd":4,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.58599999999999997}]},{"osd":5,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.60099999999999998}]},{"osd":6,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.54700000000000004}]},{"osd":7,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.56599999999999995}]}]},{"osd":2,"up_from":14,"seq":60129542161,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27376,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344464,"statfs":{"total":96636764160,"available":96608731136,"internally_reserved":0,"allocated":360448,"data_stored":135881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":1.429,"5min":1.429,"15min":1.429},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.35199999999999998}]},{"osd":1,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":1.5269999999999999,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":1.0780000000000001},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":1.1000000000000001}]},{"osd":3,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":4.0629999999999997,"5min":4.0629999999999997,"15min":4.0629999999999997},"last":4.0629999999999997},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.381,"5min":1.381,"15min":1.381},"last":0.46000000000000002}]},{"osd":4,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.55200000000000005}]},{"osd":5,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":3.8599999999999999,"5min":3.8599999999999999,"15min":3.8599999999999999},"last":3.8599999999999999},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.464,"5min":1.464,"15min":1.464},"last":0.52300000000000002}]},{"osd":6,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":3.9820000000000002,"5min":3.9820000000000002,"15min":3.9820000000000002},"last":3.9820000000000002}]},{"osd":7,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.62},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":3.9049999999999998,"5min":3.9049999999999998,"15min":3.9049999999999998},"last":3.9049999999999998}]}]},{"osd":3,"up_from":14,"seq":60129542161,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":376,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":385024,"data_stored":143770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.441},{"interface":"front","average":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"last":0.29099999999999998}]},{"osd":1,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"last":0.501}]},{"osd":2,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.437,"5min":0.437,"15min":0.437},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.55800000000000005}]},{"osd":4,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":6.0350000000000001,"5min":6.0350000000000001,"15min":6.0350000000000001},"last":0.66800000000000004}]},{"osd":5,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":6.0540000000000003,"5min":6.0540000000000003,"15min":6.0540000000000003},"last":0.59599999999999997}]},{"osd":6,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.61299999999999999}]},{"osd":7,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":6.0819999999999999,"5min":6.0819999999999999,"15min":6.0819999999999999},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.63300000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542160,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":364544,"data_stored":135915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 10:36:56 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.59499999999999997}]},{"osd":1,"last update":"Sat Oct 14 10:36:56 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.64600000000000002}]},{"osd":2,"last update":"Sat Oct 14 10:36:56 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.51100000000000001}]},{"osd":3,"last update":"Sat Oct 14 10:36:56 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.624},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.68000000000000005}]},{"osd":5,"last update":"Sat Oct 14 10:36:56 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.70199999999999996}]},{"osd":6,"last update":"Sat Oct 14 10:36:56 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.84099999999999997}]},{"osd":7,"last update":"Sat Oct 14 10:36:56 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.75800000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542161,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":364544,"data_stored":136483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.72399999999999998}]},{"osd":1,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.627},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.66600000000000004}]},{"osd":2,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.441}]},{"osd":3,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.495},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.68500000000000005}]},{"osd":4,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.60099999999999998}]},{"osd":6,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.70499999999999996}]},{"osd":7,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.745}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-14T10:37:04.614 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-14T10:37:04.614 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=testcephfs1' mds allow 2023-10-14T10:37:04.780 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.778+0000 7efc1dd50700 1 Processor -- start 2023-10-14T10:37:04.780 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.778+0000 7efc1dd50700 1 -- start start 2023-10-14T10:37:04.782 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.778+0000 7efc1dd50700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7efc1806af50 0x7efc180b1f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:04.783 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.778+0000 7efc1dd50700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7efc180cc7d0 0x7efc180cab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:04.783 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.778+0000 7efc1dd50700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc1815df90 con 0x7efc1815dbb0 2023-10-14T10:37:04.783 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.778+0000 7efc1dd50700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc180cb040 con 0x7efc180cc7d0 2023-10-14T10:37:04.783 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.778+0000 7efc1dd50700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc18161a60 con 0x7efc1806af50 2023-10-14T10:37:04.783 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.779+0000 7efc17fff700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7efc180cc7d0 0x7efc180cab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:04.784 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.779+0000 7efc177fe700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7efc1815dbb0 0x7efc1806a840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:53628/0 (socket says 172.21.15.67:53628) 2023-10-14T10:37:04.784 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.779+0000 7efc17fff700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7efc180cc7d0 0x7efc180cab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45864/0 (socket says 172.21.15.67:45864) 2023-10-14T10:37:04.784 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.779+0000 7efc177fe700 1 -- 172.21.15.67:0/340645070 learned_addr learned my addr 172.21.15.67:0/340645070 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:37:04.784 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.779+0000 7efc16ffd700 1 --2- 172.21.15.67:0/340645070 >> v2:172.21.15.72:3300/0 conn(0x7efc1806af50 0x7efc180b1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:04.784 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.779+0000 7efc17fff700 1 -- 172.21.15.67:0/340645070 >> v2:172.21.15.72:3300/0 conn(0x7efc1806af50 msgr2=0x7efc180b1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:37:04.784 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.779+0000 7efc17fff700 1 --2- 172.21.15.67:0/340645070 >> v2:172.21.15.72:3300/0 conn(0x7efc1806af50 0x7efc180b1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:04.785 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.779+0000 7efc17fff700 1 -- 172.21.15.67:0/340645070 >> v1:172.21.15.67:6789/0 conn(0x7efc1815dbb0 legacy=0x7efc1806a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:04.785 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.779+0000 7efc17fff700 1 -- 172.21.15.67:0/340645070 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc18161d40 con 0x7efc180cc7d0 2023-10-14T10:37:04.785 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.779+0000 7efc167fc700 1 -- 172.21.15.67:0/340645070 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3397512029 0 0) 0x7efc1815df90 con 0x7efc1815dbb0 2023-10-14T10:37:04.785 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.779+0000 7efc16ffd700 1 --2- 172.21.15.67:0/340645070 >> v2:172.21.15.72:3300/0 conn(0x7efc1806af50 0x7efc180b1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-14T10:37:04.785 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.779+0000 7efc17fff700 1 --2- 172.21.15.67:0/340645070 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc180cc7d0 0x7efc180cab00 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7efc180cc710 tx=0x7efc1000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0c6d8a15a5805b4 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:04.786 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.779+0000 7efc167fc700 1 -- 172.21.15.67:0/340645070 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc1000e070 con 0x7efc180cc7d0 2023-10-14T10:37:04.786 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.779+0000 7efc167fc700 1 -- 172.21.15.67:0/340645070 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc1000bbd0 con 0x7efc180cc7d0 2023-10-14T10:37:04.786 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.779+0000 7efc167fc700 1 -- 172.21.15.67:0/340645070 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc10012430 con 0x7efc180cc7d0 2023-10-14T10:37:04.786 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.780+0000 7efc1dd50700 1 -- 172.21.15.67:0/340645070 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc180cc7d0 msgr2=0x7efc180cab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:04.786 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.780+0000 7efc1dd50700 1 --2- 172.21.15.67:0/340645070 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc180cc7d0 0x7efc180cab00 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7efc180cc710 tx=0x7efc1000b330 comp rx=0 tx=0).stop 2023-10-14T10:37:04.787 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.780+0000 7efc1dd50700 1 -- 172.21.15.67:0/340645070 shutdown_connections 2023-10-14T10:37:04.787 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.780+0000 7efc1dd50700 1 --2- 172.21.15.67:0/340645070 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc180cc7d0 0x7efc180cab00 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:04.787 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.780+0000 7efc1dd50700 1 --2- 172.21.15.67:0/340645070 >> v2:172.21.15.72:3300/0 conn(0x7efc1806af50 0x7efc180b1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:04.787 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.780+0000 7efc1dd50700 1 -- 172.21.15.67:0/340645070 >> 172.21.15.67:0/340645070 conn(0x7efc1806dac0 msgr2=0x7efc1806dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:37:04.787 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.780+0000 7efc1dd50700 1 -- 172.21.15.67:0/340645070 shutdown_connections 2023-10-14T10:37:04.788 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.780+0000 7efc1dd50700 1 -- 172.21.15.67:0/340645070 wait complete. 2023-10-14T10:37:04.790 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.781+0000 7efc1dd50700 1 Processor -- start 2023-10-14T10:37:04.791 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.781+0000 7efc1dd50700 1 -- start start 2023-10-14T10:37:04.791 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.782+0000 7efc1dd50700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc1806af50 0x7efc181f2db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:04.791 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.782+0000 7efc1dd50700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7efc180cc7d0 0x7efc18160aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:04.791 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.782+0000 7efc1dd50700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efc1815dbb0 0x7efc1815ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:04.791 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.782+0000 7efc1dd50700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc181f32f0 con 0x7efc1806af50 2023-10-14T10:37:04.792 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.782+0000 7efc1dd50700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc180af4d0 con 0x7efc1815dbb0 2023-10-14T10:37:04.792 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.782+0000 7efc1dd50700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc1815f4d0 con 0x7efc180cc7d0 2023-10-14T10:37:04.792 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.782+0000 7efc177fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc1806af50 0x7efc181f2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:04.792 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.782+0000 7efc177fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc1806af50 0x7efc181f2db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45866/0 (socket says 172.21.15.67:45866) 2023-10-14T10:37:04.792 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.782+0000 7efc177fe700 1 -- 172.21.15.67:0/2251916777 learned_addr learned my addr 172.21.15.67:0/2251916777 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:37:04.793 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.782+0000 7efc16ffd700 1 --2- 172.21.15.67:0/2251916777 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7efc180cc7d0 0x7efc18160aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:04.793 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.782+0000 7efc177fe700 1 -- 172.21.15.67:0/2251916777 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7efc180cc7d0 msgr2=0x7efc18160aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:04.793 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.782+0000 7efc177fe700 1 --2- 172.21.15.67:0/2251916777 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7efc180cc7d0 0x7efc18160aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:04.793 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.782+0000 7efc17fff700 1 --2- 172.21.15.67:0/2251916777 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efc1815dbb0 0x7efc1815ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:04.793 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.782+0000 7efc177fe700 1 -- 172.21.15.67:0/2251916777 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efc1815dbb0 msgr2=0x7efc1815ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:04.793 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.782+0000 7efc177fe700 1 --2- 172.21.15.67:0/2251916777 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efc1815dbb0 0x7efc1815ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:04.794 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.782+0000 7efc177fe700 1 -- 172.21.15.67:0/2251916777 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc10009a70 con 0x7efc1806af50 2023-10-14T10:37:04.794 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.783+0000 7efc177fe700 1 --2- 172.21.15.67:0/2251916777 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc1806af50 0x7efc181f2db0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7efc0800c550 tx=0x7efc0800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:04.794 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.783+0000 7efbfffff700 1 -- 172.21.15.67:0/2251916777 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc08004c80 con 0x7efc1806af50 2023-10-14T10:37:04.794 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.783+0000 7efbfffff700 1 -- 172.21.15.67:0/2251916777 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc08004e00 con 0x7efc1806af50 2023-10-14T10:37:04.794 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.783+0000 7efc1dd50700 1 -- 172.21.15.67:0/2251916777 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc1815f7b0 con 0x7efc1806af50 2023-10-14T10:37:04.795 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.783+0000 7efbfffff700 1 -- 172.21.15.67:0/2251916777 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc08007310 con 0x7efc1806af50 2023-10-14T10:37:04.795 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.783+0000 7efc1dd50700 1 -- 172.21.15.67:0/2251916777 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc1815f1c0 con 0x7efc1806af50 2023-10-14T10:37:04.795 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.784+0000 7efc1dd50700 1 -- 172.21.15.67:0/2251916777 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc18067d20 con 0x7efc1806af50 2023-10-14T10:37:04.795 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.784+0000 7efbfffff700 1 -- 172.21.15.67:0/2251916777 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7efc08007470 con 0x7efc1806af50 2023-10-14T10:37:04.795 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.785+0000 7efbfffff700 1 --2- 172.21.15.67:0/2251916777 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7efc0406af00 0x7efc0406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:04.796 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.785+0000 7efbfffff700 1 -- 172.21.15.67:0/2251916777 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5729+0+0 (secure 0 0 0) 0x7efc08089280 con 0x7efc1806af50 2023-10-14T10:37:04.796 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.785+0000 7efc16ffd700 1 --2- 172.21.15.67:0/2251916777 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7efc0406af00 0x7efc0406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:04.796 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.785+0000 7efc16ffd700 1 --2- 172.21.15.67:0/2251916777 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7efc0406af00 0x7efc0406d3c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7efc180c6fd0 tx=0x7efc0c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:04.796 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:04.789+0000 7efbfffff700 1 -- 172.21.15.67:0/2251916777 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7efc080543a0 con 0x7efc1806af50 2023-10-14T10:37:05.071 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:05.069+0000 7efc1dd50700 1 -- 172.21.15.67:0/2251916777 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=testcephfs1", "mds", "allow"]} v 0) v1 -- 0x7efc18161ff0 con 0x7efc1806af50 2023-10-14T10:37:05.073 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:05.071+0000 7efbfffff700 1 -- 172.21.15.67:0/2251916777 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=testcephfs1", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 195+0+152 (secure 0 0 0) 0x7efc080579c0 con 0x7efc1806af50 2023-10-14T10:37:05.074 INFO:teuthology.orchestra.run.smithi067.stdout:[client.0] 2023-10-14T10:37:05.074 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQDgbiplgPnBERAAH7t74KYvneMtdrMTWfuThw== 2023-10-14T10:37:05.074 INFO:teuthology.orchestra.run.smithi067.stdout: caps mds = "allow" 2023-10-14T10:37:05.074 INFO:teuthology.orchestra.run.smithi067.stdout: caps mon = "allow r" 2023-10-14T10:37:05.074 INFO:teuthology.orchestra.run.smithi067.stdout: caps osd = "allow rw tag cephfs data=testcephfs1" 2023-10-14T10:37:05.076 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:05.074+0000 7efc1dd50700 1 -- 172.21.15.67:0/2251916777 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7efc0406af00 msgr2=0x7efc0406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:05.076 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:05.074+0000 7efc1dd50700 1 --2- 172.21.15.67:0/2251916777 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7efc0406af00 0x7efc0406d3c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7efc180c6fd0 tx=0x7efc0c008040 comp rx=0 tx=0).stop 2023-10-14T10:37:05.076 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:05.074+0000 7efc1dd50700 1 -- 172.21.15.67:0/2251916777 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc1806af50 msgr2=0x7efc181f2db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:05.077 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:05.074+0000 7efc1dd50700 1 --2- 172.21.15.67:0/2251916777 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc1806af50 0x7efc181f2db0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7efc0800c550 tx=0x7efc0800c910 comp rx=0 tx=0).stop 2023-10-14T10:37:05.077 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:05.074+0000 7efc1dd50700 1 -- 172.21.15.67:0/2251916777 shutdown_connections 2023-10-14T10:37:05.077 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:05.074+0000 7efc1dd50700 1 --2- 172.21.15.67:0/2251916777 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7efc0406af00 0x7efc0406d3c0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:05.077 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:05.074+0000 7efc1dd50700 1 --2- 172.21.15.67:0/2251916777 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efc1815dbb0 0x7efc1815ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:05.077 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:05.074+0000 7efc1dd50700 1 --2- 172.21.15.67:0/2251916777 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc1806af50 0x7efc181f2db0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:05.078 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:05.074+0000 7efc1dd50700 1 --2- 172.21.15.67:0/2251916777 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7efc180cc7d0 0x7efc18160aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:05.078 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:05.074+0000 7efc1dd50700 1 -- 172.21.15.67:0/2251916777 >> 172.21.15.67:0/2251916777 conn(0x7efc1806dac0 msgr2=0x7efc180ce900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:37:05.078 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:05.074+0000 7efc1dd50700 1 -- 172.21.15.67:0/2251916777 shutdown_connections 2023-10-14T10:37:05.078 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:05.074+0000 7efc1dd50700 1 -- 172.21.15.67:0/2251916777 wait complete. 2023-10-14T10:37:05.079 INFO:teuthology.orchestra.run.smithi067.stderr:updated caps for client.0 2023-10-14T10:37:05.088 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-14T10:37:05.213 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:05.211+0000 7ff1dd776700 1 Processor -- start 2023-10-14T10:37:05.213 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:05.211+0000 7ff1dd776700 1 -- start start 2023-10-14T10:37:05.216 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:05.211+0000 7ff1dd776700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7ff1d815eab0 0x7ff1d80cf190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:05.216 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:05.211+0000 7ff1dd776700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7ff1d80cf6d0 0x7ff1d80d1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:05.216 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:05.211+0000 7ff1dd776700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff1d80d1bc0 0x7ff1d8161510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:05.216 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:05.211+0000 7ff1dd776700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1d80cfb50 con 0x7ff1d80d1bc0 2023-10-14T10:37:05.216 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:05.211+0000 7ff1dd776700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1d815ee90 con 0x7ff1d80cf6d0 2023-10-14T10:37:05.217 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:05.211+0000 7ff1dd776700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d8158370 con 0x7ff1d815eab0 2023-10-14T10:37:05.217 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:05.212+0000 7ff1d77fe700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff1d80d1bc0 0x7ff1d8161510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:05.217 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:05.212+0000 7ff1d77fe700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff1d80d1bc0 0x7ff1d8161510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45876/0 (socket says 172.21.15.67:45876) 2023-10-14T10:37:05.217 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:05.212+0000 7ff1d77fe700 1 -- 172.21.15.67:0/3559985443 learned_addr learned my addr 172.21.15.67:0/3559985443 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:37:05.217 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:05.212+0000 7ff1d6ffd700 1 --2- 172.21.15.67:0/3559985443 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7ff1d815eab0 0x7ff1d80cf190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:05.218 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:05.212+0000 7ff1d67fc700 1 --2- 172.21.15.67:0/3559985443 >> v2:172.21.15.72:3300/0 conn(0x7ff1d80cf6d0 0x7ff1d80d1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:05.218 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:05.212+0000 7ff1d77fe700 1 -- 172.21.15.67:0/3559985443 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7ff1d815eab0 msgr2=0x7ff1d80cf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:37:05.218 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:05.212+0000 7ff1d77fe700 1 --2- 172.21.15.67:0/3559985443 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7ff1d815eab0 0x7ff1d80cf190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:05.218 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:05.212+0000 7ff1d77fe700 1 -- 172.21.15.67:0/3559985443 >> v2:172.21.15.72:3300/0 conn(0x7ff1d80cf6d0 msgr2=0x7ff1d80d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:37:05.218 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:05.212+0000 7ff1d77fe700 1 --2- 172.21.15.67:0/3559985443 >> v2:172.21.15.72:3300/0 conn(0x7ff1d80cf6d0 0x7ff1d80d1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:05.219 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:05.212+0000 7ff1d77fe700 1 -- 172.21.15.67:0/3559985443 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1d80d23b0 con 0x7ff1d80d1bc0 2023-10-14T10:37:05.219 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:05.212+0000 7ff1d67fc700 1 --2- 172.21.15.67:0/3559985443 >> v2:172.21.15.72:3300/0 conn(0x7ff1d80cf6d0 0x7ff1d80d1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-14T10:37:05.219 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:05.212+0000 7ff1d77fe700 1 --2- 172.21.15.67:0/3559985443 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1d80d1bc0 0x7ff1d8161510 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ff1d806fd00 tx=0x7ff1d000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70cc02ad80594a99 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:05.219 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:05.212+0000 7ff1d5ffb700 1 -- 172.21.15.67:0/3559985443 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d000e070 con 0x7ff1d80d1bc0 2023-10-14T10:37:05.219 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:05.212+0000 7ff1d5ffb700 1 -- 172.21.15.67:0/3559985443 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1d000b9c0 con 0x7ff1d80d1bc0 2023-10-14T10:37:05.219 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:05.212+0000 7ff1d5ffb700 1 -- 172.21.15.67:0/3559985443 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d000bd70 con 0x7ff1d80d1bc0 2023-10-14T10:37:05.220 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:05.213+0000 7ff1dd776700 1 -- 172.21.15.67:0/3559985443 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1d80d1bc0 msgr2=0x7ff1d8161510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:05.220 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:05.213+0000 7ff1dd776700 1 --2- 172.21.15.67:0/3559985443 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1d80d1bc0 0x7ff1d8161510 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ff1d806fd00 tx=0x7ff1d000b330 comp rx=0 tx=0).stop 2023-10-14T10:37:05.220 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:05.213+0000 7ff1dd776700 1 -- 172.21.15.67:0/3559985443 shutdown_connections 2023-10-14T10:37:05.220 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:05.213+0000 7ff1dd776700 1 --2- 172.21.15.67:0/3559985443 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1d80d1bc0 0x7ff1d8161510 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:05.220 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:05.213+0000 7ff1dd776700 1 --2- 172.21.15.67:0/3559985443 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7ff1d815eab0 0x7ff1d80cf190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:05.221 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:05.213+0000 7ff1dd776700 1 --2- 172.21.15.67:0/3559985443 >> v2:172.21.15.72:3300/0 conn(0x7ff1d80cf6d0 0x7ff1d80d1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:05.221 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:05.213+0000 7ff1dd776700 1 -- 172.21.15.67:0/3559985443 >> 172.21.15.67:0/3559985443 conn(0x7ff1d80665a0 msgr2=0x7ff1d8066980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:37:05.221 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:05.213+0000 7ff1dd776700 1 -- 172.21.15.67:0/3559985443 shutdown_connections 2023-10-14T10:37:05.221 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:05.213+0000 7ff1dd776700 1 -- 172.21.15.67:0/3559985443 wait complete. 2023-10-14T10:37:05.224 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:05.214+0000 7ff1dd776700 1 Processor -- start 2023-10-14T10:37:05.224 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:05.214+0000 7ff1dd776700 1 -- start start 2023-10-14T10:37:05.224 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:05.215+0000 7ff1dd776700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7ff1d80cf6d0 0x7ff1d80b6c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:05.225 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:05.215+0000 7ff1dd776700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff1d80d1bc0 0x7ff1d80b59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:05.225 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:05.215+0000 7ff1dd776700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1d815eab0 0x7ff1d80c6960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:05.225 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:05.215+0000 7ff1dd776700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d80b5ee0 con 0x7ff1d815eab0 2023-10-14T10:37:05.225 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:05.215+0000 7ff1dd776700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d80c6ea0 con 0x7ff1d80d1bc0 2023-10-14T10:37:05.225 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:05.215+0000 7ff1dd776700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d80b6140 con 0x7ff1d80cf6d0 2023-10-14T10:37:05.226 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:05.215+0000 7ff1d77fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1d815eab0 0x7ff1d80c6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:05.226 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:05.215+0000 7ff1d77fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1d815eab0 0x7ff1d80c6960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45884/0 (socket says 172.21.15.67:45884) 2023-10-14T10:37:05.226 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:05.215+0000 7ff1d77fe700 1 -- 172.21.15.67:0/764471017 learned_addr learned my addr 172.21.15.67:0/764471017 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:37:05.226 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:05.215+0000 7ff1d67fc700 1 --2- 172.21.15.67:0/764471017 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff1d80d1bc0 0x7ff1d80b59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:05.226 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:05.215+0000 7ff1d6ffd700 1 --2- 172.21.15.67:0/764471017 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7ff1d80cf6d0 0x7ff1d80b6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:05.227 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:05.215+0000 7ff1d77fe700 1 -- 172.21.15.67:0/764471017 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7ff1d80cf6d0 msgr2=0x7ff1d80b6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:05.227 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:05.215+0000 7ff1d77fe700 1 --2- 172.21.15.67:0/764471017 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7ff1d80cf6d0 0x7ff1d80b6c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:05.227 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:05.215+0000 7ff1d77fe700 1 -- 172.21.15.67:0/764471017 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff1d80d1bc0 msgr2=0x7ff1d80b59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:05.227 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:05.215+0000 7ff1d77fe700 1 --2- 172.21.15.67:0/764471017 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff1d80d1bc0 0x7ff1d80b59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:05.227 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:05.215+0000 7ff1d77fe700 1 -- 172.21.15.67:0/764471017 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1d0009b30 con 0x7ff1d815eab0 2023-10-14T10:37:05.228 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:05.215+0000 7ff1d67fc700 1 --2- 172.21.15.67:0/764471017 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff1d80d1bc0 0x7ff1d80b59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-14T10:37:05.228 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:05.216+0000 7ff1d77fe700 1 --2- 172.21.15.67:0/764471017 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1d815eab0 0x7ff1d80c6960 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7ff1d80cee90 tx=0x7ff1d0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:05.228 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:05.216+0000 7ff1bf7fe700 1 -- 172.21.15.67:0/764471017 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d0004250 con 0x7ff1d815eab0 2023-10-14T10:37:05.228 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:05.216+0000 7ff1bf7fe700 1 -- 172.21.15.67:0/764471017 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1d00043d0 con 0x7ff1d815eab0 2023-10-14T10:37:05.228 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:05.216+0000 7ff1bf7fe700 1 -- 172.21.15.67:0/764471017 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d0004780 con 0x7ff1d815eab0 2023-10-14T10:37:05.228 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:05.216+0000 7ff1dd776700 1 -- 172.21.15.67:0/764471017 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1d80c5ba0 con 0x7ff1d815eab0 2023-10-14T10:37:05.229 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:05.216+0000 7ff1dd776700 1 -- 172.21.15.67:0/764471017 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1d80c6160 con 0x7ff1d815eab0 2023-10-14T10:37:05.229 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:05.217+0000 7ff1dd776700 1 -- 172.21.15.67:0/764471017 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1d8067d20 con 0x7ff1d815eab0 2023-10-14T10:37:05.229 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:05.217+0000 7ff1bf7fe700 1 -- 172.21.15.67:0/764471017 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff1d001c440 con 0x7ff1d815eab0 2023-10-14T10:37:05.229 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:05.218+0000 7ff1bf7fe700 1 --2- 172.21.15.67:0/764471017 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7ff1c406af00 0x7ff1c406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:05.229 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:05.218+0000 7ff1bf7fe700 1 -- 172.21.15.67:0/764471017 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5729+0+0 (secure 0 0 0) 0x7ff1d008ab40 con 0x7ff1d815eab0 2023-10-14T10:37:05.229 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:05.218+0000 7ff1d6ffd700 1 --2- 172.21.15.67:0/764471017 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7ff1c406af00 0x7ff1c406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:05.230 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:05.219+0000 7ff1d6ffd700 1 --2- 172.21.15.67:0/764471017 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7ff1c406af00 0x7ff1c406d3c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff1c8009950 tx=0x7ff1c8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:05.230 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:05.222+0000 7ff1bf7fe700 1 -- 172.21.15.67:0/764471017 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff1d0055c60 con 0x7ff1d815eab0 2023-10-14T10:37:05.465 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:05.463+0000 7ff1dd776700 1 -- 172.21.15.67:0/764471017 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff1d80b7540 con 0x7ff1d815eab0 2023-10-14T10:37:05.466 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:05.464+0000 7ff1bf7fe700 1 -- 172.21.15.67:0/764471017 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4918 (secure 0 0 0) 0x7ff1d002a090 con 0x7ff1d815eab0 2023-10-14T10:37:05.466 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-14T10:37:05.466 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6836/3371338659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":3371338659},{"type":"v1","addr":"172.21.15.72:6836","nonce":3371338659}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4688,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/2815272197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6835","nonce":2815272197},{"type":"v1","addr":"172.21.15.72:6837","nonce":2815272197}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5094,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6834/3494180260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":3494180260},{"type":"v1","addr":"172.21.15.67:6834","nonce":3494180260}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-14T10:36:59.462859+0000","modified":"2023-10-14T10:37:02.493817+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5097},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5097":{"gid":5097,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.67:6835/3065963596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6833","nonce":3065963596},{"type":"v1","addr":"172.21.15.67:6835","nonce":3065963596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"testcephfs1","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-14T10:37:05.468 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:05.466+0000 7ff1dd776700 1 -- 172.21.15.67:0/764471017 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7ff1c406af00 msgr2=0x7ff1c406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:05.468 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:05.466+0000 7ff1dd776700 1 --2- 172.21.15.67:0/764471017 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7ff1c406af00 0x7ff1c406d3c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff1c8009950 tx=0x7ff1c8008040 comp rx=0 tx=0).stop 2023-10-14T10:37:05.469 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:05.466+0000 7ff1dd776700 1 -- 172.21.15.67:0/764471017 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1d815eab0 msgr2=0x7ff1d80c6960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:05.469 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:05.466+0000 7ff1dd776700 1 --2- 172.21.15.67:0/764471017 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1d815eab0 0x7ff1d80c6960 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7ff1d80cee90 tx=0x7ff1d0004ad0 comp rx=0 tx=0).stop 2023-10-14T10:37:05.469 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:05.466+0000 7ff1dd776700 1 -- 172.21.15.67:0/764471017 shutdown_connections 2023-10-14T10:37:05.469 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:05.466+0000 7ff1dd776700 1 --2- 172.21.15.67:0/764471017 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7ff1c406af00 0x7ff1c406d3c0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:05.469 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:05.466+0000 7ff1dd776700 1 --2- 172.21.15.67:0/764471017 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1d815eab0 0x7ff1d80c6960 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:05.470 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:05.466+0000 7ff1dd776700 1 --2- 172.21.15.67:0/764471017 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7ff1d80cf6d0 0x7ff1d80b6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:05.470 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:05.466+0000 7ff1dd776700 1 --2- 172.21.15.67:0/764471017 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff1d80d1bc0 0x7ff1d80b59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:05.470 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:05.466+0000 7ff1dd776700 1 -- 172.21.15.67:0/764471017 >> 172.21.15.67:0/764471017 conn(0x7ff1d80665a0 msgr2=0x7ff1d80b12f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:37:05.470 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:05.466+0000 7ff1dd776700 1 -- 172.21.15.67:0/764471017 shutdown_connections 2023-10-14T10:37:05.470 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:05.466+0000 7ff1dd776700 1 -- 172.21.15.67:0/764471017 wait complete. 2023-10-14T10:37:05.471 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 16 2023-10-14T10:37:05.882 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-14T10:36:59.462859+0000', 'modified': '2023-10-14T10:37:02.493817+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5097}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5097': {'gid': 5097, 'name': 'c', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.67:6835/3065963596', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6833', 'nonce': 3065963596}, {'type': 'v1', 'addr': '172.21.15.67:6835', 'nonce': 3065963596}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'testcephfs1', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-14T10:37:05.883 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-14T10:37:05.883 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-14T10:37:06.012 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.010+0000 7f01b7ff1700 1 Processor -- start 2023-10-14T10:37:06.013 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.011+0000 7f01b7ff1700 1 -- start start 2023-10-14T10:37:06.015 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.011+0000 7f01b7ff1700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f01b015b8b0 0x7f01b00b2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:06.015 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.011+0000 7f01b7ff1700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01b0072a10 con 0x7f01b00b4640 2023-10-14T10:37:06.015 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.011+0000 7f01b7ff1700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01b00cdce0 con 0x7f01b00ce3b0 2023-10-14T10:37:06.016 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.011+0000 7f01b7ff1700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01b00b2230 con 0x7f01b015b8b0 2023-10-14T10:37:06.016 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.011+0000 7f01b5d8d700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f01b015b8b0 0x7f01b00b2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:06.016 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.012+0000 7f01b658e700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f01b00b4640 0x7f01b00cd5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:53656/0 (socket says 172.21.15.67:53656) 2023-10-14T10:37:06.017 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.012+0000 7f01b658e700 1 -- 172.21.15.67:0/2256674083 learned_addr learned my addr 172.21.15.67:0/2256674083 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:37:06.017 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.012+0000 7f01b5d8d700 1 -- 172.21.15.67:0/2256674083 >> v1:172.21.15.72:6789/0 conn(0x7f01b00ce3b0 legacy=0x7f01b00ccec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:06.017 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.012+0000 7f01b5d8d700 1 -- 172.21.15.67:0/2256674083 >> v1:172.21.15.67:6789/0 conn(0x7f01b00b4640 legacy=0x7f01b00cd5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:06.017 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.012+0000 7f01b4d8b700 1 -- 172.21.15.67:0/2256674083 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3152027716 0 0) 0x7f01b0072a10 con 0x7f01b00b4640 2023-10-14T10:37:06.017 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.012+0000 7f01b5d8d700 1 -- 172.21.15.67:0/2256674083 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01b00b2e90 con 0x7f01b015b8b0 2023-10-14T10:37:06.018 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.012+0000 7f01b5d8d700 1 --2- 172.21.15.67:0/2256674083 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01b015b8b0 0x7f01b00b2950 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f01a0009840 tx=0x7f01a0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6018fc0a9033e55a server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:06.018 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.012+0000 7f01b4d8b700 1 -- 172.21.15.67:0/2256674083 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a0011830 con 0x7f01b015b8b0 2023-10-14T10:37:06.018 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.012+0000 7f01b4d8b700 1 -- 172.21.15.67:0/2256674083 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01a0011990 con 0x7f01b015b8b0 2023-10-14T10:37:06.018 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.012+0000 7f01b4d8b700 1 -- 172.21.15.67:0/2256674083 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a0011d20 con 0x7f01b015b8b0 2023-10-14T10:37:06.019 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.013+0000 7f01b7ff1700 1 -- 172.21.15.67:0/2256674083 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01b015b8b0 msgr2=0x7f01b00b2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:06.019 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.013+0000 7f01b7ff1700 1 --2- 172.21.15.67:0/2256674083 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01b015b8b0 0x7f01b00b2950 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f01a0009840 tx=0x7f01a0009b50 comp rx=0 tx=0).stop 2023-10-14T10:37:06.019 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.013+0000 7f01b7ff1700 1 -- 172.21.15.67:0/2256674083 shutdown_connections 2023-10-14T10:37:06.019 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.013+0000 7f01b7ff1700 1 --2- 172.21.15.67:0/2256674083 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01b015b8b0 0x7f01b00b2950 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:06.019 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.013+0000 7f01b7ff1700 1 -- 172.21.15.67:0/2256674083 >> 172.21.15.67:0/2256674083 conn(0x7f01b006dac0 msgr2=0x7f01b006dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:37:06.020 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.013+0000 7f01b7ff1700 1 -- 172.21.15.67:0/2256674083 shutdown_connections 2023-10-14T10:37:06.020 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.013+0000 7f01b7ff1700 1 -- 172.21.15.67:0/2256674083 wait complete. 2023-10-14T10:37:06.030 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.014+0000 7f01b7ff1700 1 Processor -- start 2023-10-14T10:37:06.030 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.014+0000 7f01b7ff1700 1 -- start start 2023-10-14T10:37:06.030 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.015+0000 7f01b7ff1700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f01b00b4640 0x7f01b0150ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:06.030 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.015+0000 7f01b7ff1700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01b00ce3b0 0x7f01b015cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:06.031 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.015+0000 7f01b7ff1700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f01b015b8b0 0x7f01b01e51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:06.031 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.015+0000 7f01b7ff1700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01b00c6660 con 0x7f01b00ce3b0 2023-10-14T10:37:06.031 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.015+0000 7f01b7ff1700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01b00c5a50 con 0x7f01b015b8b0 2023-10-14T10:37:06.031 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.015+0000 7f01b7ff1700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01b00c5d30 con 0x7f01b00b4640 2023-10-14T10:37:06.031 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.015+0000 7f01b558c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01b00ce3b0 0x7f01b015cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:06.032 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.015+0000 7f01b558c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01b00ce3b0 0x7f01b015cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45896/0 (socket says 172.21.15.67:45896) 2023-10-14T10:37:06.032 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.015+0000 7f01b558c700 1 -- 172.21.15.67:0/2808007079 learned_addr learned my addr 172.21.15.67:0/2808007079 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:37:06.032 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.015+0000 7f01b5d8d700 1 --2- 172.21.15.67:0/2808007079 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f01b00b4640 0x7f01b0150ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:06.032 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.015+0000 7f01b658e700 1 --2- 172.21.15.67:0/2808007079 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f01b015b8b0 0x7f01b01e51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:06.032 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.015+0000 7f01b558c700 1 -- 172.21.15.67:0/2808007079 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f01b00b4640 msgr2=0x7f01b0150ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:06.033 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.015+0000 7f01b558c700 1 --2- 172.21.15.67:0/2808007079 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f01b00b4640 0x7f01b0150ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:06.033 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.015+0000 7f01b558c700 1 -- 172.21.15.67:0/2808007079 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f01b015b8b0 msgr2=0x7f01b01e51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:06.033 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.015+0000 7f01b558c700 1 --2- 172.21.15.67:0/2808007079 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f01b015b8b0 0x7f01b01e51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:06.033 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.015+0000 7f01b558c700 1 -- 172.21.15.67:0/2808007079 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01a00094a0 con 0x7f01b00ce3b0 2023-10-14T10:37:06.033 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.015+0000 7f01b558c700 1 --2- 172.21.15.67:0/2808007079 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01b00ce3b0 0x7f01b015cfb0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f01ac00c4e0 tx=0x7f01ac00c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:06.033 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.015+0000 7f01a67fc700 1 -- 172.21.15.67:0/2808007079 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01ac0052d0 con 0x7f01b00ce3b0 2023-10-14T10:37:06.034 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.016+0000 7f01a67fc700 1 -- 172.21.15.67:0/2808007079 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01ac005430 con 0x7f01b00ce3b0 2023-10-14T10:37:06.034 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.016+0000 7f01b7ff1700 1 -- 172.21.15.67:0/2808007079 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01b00c6010 con 0x7f01b00ce3b0 2023-10-14T10:37:06.034 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.016+0000 7f01a67fc700 1 -- 172.21.15.67:0/2808007079 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01ac0038c0 con 0x7f01b00ce3b0 2023-10-14T10:37:06.034 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.016+0000 7f01b7ff1700 1 -- 172.21.15.67:0/2808007079 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01b015d4f0 con 0x7f01b00ce3b0 2023-10-14T10:37:06.035 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.017+0000 7f01a67fc700 1 -- 172.21.15.67:0/2808007079 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f01ac003a20 con 0x7f01b00ce3b0 2023-10-14T10:37:06.035 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.018+0000 7f01a67fc700 1 --2- 172.21.15.67:0/2808007079 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f019406af00 0x7f019406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:06.035 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.018+0000 7f01a67fc700 1 -- 172.21.15.67:0/2808007079 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5729+0+0 (secure 0 0 0) 0x7f01ac0889c0 con 0x7f01b00ce3b0 2023-10-14T10:37:06.035 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.018+0000 7f01b5d8d700 1 --2- 172.21.15.67:0/2808007079 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f019406af00 0x7f019406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:06.035 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.018+0000 7f01b5d8d700 1 --2- 172.21.15.67:0/2808007079 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f019406af00 0x7f019406d3c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f01a00090b0 tx=0x7f01a001c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:06.036 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.020+0000 7f01c41dbb80 1 Processor -- start 2023-10-14T10:37:06.036 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.020+0000 7f01c41dbb80 1 -- start start 2023-10-14T10:37:06.036 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.020+0000 7f01c41dbb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564ba8707220 0x564ba870ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:06.036 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.020+0000 7f01c41dbb80 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x564ba870c630 0x564ba870efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:06.036 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.020+0000 7f01c41dbb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564ba86e2010 0x564ba86e26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:06.037 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.020+0000 7f01c41dbb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ba8658c20 con 0x564ba8707220 2023-10-14T10:37:06.037 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.020+0000 7f01c41dbb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ba863dc10 con 0x564ba86e2010 2023-10-14T10:37:06.037 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.020+0000 7f01c41dbb80 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x564ba863b530 con 0x564ba870c630 2023-10-14T10:37:06.037 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.020+0000 7f01b5d8d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564ba8707220 0x564ba870ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:06.037 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.021+0000 7f01b5d8d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564ba8707220 0x564ba870ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45902/0 (socket says 172.21.15.67:45902) 2023-10-14T10:37:06.038 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.021+0000 7f01b5d8d700 1 -- 172.21.15.67:0/1485196771 learned_addr learned my addr 172.21.15.67:0/1485196771 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:37:06.038 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.021+0000 7f01b658e700 1 --2- 172.21.15.67:0/1485196771 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564ba86e2010 0x564ba86e26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:06.038 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.021+0000 7f01b558c700 1 --2- 172.21.15.67:0/1485196771 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x564ba870c630 0x564ba870efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:06.038 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.021+0000 7f01b5d8d700 1 -- 172.21.15.67:0/1485196771 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x564ba870c630 msgr2=0x564ba870efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:37:06.038 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.021+0000 7f01b5d8d700 1 --2- 172.21.15.67:0/1485196771 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x564ba870c630 0x564ba870efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:06.039 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.021+0000 7f01b5d8d700 1 -- 172.21.15.67:0/1485196771 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564ba86e2010 msgr2=0x564ba86e26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:37:06.039 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.021+0000 7f01b5d8d700 1 --2- 172.21.15.67:0/1485196771 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564ba86e2010 0x564ba86e26c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:06.039 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.021+0000 7f01b5d8d700 1 -- 172.21.15.67:0/1485196771 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564ba85f43a0 con 0x564ba8707220 2023-10-14T10:37:06.039 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.021+0000 7f01b658e700 1 --2- 172.21.15.67:0/1485196771 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564ba86e2010 0x564ba86e26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-14T10:37:06.039 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.021+0000 7f01b5d8d700 1 --2- 172.21.15.67:0/1485196771 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564ba8707220 0x564ba870ea70 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f01a0011960 tx=0x7f01a0010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c6eb6a75f79e854 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:06.040 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.021+0000 7f01937fe700 1 -- 172.21.15.67:0/1485196771 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a00259d0 con 0x564ba8707220 2023-10-14T10:37:06.040 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.021+0000 7f01937fe700 1 -- 172.21.15.67:0/1485196771 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01a002b040 con 0x564ba8707220 2023-10-14T10:37:06.040 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.021+0000 7f01937fe700 1 -- 172.21.15.67:0/1485196771 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a0025d30 con 0x564ba8707220 2023-10-14T10:37:06.040 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.022+0000 7f01c41dbb80 1 -- 172.21.15.67:0/1485196771 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564ba8707220 msgr2=0x564ba870ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:06.040 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.022+0000 7f01c41dbb80 1 --2- 172.21.15.67:0/1485196771 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564ba8707220 0x564ba870ea70 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f01a0011960 tx=0x7f01a0010000 comp rx=0 tx=0).stop 2023-10-14T10:37:06.041 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.022+0000 7f01c41dbb80 1 -- 172.21.15.67:0/1485196771 shutdown_connections 2023-10-14T10:37:06.041 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.022+0000 7f01c41dbb80 1 --2- 172.21.15.67:0/1485196771 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564ba86e2010 0x564ba86e26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:06.041 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.022+0000 7f01c41dbb80 1 --2- 172.21.15.67:0/1485196771 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564ba8707220 0x564ba870ea70 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:06.041 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.022+0000 7f01c41dbb80 1 --2- 172.21.15.67:0/1485196771 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x564ba870c630 0x564ba870efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:06.041 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.022+0000 7f01c41dbb80 1 -- 172.21.15.67:0/1485196771 >> 172.21.15.67:0/1485196771 conn(0x564ba8611d80 msgr2=0x564ba872cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:37:06.042 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.022+0000 7f01c41dbb80 1 -- 172.21.15.67:0/1485196771 shutdown_connections 2023-10-14T10:37:06.042 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.022+0000 7f01c41dbb80 1 -- 172.21.15.67:0/1485196771 wait complete. 2023-10-14T10:37:06.042 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.022+0000 7f01c41dbb80 1 -- start start 2023-10-14T10:37:06.042 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.023+0000 7f01c41dbb80 1 Processor -- start 2023-10-14T10:37:06.042 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.023+0000 7f01c41dbb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564ba86e2010 0x564ba8706240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:06.043 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.024+0000 7f01c41dbb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564ba8707220 0x564ba8708330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:06.043 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.024+0000 7f01c41dbb80 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x564ba870c630 0x564ba8708870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:06.043 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.024+0000 7f01c41dbb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ba8636e20 con 0x564ba86e2010 2023-10-14T10:37:06.043 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.024+0000 7f01c41dbb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ba85d36c0 con 0x564ba8707220 2023-10-14T10:37:06.044 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.024+0000 7f01c41dbb80 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x564ba85d3360 con 0x564ba870c630 2023-10-14T10:37:06.044 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.024+0000 7f01c41dbb80 10 client.0 _ll_register_callbacks cb 0x564ba84307b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-14T10:37:06.044 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.024+0000 7f01b5d8d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564ba86e2010 0x564ba8706240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:06.044 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.024+0000 7f01b5d8d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564ba86e2010 0x564ba8706240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45908/0 (socket says 172.21.15.67:45908) 2023-10-14T10:37:06.044 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.024+0000 7f01b5d8d700 1 -- 172.21.15.67:0/875447513 learned_addr learned my addr 172.21.15.67:0/875447513 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:37:06.044 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.024+0000 7f01b558c700 1 --2- 172.21.15.67:0/875447513 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564ba8707220 0x564ba8708330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:06.045 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.024+0000 7f01b658e700 1 --2- 172.21.15.67:0/875447513 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x564ba870c630 0x564ba8708870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:06.045 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.024+0000 7f01b5d8d700 1 -- 172.21.15.67:0/875447513 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x564ba870c630 msgr2=0x564ba8708870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:37:06.045 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.024+0000 7f01b5d8d700 1 --2- 172.21.15.67:0/875447513 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x564ba870c630 0x564ba8708870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:06.045 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.024+0000 7f01b5d8d700 1 -- 172.21.15.67:0/875447513 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564ba8707220 msgr2=0x564ba8708330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:37:06.045 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.024+0000 7f01b5d8d700 1 --2- 172.21.15.67:0/875447513 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564ba8707220 0x564ba8708330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:06.046 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.024+0000 7f01b5d8d700 1 -- 172.21.15.67:0/875447513 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01a8001f20 con 0x564ba86e2010 2023-10-14T10:37:06.046 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.024+0000 7f01b558c700 1 --2- 172.21.15.67:0/875447513 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564ba8707220 0x564ba8708330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-14T10:37:06.046 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.024+0000 7f01b5d8d700 1 --2- 172.21.15.67:0/875447513 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564ba86e2010 0x564ba8706240 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f01a0009b90 tx=0x7f01a00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2441368dd9c4d5e3 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:06.046 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.024+0000 7f0190ff9700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2023-10-14T10:37:06.046 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.024+0000 7f0190ff9700 1 -- 172.21.15.67:0/875447513 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a0026260 con 0x564ba86e2010 2023-10-14T10:37:06.047 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.024+0000 7f0190ff9700 1 -- 172.21.15.67:0/875447513 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(33..33 src has 1..33) v4 ==== 5729+0+0 (secure 0 0 0) 0x7f01a001f040 con 0x564ba86e2010 2023-10-14T10:37:06.047 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.024+0000 7f01c41dbb80 1 -- 172.21.15.67:0/875447513 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564ba86d75b0 con 0x564ba86e2010 2023-10-14T10:37:06.047 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.024+0000 7f0190ff9700 1 -- 172.21.15.67:0/875447513 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f01a00263c0 con 0x564ba86e2010 2023-10-14T10:37:06.047 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.025+0000 7f01c41dbb80 10 client.5268 fetch_fsmap learned FSMap version 16 2023-10-14T10:37:06.047 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.025+0000 7f01c41dbb80 1 -- 172.21.15.67:0/875447513 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x7f01b00b2e90 con 0x564ba86e2010 2023-10-14T10:37:06.048 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.025+0000 7f0190ff9700 1 -- 172.21.15.67:0/875447513 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564ba86d75b0 con 0x564ba86e2010 2023-10-14T10:37:06.048 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.025+0000 7f0190ff9700 1 -- 172.21.15.67:0/875447513 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 16) v1 ==== 3299+0+0 (secure 0 0 0) 0x7f01a003e020 con 0x564ba86e2010 2023-10-14T10:37:06.048 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.025+0000 7f01c41dbb80 10 client.5268 fetch_fsmap finished waiting for FSMap version 16 2023-10-14T10:37:06.048 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.025+0000 7f01c41dbb80 10 client.5268 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-14T10:37:06.048 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.025+0000 7f01c41dbb80 20 client.5268 populate_metadata read hostname 'smithi067' 2023-10-14T10:37:06.049 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.025+0000 7f01c41dbb80 1 --2- 172.21.15.67:0/875447513 >> [v2:172.21.15.67:6833/3065963596,v1:172.21.15.67:6835/3065963596] conn(0x564ba87112e0 0x564ba87116c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:06.049 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.025+0000 7f01c41dbb80 4 client.5268 mds_command: new command op to 5097 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-14T10:37:06.049 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.025+0000 7f01c41dbb80 1 -- 172.21.15.67:0/875447513 --> [v2:172.21.15.67:6833/3065963596,v1:172.21.15.67:6835/3065963596] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564ba862fe00 con 0x564ba87112e0 2023-10-14T10:37:06.049 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.025+0000 7f01b558c700 1 --2- 172.21.15.67:0/875447513 >> [v2:172.21.15.67:6833/3065963596,v1:172.21.15.67:6835/3065963596] conn(0x564ba87112e0 0x564ba87116c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:06.049 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.025+0000 7f0190ff9700 10 client.5268 ms_handle_connect on v2:172.21.15.67:6833/3065963596 2023-10-14T10:37:06.050 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.025+0000 7f01b558c700 1 --2- 172.21.15.67:0/875447513 >> [v2:172.21.15.67:6833/3065963596,v1:172.21.15.67:6835/3065963596] conn(0x564ba87112e0 0x564ba87116c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=390a1e0eaa4f1b25 server_cookie=defa38f67f7e0ff6 in_seq=0 out_seq=0 2023-10-14T10:37:06.050 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.026+0000 7f0190ff9700 1 -- 172.21.15.67:0/875447513 <== mds.0 v2:172.21.15.67:6833/3065963596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564ba862fe00 con 0x564ba87112e0 2023-10-14T10:37:06.050 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.026+0000 7f0190ff9700 10 client.5268 handle_command_reply: tid=0 2023-10-14T10:37:06.050 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.026+0000 7f01c41dbb80 1 client.5268 shutdown 2023-10-14T10:37:06.050 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.026+0000 7f01c41dbb80 1 -- 172.21.15.67:0/875447513 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564ba86e2010 msgr2=0x564ba8706240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:06.051 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.026+0000 7f01c41dbb80 1 --2- 172.21.15.67:0/875447513 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564ba86e2010 0x564ba8706240 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f01a0009b90 tx=0x7f01a00047c0 comp rx=0 tx=0).stop 2023-10-14T10:37:06.051 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.026+0000 7f01c41dbb80 1 -- 172.21.15.67:0/875447513 shutdown_connections 2023-10-14T10:37:06.051 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.026+0000 7f01c41dbb80 1 --2- 172.21.15.67:0/875447513 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x564ba870c630 0x564ba8708870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:06.051 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.026+0000 7f01c41dbb80 1 --2- 172.21.15.67:0/875447513 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564ba86e2010 0x564ba8706240 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:06.051 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.026+0000 7f01c41dbb80 1 --2- 172.21.15.67:0/875447513 >> [v2:172.21.15.67:6833/3065963596,v1:172.21.15.67:6835/3065963596] conn(0x564ba87112e0 0x564ba87116c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:06.051 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.026+0000 7f01c41dbb80 1 --2- 172.21.15.67:0/875447513 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564ba8707220 0x564ba8708330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:06.052 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.026+0000 7f01c41dbb80 1 -- 172.21.15.67:0/875447513 >> 172.21.15.67:0/875447513 conn(0x564ba8611d80 msgr2=0x564ba872a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:37:06.052 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.026+0000 7f0190ff9700 0 client.5268 ms_handle_reset on v2:172.21.15.67:6833/3065963596 2023-10-14T10:37:06.052 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.026+0000 7f01c41dbb80 1 -- 172.21.15.67:0/875447513 shutdown_connections 2023-10-14T10:37:06.052 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.026+0000 7f01c41dbb80 1 -- 172.21.15.67:0/875447513 wait complete. 2023-10-14T10:37:06.052 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.026+0000 7f01c41dbb80 20 client.5268 trim_cache size 0 max 16384 2023-10-14T10:37:06.058 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.044+0000 7f01c41dbb80 1 Processor -- start 2023-10-14T10:37:06.058 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.044+0000 7f01c41dbb80 1 -- start start 2023-10-14T10:37:06.058 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.045+0000 7f01c41dbb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564ba86e2010 0x564ba8738140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:06.059 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.045+0000 7f01c41dbb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564ba8707220 0x564ba8738680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:06.059 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.045+0000 7f01c41dbb80 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x564ba870c630 0x564ba86dea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:06.059 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.045+0000 7f01c41dbb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ba84b78d0 con 0x564ba86e2010 2023-10-14T10:37:06.059 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.045+0000 7f01c41dbb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ba8706eb0 con 0x564ba8707220 2023-10-14T10:37:06.059 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.045+0000 7f01c41dbb80 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x564ba87061e0 con 0x564ba870c630 2023-10-14T10:37:06.060 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.045+0000 7f01b5d8d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564ba86e2010 0x564ba8738140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:06.060 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.045+0000 7f01b5d8d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564ba86e2010 0x564ba8738140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45916/0 (socket says 172.21.15.67:45916) 2023-10-14T10:37:06.060 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.045+0000 7f01b5d8d700 1 -- 172.21.15.67:0/3990861225 learned_addr learned my addr 172.21.15.67:0/3990861225 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:37:06.060 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.045+0000 7f01b5d8d700 1 -- 172.21.15.67:0/3990861225 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x564ba870c630 msgr2=0x564ba86dea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:37:06.060 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.045+0000 7f01b5d8d700 1 --2- 172.21.15.67:0/3990861225 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x564ba870c630 0x564ba86dea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:06.061 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.045+0000 7f01b558c700 1 --2- 172.21.15.67:0/3990861225 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564ba8707220 0x564ba8738680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:06.061 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.045+0000 7f01b5d8d700 1 -- 172.21.15.67:0/3990861225 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564ba8707220 msgr2=0x564ba8738680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:37:06.061 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.045+0000 7f01b5d8d700 1 --2- 172.21.15.67:0/3990861225 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564ba8707220 0x564ba8738680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:06.061 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.045+0000 7f01b5d8d700 1 -- 172.21.15.67:0/3990861225 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564ba8461d90 con 0x564ba86e2010 2023-10-14T10:37:06.061 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.045+0000 7f01b5d8d700 1 --2- 172.21.15.67:0/3990861225 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564ba86e2010 0x564ba8738140 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f01a00047f0 tx=0x7f01a0039000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80af2d4556cfb47c server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:06.061 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.046+0000 7f0193fff700 1 -- 172.21.15.67:0/3990861225 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a0026cf0 con 0x564ba86e2010 2023-10-14T10:37:06.062 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.046+0000 7f0193fff700 1 -- 172.21.15.67:0/3990861225 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01a0026e50 con 0x564ba86e2010 2023-10-14T10:37:06.062 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.046+0000 7f0193fff700 1 -- 172.21.15.67:0/3990861225 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a0025c60 con 0x564ba86e2010 2023-10-14T10:37:06.062 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.046+0000 7f01c41dbb80 1 -- 172.21.15.67:0/3990861225 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564ba86e2010 msgr2=0x564ba8738140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:06.062 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.046+0000 7f01c41dbb80 1 --2- 172.21.15.67:0/3990861225 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564ba86e2010 0x564ba8738140 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f01a00047f0 tx=0x7f01a0039000 comp rx=0 tx=0).stop 2023-10-14T10:37:06.062 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.046+0000 7f01c41dbb80 1 -- 172.21.15.67:0/3990861225 shutdown_connections 2023-10-14T10:37:06.063 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.046+0000 7f01c41dbb80 1 --2- 172.21.15.67:0/3990861225 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x564ba870c630 0x564ba86dea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:06.063 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.046+0000 7f01c41dbb80 1 --2- 172.21.15.67:0/3990861225 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564ba86e2010 0x564ba8738140 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:06.063 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.046+0000 7f01c41dbb80 1 --2- 172.21.15.67:0/3990861225 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564ba8707220 0x564ba8738680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:06.063 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.046+0000 7f01c41dbb80 1 -- 172.21.15.67:0/3990861225 >> 172.21.15.67:0/3990861225 conn(0x564ba8611d80 msgr2=0x564ba86d90f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:37:06.063 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.046+0000 7f01c41dbb80 1 -- 172.21.15.67:0/3990861225 shutdown_connections 2023-10-14T10:37:06.064 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.046+0000 7f01c41dbb80 1 -- 172.21.15.67:0/3990861225 wait complete. 2023-10-14T10:37:06.064 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.047+0000 7f01c41dbb80 1 -- start start 2023-10-14T10:37:06.064 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.047+0000 7f01c41dbb80 1 Processor -- start 2023-10-14T10:37:06.064 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.048+0000 7f01c41dbb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564ba86e2010 0x564ba8747650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:06.064 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.048+0000 7f01c41dbb80 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x564ba8707220 0x564ba8742850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:06.064 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.048+0000 7f01c41dbb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564ba870c630 0x564ba8738140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:06.065 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.048+0000 7f01c41dbb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ba8710d90 con 0x564ba870c630 2023-10-14T10:37:06.065 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.048+0000 7f01c41dbb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ba8747b90 con 0x564ba86e2010 2023-10-14T10:37:06.065 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.048+0000 7f01c41dbb80 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x564ba8747ff0 con 0x564ba8707220 2023-10-14T10:37:06.065 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.048+0000 7f01c41dbb80 10 client.0 _ll_register_callbacks cb 0x7f01a8001d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-14T10:37:06.065 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.048+0000 7f01b658e700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564ba870c630 0x564ba8738140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:06.065 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.048+0000 7f01b658e700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564ba870c630 0x564ba8738140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45926/0 (socket says 172.21.15.67:45926) 2023-10-14T10:37:06.066 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.048+0000 7f01b658e700 1 -- 172.21.15.67:0/19810171 learned_addr learned my addr 172.21.15.67:0/19810171 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:37:06.066 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.048+0000 7f01b558c700 1 --2- 172.21.15.67:0/19810171 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x564ba8707220 0x564ba8742850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:06.066 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.048+0000 7f01b5d8d700 1 --2- 172.21.15.67:0/19810171 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564ba86e2010 0x564ba8747650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:06.066 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.048+0000 7f01b658e700 1 -- 172.21.15.67:0/19810171 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x564ba8707220 msgr2=0x564ba8742850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:37:06.067 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.048+0000 7f01b658e700 1 --2- 172.21.15.67:0/19810171 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x564ba8707220 0x564ba8742850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:06.067 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.048+0000 7f01b658e700 1 -- 172.21.15.67:0/19810171 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564ba86e2010 msgr2=0x564ba8747650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:37:06.067 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.048+0000 7f01b658e700 1 --2- 172.21.15.67:0/19810171 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564ba86e2010 0x564ba8747650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:06.067 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.048+0000 7f01b658e700 1 -- 172.21.15.67:0/19810171 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01b00b2e90 con 0x564ba870c630 2023-10-14T10:37:06.067 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.048+0000 7f01b5d8d700 1 --2- 172.21.15.67:0/19810171 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564ba86e2010 0x564ba8747650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-14T10:37:06.068 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.048+0000 7f01b558c700 1 --2- 172.21.15.67:0/19810171 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x564ba8707220 0x564ba8742850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-14T10:37:06.068 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.048+0000 7f01b658e700 1 --2- 172.21.15.67:0/19810171 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564ba870c630 0x564ba8738140 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f01a80012e0 tx=0x7f01a8014620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed928e412b2c279d server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:06.068 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.048+0000 7f0190ff9700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2023-10-14T10:37:06.068 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.048+0000 7f0190ff9700 1 -- 172.21.15.67:0/19810171 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a8014dd0 con 0x564ba870c630 2023-10-14T10:37:06.068 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.049+0000 7f0190ff9700 1 -- 172.21.15.67:0/19810171 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(33..33 src has 1..33) v4 ==== 5729+0+0 (secure 0 0 0) 0x7f01a801a840 con 0x564ba870c630 2023-10-14T10:37:06.068 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.049+0000 7f01c41dbb80 1 -- 172.21.15.67:0/19810171 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f01a002b040 con 0x564ba870c630 2023-10-14T10:37:06.069 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.049+0000 7f0190ff9700 1 -- 172.21.15.67:0/19810171 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f01a801aa00 con 0x564ba870c630 2023-10-14T10:37:06.069 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.049+0000 7f01c41dbb80 10 client.5274 fetch_fsmap learned FSMap version 16 2023-10-14T10:37:06.069 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.049+0000 7f01c41dbb80 1 -- 172.21.15.67:0/19810171 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x564ba844ac10 con 0x564ba870c630 2023-10-14T10:37:06.069 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.049+0000 7f0190ff9700 1 -- 172.21.15.67:0/19810171 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01a002b040 con 0x564ba870c630 2023-10-14T10:37:06.069 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.049+0000 7f0190ff9700 1 -- 172.21.15.67:0/19810171 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 16) v1 ==== 3299+0+0 (secure 0 0 0) 0x7f01a8012050 con 0x564ba870c630 2023-10-14T10:37:06.070 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.049+0000 7f01c41dbb80 10 client.5274 fetch_fsmap finished waiting for FSMap version 16 2023-10-14T10:37:06.070 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.049+0000 7f01c41dbb80 10 client.5274 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-14T10:37:06.070 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.049+0000 7f01c41dbb80 20 client.5274 populate_metadata read hostname 'smithi067' 2023-10-14T10:37:06.070 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.049+0000 7f01c41dbb80 1 --2- 172.21.15.67:0/19810171 >> [v2:172.21.15.67:6833/3065963596,v1:172.21.15.67:6835/3065963596] conn(0x564ba87929b0 0x564ba8738680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:06.070 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.049+0000 7f01c41dbb80 4 client.5274 mds_command: new command op to 5097 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-14T10:37:06.071 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.049+0000 7f01c41dbb80 1 -- 172.21.15.67:0/19810171 --> [v2:172.21.15.67:6833/3065963596,v1:172.21.15.67:6835/3065963596] -- command(tid 0: {"prefix": "status"}) v1 -- 0x564ba862fb20 con 0x564ba87929b0 2023-10-14T10:37:06.071 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.049+0000 7f01b5d8d700 1 --2- 172.21.15.67:0/19810171 >> [v2:172.21.15.67:6833/3065963596,v1:172.21.15.67:6835/3065963596] conn(0x564ba87929b0 0x564ba8738680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:06.071 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.049+0000 7f01b5d8d700 1 --2- 172.21.15.67:0/19810171 >> [v2:172.21.15.67:6833/3065963596,v1:172.21.15.67:6835/3065963596] conn(0x564ba87929b0 0x564ba8738680 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4b53b4f79c22a9 server_cookie=54a768c2a3012d27 in_seq=0 out_seq=0 2023-10-14T10:37:06.071 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.049+0000 7f0190ff9700 10 client.5274 ms_handle_connect on v2:172.21.15.67:6833/3065963596 2023-10-14T10:37:06.071 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.050+0000 7f0190ff9700 1 -- 172.21.15.67:0/19810171 <== mds.0 v2:172.21.15.67:6833/3065963596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x564ba862fb20 con 0x564ba87929b0 2023-10-14T10:37:06.071 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.050+0000 7f0190ff9700 10 client.5274 handle_command_reply: tid=0 2023-10-14T10:37:06.072 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.050+0000 7f01c41dbb80 1 client.5274 shutdown 2023-10-14T10:37:06.072 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.050+0000 7f01c41dbb80 1 -- 172.21.15.67:0/19810171 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564ba870c630 msgr2=0x564ba8738140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:06.072 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.050+0000 7f01c41dbb80 1 --2- 172.21.15.67:0/19810171 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564ba870c630 0x564ba8738140 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f01a80012e0 tx=0x7f01a8014620 comp rx=0 tx=0).stop 2023-10-14T10:37:06.072 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.050+0000 7f01c41dbb80 1 -- 172.21.15.67:0/19810171 shutdown_connections 2023-10-14T10:37:06.073 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.050+0000 7f01c41dbb80 1 --2- 172.21.15.67:0/19810171 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564ba870c630 0x564ba8738140 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:06.073 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.050+0000 7f01c41dbb80 1 --2- 172.21.15.67:0/19810171 >> [v2:172.21.15.67:6833/3065963596,v1:172.21.15.67:6835/3065963596] conn(0x564ba87929b0 0x564ba8738680 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:06.073 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.050+0000 7f01c41dbb80 1 --2- 172.21.15.67:0/19810171 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564ba86e2010 0x564ba8747650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:06.073 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.050+0000 7f01c41dbb80 1 --2- 172.21.15.67:0/19810171 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x564ba8707220 0x564ba8742850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:06.073 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.050+0000 7f0190ff9700 0 client.5274 ms_handle_reset on v2:172.21.15.67:6833/3065963596 2023-10-14T10:37:06.074 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.050+0000 7f01c41dbb80 1 -- 172.21.15.67:0/19810171 >> 172.21.15.67:0/19810171 conn(0x564ba8611d80 msgr2=0x564ba86d90f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:37:06.074 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.050+0000 7f01c41dbb80 1 -- 172.21.15.67:0/19810171 shutdown_connections 2023-10-14T10:37:06.074 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.050+0000 7f01c41dbb80 1 -- 172.21.15.67:0/19810171 wait complete. 2023-10-14T10:37:06.074 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.050+0000 7f01c41dbb80 20 client.5274 trim_cache size 0 max 16384 2023-10-14T10:37:06.075 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2023-10-14T10:37:06.075 INFO:teuthology.orchestra.run.smithi067.stdout: "cluster_fsid": "37f9ab46-3082-4b71-8b24-08d39897b008", 2023-10-14T10:37:06.075 INFO:teuthology.orchestra.run.smithi067.stdout: "whoami": 0, 2023-10-14T10:37:06.075 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 5097, 2023-10-14T10:37:06.076 INFO:teuthology.orchestra.run.smithi067.stdout: "want_state": "up:active", 2023-10-14T10:37:06.076 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "up:active", 2023-10-14T10:37:06.076 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "testcephfs1", 2023-10-14T10:37:06.076 INFO:teuthology.orchestra.run.smithi067.stdout: "rank_uptime": 5.9776482599999996, 2023-10-14T10:37:06.076 INFO:teuthology.orchestra.run.smithi067.stdout: "mdsmap_epoch": 16, 2023-10-14T10:37:06.077 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch": 33, 2023-10-14T10:37:06.077 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch_barrier": 33, 2023-10-14T10:37:06.077 INFO:teuthology.orchestra.run.smithi067.stdout: "uptime": 13.869935243 2023-10-14T10:37:06.077 INFO:teuthology.orchestra.run.smithi067.stdout:} 2023-10-14T10:37:06.079 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.050+0000 7f01b7ff1700 1 -- 172.21.15.67:0/2808007079 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f019406af00 msgr2=0x7f019406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:06.079 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.050+0000 7f01b7ff1700 1 --2- 172.21.15.67:0/2808007079 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f019406af00 0x7f019406d3c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f01a00090b0 tx=0x7f01a001c040 comp rx=0 tx=0).stop 2023-10-14T10:37:06.079 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.050+0000 7f01b7ff1700 1 -- 172.21.15.67:0/2808007079 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01b00ce3b0 msgr2=0x7f01b015cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:06.079 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.050+0000 7f01b7ff1700 1 --2- 172.21.15.67:0/2808007079 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01b00ce3b0 0x7f01b015cfb0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f01ac00c4e0 tx=0x7f01ac00c7f0 comp rx=0 tx=0).stop 2023-10-14T10:37:06.080 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.051+0000 7f01b7ff1700 1 -- 172.21.15.67:0/2808007079 shutdown_connections 2023-10-14T10:37:06.080 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.051+0000 7f01b7ff1700 1 --2- 172.21.15.67:0/2808007079 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f019406af00 0x7f019406d3c0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:06.080 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.051+0000 7f01b7ff1700 1 --2- 172.21.15.67:0/2808007079 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f01b015b8b0 0x7f01b01e51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:06.080 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.051+0000 7f01b7ff1700 1 --2- 172.21.15.67:0/2808007079 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f01b00b4640 0x7f01b0150ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:06.080 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.051+0000 7f01b7ff1700 1 --2- 172.21.15.67:0/2808007079 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01b00ce3b0 0x7f01b015cfb0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:06.081 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.051+0000 7f01b7ff1700 1 -- 172.21.15.67:0/2808007079 >> 172.21.15.67:0/2808007079 conn(0x7f01b006dac0 msgr2=0x7f01b00b0940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:37:06.081 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.051+0000 7f01b7ff1700 1 -- 172.21.15.67:0/2808007079 shutdown_connections 2023-10-14T10:37:06.081 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.051+0000 7f01b7ff1700 1 -- 172.21.15.67:0/2808007079 wait complete. 2023-10-14T10:37:06.081 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-14T10:37:06.207 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.205+0000 7f1d28b91700 1 Processor -- start 2023-10-14T10:37:06.208 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.205+0000 7f1d28b91700 1 -- start start 2023-10-14T10:37:06.210 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.206+0000 7f1d28b91700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f1d2415f130 0x7f1d240cede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:06.210 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.206+0000 7f1d28b91700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f1d2406af50 0x7f1d240b3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:06.210 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.206+0000 7f1d28b91700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d240b3530 con 0x7f1d240b28b0 2023-10-14T10:37:06.211 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.206+0000 7f1d28b91700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d240cde00 con 0x7f1d2415f130 2023-10-14T10:37:06.211 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.206+0000 7f1d28b91700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d240cd180 con 0x7f1d2406af50 2023-10-14T10:37:06.211 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.206+0000 7f1d2259c700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f1d2415f130 0x7f1d240cede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:06.211 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.206+0000 7f1d21d9b700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f1d240b28b0 0x7f1d2406a840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:53698/0 (socket says 172.21.15.67:53698) 2023-10-14T10:37:06.211 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.206+0000 7f1d2259c700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f1d2415f130 0x7f1d240cede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45930/0 (socket says 172.21.15.67:45930) 2023-10-14T10:37:06.212 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.206+0000 7f1d21d9b700 1 -- 172.21.15.67:0/641371904 learned_addr learned my addr 172.21.15.67:0/641371904 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:37:06.212 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.207+0000 7f1d22d9d700 1 --2- 172.21.15.67:0/641371904 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f1d2406af50 0x7f1d240b3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:06.212 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.207+0000 7f1d2259c700 1 -- 172.21.15.67:0/641371904 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f1d2406af50 msgr2=0x7f1d240b3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:37:06.212 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.207+0000 7f1d2259c700 1 --2- 172.21.15.67:0/641371904 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f1d2406af50 0x7f1d240b3960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:06.212 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.207+0000 7f1d2259c700 1 -- 172.21.15.67:0/641371904 >> v1:172.21.15.67:6789/0 conn(0x7f1d240b28b0 legacy=0x7f1d2406a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:06.213 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.207+0000 7f1d2259c700 1 -- 172.21.15.67:0/641371904 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d240cd460 con 0x7f1d2415f130 2023-10-14T10:37:06.213 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.207+0000 7f1d2259c700 1 --2- 172.21.15.67:0/641371904 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1d2415f130 0x7f1d240cede0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f1d18009840 tx=0x7f1d18009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73faac284c903c6b server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:06.213 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.207+0000 7f1d2159a700 1 -- 172.21.15.67:0/641371904 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d18011830 con 0x7f1d2415f130 2023-10-14T10:37:06.213 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.207+0000 7f1d2159a700 1 -- 172.21.15.67:0/641371904 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d18011990 con 0x7f1d2415f130 2023-10-14T10:37:06.213 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.207+0000 7f1d2159a700 1 -- 172.21.15.67:0/641371904 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d18011d20 con 0x7f1d2415f130 2023-10-14T10:37:06.214 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.208+0000 7f1d28b91700 1 -- 172.21.15.67:0/641371904 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1d2415f130 msgr2=0x7f1d240cede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:06.214 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.208+0000 7f1d28b91700 1 --2- 172.21.15.67:0/641371904 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1d2415f130 0x7f1d240cede0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f1d18009840 tx=0x7f1d18009b50 comp rx=0 tx=0).stop 2023-10-14T10:37:06.214 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.208+0000 7f1d28b91700 1 -- 172.21.15.67:0/641371904 shutdown_connections 2023-10-14T10:37:06.214 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.208+0000 7f1d28b91700 1 --2- 172.21.15.67:0/641371904 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f1d2406af50 0x7f1d240b3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:06.214 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.208+0000 7f1d28b91700 1 --2- 172.21.15.67:0/641371904 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1d2415f130 0x7f1d240cede0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:06.215 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.208+0000 7f1d28b91700 1 -- 172.21.15.67:0/641371904 >> 172.21.15.67:0/641371904 conn(0x7f1d2406dae0 msgr2=0x7f1d2406def0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:37:06.215 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.208+0000 7f1d28b91700 1 -- 172.21.15.67:0/641371904 shutdown_connections 2023-10-14T10:37:06.215 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.208+0000 7f1d28b91700 1 -- 172.21.15.67:0/641371904 wait complete. 2023-10-14T10:37:06.218 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.209+0000 7f1d28b91700 1 Processor -- start 2023-10-14T10:37:06.218 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.209+0000 7f1d28b91700 1 -- start start 2023-10-14T10:37:06.218 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.210+0000 7f1d28b91700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f1d2406af50 0x7f1d240c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:06.218 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.210+0000 7f1d28b91700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1d240b28b0 0x7f1d241f2be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:06.219 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.210+0000 7f1d28b91700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1d2415f130 0x7f1d240cbb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:06.219 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.210+0000 7f1d28b91700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d240c8f30 con 0x7f1d240b28b0 2023-10-14T10:37:06.219 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.210+0000 7f1d28b91700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d240c8320 con 0x7f1d2415f130 2023-10-14T10:37:06.219 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.210+0000 7f1d28b91700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d240c8600 con 0x7f1d2406af50 2023-10-14T10:37:06.220 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.210+0000 7f1d21d9b700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1d240b28b0 0x7f1d241f2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:06.220 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.210+0000 7f1d21d9b700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1d240b28b0 0x7f1d241f2be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45936/0 (socket says 172.21.15.67:45936) 2023-10-14T10:37:06.220 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.210+0000 7f1d21d9b700 1 -- 172.21.15.67:0/1363931275 learned_addr learned my addr 172.21.15.67:0/1363931275 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:37:06.220 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.210+0000 7f1d22d9d700 1 --2- 172.21.15.67:0/1363931275 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1d2415f130 0x7f1d240cbb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:06.220 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.210+0000 7f1d2259c700 1 --2- 172.21.15.67:0/1363931275 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f1d2406af50 0x7f1d240c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:06.221 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.210+0000 7f1d21d9b700 1 -- 172.21.15.67:0/1363931275 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f1d2406af50 msgr2=0x7f1d240c93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:06.221 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.210+0000 7f1d21d9b700 1 --2- 172.21.15.67:0/1363931275 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f1d2406af50 0x7f1d240c93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:06.221 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.210+0000 7f1d21d9b700 1 -- 172.21.15.67:0/1363931275 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1d2415f130 msgr2=0x7f1d240cbb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:06.221 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.210+0000 7f1d21d9b700 1 --2- 172.21.15.67:0/1363931275 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1d2415f130 0x7f1d240cbb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:06.221 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.210+0000 7f1d21d9b700 1 -- 172.21.15.67:0/1363931275 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d180094a0 con 0x7f1d240b28b0 2023-10-14T10:37:06.222 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.210+0000 7f1d21d9b700 1 --2- 172.21.15.67:0/1363931275 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1d240b28b0 0x7f1d241f2be0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f1d1400c4e0 tx=0x7f1d1400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:06.222 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.211+0000 7f1d12ffd700 1 -- 172.21.15.67:0/1363931275 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d140074e0 con 0x7f1d240b28b0 2023-10-14T10:37:06.222 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.211+0000 7f1d12ffd700 1 -- 172.21.15.67:0/1363931275 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d14007640 con 0x7f1d240b28b0 2023-10-14T10:37:06.222 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.211+0000 7f1d28b91700 1 -- 172.21.15.67:0/1363931275 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d240c88e0 con 0x7f1d240b28b0 2023-10-14T10:37:06.222 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.211+0000 7f1d12ffd700 1 -- 172.21.15.67:0/1363931275 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d14005a50 con 0x7f1d240b28b0 2023-10-14T10:37:06.223 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.211+0000 7f1d28b91700 1 -- 172.21.15.67:0/1363931275 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d241f3120 con 0x7f1d240b28b0 2023-10-14T10:37:06.223 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.212+0000 7f1d28b91700 1 -- 172.21.15.67:0/1363931275 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d24067d20 con 0x7f1d240b28b0 2023-10-14T10:37:06.223 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.212+0000 7f1d12ffd700 1 -- 172.21.15.67:0/1363931275 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f1d14005bb0 con 0x7f1d240b28b0 2023-10-14T10:37:06.223 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.213+0000 7f1d12ffd700 1 --2- 172.21.15.67:0/1363931275 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f1d0406af00 0x7f1d0406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:06.223 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.213+0000 7f1d12ffd700 1 -- 172.21.15.67:0/1363931275 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5729+0+0 (secure 0 0 0) 0x7f1d14088b10 con 0x7f1d240b28b0 2023-10-14T10:37:06.224 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.213+0000 7f1d2259c700 1 --2- 172.21.15.67:0/1363931275 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f1d0406af00 0x7f1d0406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:06.224 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.213+0000 7f1d2259c700 1 --2- 172.21.15.67:0/1363931275 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f1d0406af00 0x7f1d0406d3c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1d180090b0 tx=0x7f1d18008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:06.224 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.217+0000 7f1d12ffd700 1 -- 172.21.15.67:0/1363931275 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1d14053ba0 con 0x7f1d240b28b0 2023-10-14T10:37:06.464 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.462+0000 7f1d28b91700 1 -- 172.21.15.67:0/1363931275 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f1d240d06f0 con 0x7f1d240b28b0 2023-10-14T10:37:06.465 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.462+0000 7f1d12ffd700 1 -- 172.21.15.67:0/1363931275 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+88 (secure 0 0 0) 0x7f1d14017020 con 0x7f1d240b28b0 2023-10-14T10:37:06.465 INFO:teuthology.orchestra.run.smithi067.stdout:name: testcephfs1, metadata pool: testcephfs1_metadata, data pools: [testcephfs1_data ] 2023-10-14T10:37:06.467 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.464+0000 7f1d28b91700 1 -- 172.21.15.67:0/1363931275 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f1d0406af00 msgr2=0x7f1d0406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:06.467 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.465+0000 7f1d28b91700 1 --2- 172.21.15.67:0/1363931275 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f1d0406af00 0x7f1d0406d3c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1d180090b0 tx=0x7f1d18008030 comp rx=0 tx=0).stop 2023-10-14T10:37:06.467 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.465+0000 7f1d28b91700 1 -- 172.21.15.67:0/1363931275 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1d240b28b0 msgr2=0x7f1d241f2be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:06.468 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.465+0000 7f1d28b91700 1 --2- 172.21.15.67:0/1363931275 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1d240b28b0 0x7f1d241f2be0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f1d1400c4e0 tx=0x7f1d1400c8a0 comp rx=0 tx=0).stop 2023-10-14T10:37:06.468 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.465+0000 7f1d28b91700 1 -- 172.21.15.67:0/1363931275 shutdown_connections 2023-10-14T10:37:06.468 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.465+0000 7f1d28b91700 1 --2- 172.21.15.67:0/1363931275 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f1d0406af00 0x7f1d0406d3c0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:06.468 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.465+0000 7f1d28b91700 1 --2- 172.21.15.67:0/1363931275 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1d2415f130 0x7f1d240cbb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:06.468 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.465+0000 7f1d28b91700 1 --2- 172.21.15.67:0/1363931275 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f1d2406af50 0x7f1d240c93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:06.469 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.465+0000 7f1d28b91700 1 --2- 172.21.15.67:0/1363931275 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1d240b28b0 0x7f1d241f2be0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:06.469 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.465+0000 7f1d28b91700 1 -- 172.21.15.67:0/1363931275 >> 172.21.15.67:0/1363931275 conn(0x7f1d2406dae0 msgr2=0x7f1d240b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:37:06.469 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.465+0000 7f1d28b91700 1 -- 172.21.15.67:0/1363931275 shutdown_connections 2023-10-14T10:37:06.469 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:06.465+0000 7f1d28b91700 1 -- 172.21.15.67:0/1363931275 wait complete. 2023-10-14T10:37:06.479 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-14T10:37:06.479 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-14T10:37:06.480 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi067.front.sepia.ceph.com 2023-10-14T10:37:06.480 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-14T10:37:06.480 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-14T10:37:06.480 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-14T10:37:06.481 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-14T10:37:06.481 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-14T10:37:06.481 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-14T10:37:06.482 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:37:06.482 DEBUG:teuthology.orchestra.run.smithi067:> ip addr 2023-10-14T10:37:06.502 INFO:teuthology.orchestra.run.smithi067.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-14T10:37:06.502 INFO:teuthology.orchestra.run.smithi067.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-14T10:37:06.502 INFO:teuthology.orchestra.run.smithi067.stdout: inet 127.0.0.1/8 scope host lo 2023-10-14T10:37:06.502 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2023-10-14T10:37:06.503 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 ::1/128 scope host 2023-10-14T10:37:06.503 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2023-10-14T10:37:06.503 INFO:teuthology.orchestra.run.smithi067.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-14T10:37:06.503 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:a8:b4:7e brd ff:ff:ff:ff:ff:ff 2023-10-14T10:37:06.503 INFO:teuthology.orchestra.run.smithi067.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-14T10:37:06.504 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:a8:b4:7f brd ff:ff:ff:ff:ff:ff 2023-10-14T10:37:06.504 INFO:teuthology.orchestra.run.smithi067.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-14T10:37:06.504 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:b6:fc:0a brd ff:ff:ff:ff:ff:ff 2023-10-14T10:37:06.504 INFO:teuthology.orchestra.run.smithi067.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-14T10:37:06.504 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:b6:fc:0b brd ff:ff:ff:ff:ff:ff 2023-10-14T10:37:06.505 INFO:teuthology.orchestra.run.smithi067.stdout: inet 172.21.15.67/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-14T10:37:06.505 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft 42312sec preferred_lft 42312sec 2023-10-14T10:37:06.505 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 fe80::5cc4:6a4b:a14e:a6d7/64 scope link noprefixroute 2023-10-14T10:37:06.505 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2023-10-14T10:37:06.505 INFO:teuthology.orchestra.run.smithi067.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-14T10:37:06.506 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 1a:da:46:b2:01:15 brd ff:ff:ff:ff:ff:ff 2023-10-14T10:37:06.506 INFO:teuthology.orchestra.run.smithi067.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-14T10:37:06.506 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2023-10-14T10:37:06.506 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 fe80::c86:b0ff:fe46:c30c/64 scope link 2023-10-14T10:37:06.506 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2023-10-14T10:37:06.506 INFO:teuthology.orchestra.run.smithi067.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-14T10:37:06.507 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether d6:9f:d7:a3:00:d7 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-14T10:37:06.507 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 fe80::d49f:d7ff:fea3:d7/64 scope link 2023-10-14T10:37:06.507 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2023-10-14T10:37:06.507 INFO:teuthology.orchestra.run.smithi067.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-14T10:37:06.507 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 1a:da:46:b2:01:15 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-14T10:37:06.508 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 fe80::18da:46ff:feb2:115/64 scope link 2023-10-14T10:37:06.508 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2023-10-14T10:37:06.508 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:37:06.509 DEBUG:teuthology.orchestra.run.smithi067:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-14T10:37:06.567 INFO:teuthology.orchestra.run.smithi067.stdout:1 2023-10-14T10:37:06.569 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:37:06.569 DEBUG:teuthology.orchestra.run.smithi067:> route 2023-10-14T10:37:06.628 INFO:teuthology.orchestra.run.smithi067.stdout:Kernel IP routing table 2023-10-14T10:37:06.628 INFO:teuthology.orchestra.run.smithi067.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-14T10:37:06.628 INFO:teuthology.orchestra.run.smithi067.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-14T10:37:06.629 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-14T10:37:06.629 INFO:teuthology.orchestra.run.smithi067.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-14T10:37:06.629 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:37:06.630 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec bash -c ' 2023-10-14T10:37:06.630 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2023-10-14T10:37:06.630 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-14T10:37:06.631 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-14T10:37:06.631 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-14T10:37:06.631 DEBUG:teuthology.orchestra.run.smithi067:> ') 2023-10-14T10:37:06.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:37:06.734 DEBUG:teuthology.orchestra.run.smithi067:> ip netns list 2023-10-14T10:37:06.751 INFO:teuthology.orchestra.run.smithi067.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-14T10:37:06.752 INFO:teuthology.orchestra.run.smithi067.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-14T10:37:06.752 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-14T10:37:06.752 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:37:06.753 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link set brx.0 up 2023-10-14T10:37:06.819 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-14T10:37:06.820 DEBUG:teuthology.orchestra.run.smithi067:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-14T10:37:06.876 INFO:teuthology.orchestra.run.smithi067.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-14T10:37:06.876 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-14T10:37:06.877 DEBUG:teuthology.orchestra.run.smithi067:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-14T10:37:06.932 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-14T10:37:06.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:37:06.933 DEBUG:teuthology.orchestra.run.smithi067:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-14T10:37:07.019 INFO:teuthology.orchestra.run.smithi067.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-14T10:37:07.019 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-14T10:37:07.043 INFO:teuthology.orchestra.run.smithi067.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-14T10:37:07.043 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-14T10:37:07.044 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: resolved to: "172.21.15.67:3300,172.21.15.72:3300,172.21.15.72:3301" 2023-10-14T10:37:07.044 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: trying mount with new device syntax: 0@37f9ab46-3082-4b71-8b24-08d39897b008.cephfs=/ 2023-10-14T10:37:07.044 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.67:3300/172.21.15.72:3300/172.21.15.72:3301" will pass to kernel 2023-10-14T10:37:07.044 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: v2 mount support check returned 2 2023-10-14T10:37:07.044 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-14T10:37:07.044 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-14T10:37:07.099 INFO:teuthology.orchestra.run.smithi067.stderr:mount error 2 = No such file or directory 2023-10-14T10:37:07.100 INFO:teuthology.orchestra.run.smithi067.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-14T10:37:07.100 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-14T10:37:07.100 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: resolved to: "172.21.15.67:3300,172.21.15.72:3300,172.21.15.72:3301" 2023-10-14T10:37:07.101 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: trying mount with new device syntax: 0@37f9ab46-3082-4b71-8b24-08d39897b008.cephfs=/ 2023-10-14T10:37:07.101 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.67:3300/172.21.15.72:3300/172.21.15.72:3301" will pass to kernel 2023-10-14T10:37:07.101 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: v2 mount support check returned 2 2023-10-14T10:37:07.101 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-14T10:37:07.101 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-14T10:37:07.102 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: resolved to: "172.21.15.67:6789,172.21.15.72:6789,172.21.15.72:6790" 2023-10-14T10:37:07.102 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.67:6789,172.21.15.72:6789,172.21.15.72:6790:/ 2023-10-14T10:37:07.102 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=37f9ab46-3082-4b71-8b24-08d39897b008" will pass to kernel 2023-10-14T10:37:07.103 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-14T10:37:07.103 INFO:tasks.cephfs.kernel_mount:mount command failed 2023-10-14T10:37:07.104 INFO:tasks.cephfs_test_runner:That the volume can be created and then cleans up ... ERROR 2023-10-14T10:37:07.105 INFO:tasks.cephfs_test_runner: 2023-10-14T10:37:07.105 INFO:tasks.cephfs_test_runner:====================================================================== 2023-10-14T10:37:07.105 INFO:tasks.cephfs_test_runner:ERROR: test_volume_create (tasks.cephfs.test_volumes.TestVolumes) 2023-10-14T10:37:07.106 INFO:tasks.cephfs_test_runner:That the volume can be created and then cleans up 2023-10-14T10:37:07.106 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-10-14T10:37:07.106 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-10-14T10:37:07.106 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/qa/tasks/cephfs/test_volumes.py", line 415, in setUp 2023-10-14T10:37:07.107 INFO:tasks.cephfs_test_runner: super(TestVolumesHelper, self).setUp() 2023-10-14T10:37:07.107 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/qa/tasks/cephfs/cephfs_test_case.py", line 184, in setUp 2023-10-14T10:37:07.107 INFO:tasks.cephfs_test_runner: self.mounts[i].mount_wait() 2023-10-14T10:37:07.107 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/qa/tasks/cephfs/mount.py", line 505, in mount_wait 2023-10-14T10:37:07.108 INFO:tasks.cephfs_test_runner: self.mount(**kwargs) 2023-10-14T10:37:07.108 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/qa/tasks/cephfs/kernel_mount.py", line 59, in mount 2023-10-14T10:37:07.108 INFO:tasks.cephfs_test_runner: retval = self._run_mount_cmd(mntopts, check_status) 2023-10-14T10:37:07.108 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/qa/tasks/cephfs/kernel_mount.py", line 83, in _run_mount_cmd 2023-10-14T10:37:07.109 INFO:tasks.cephfs_test_runner: self.client_remote.run(args=mount_cmd, timeout=300, 2023-10-14T10:37:07.109 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/remote.py", line 522, in run 2023-10-14T10:37:07.109 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2023-10-14T10:37:07.109 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 455, in run 2023-10-14T10:37:07.110 INFO:tasks.cephfs_test_runner: r.wait() 2023-10-14T10:37:07.110 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 161, in wait 2023-10-14T10:37:07.110 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2023-10-14T10:37:07.110 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 181, in _raise_for_status 2023-10-14T10:37:07.111 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2023-10-14T10:37:07.111 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi067 with status 32: 'sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback' 2023-10-14T10:37:07.111 INFO:tasks.cephfs_test_runner: 2023-10-14T10:37:07.111 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-10-14T10:37:07.112 INFO:tasks.cephfs_test_runner:Ran 1 test in 33.659s 2023-10-14T10:37:07.112 INFO:tasks.cephfs_test_runner: 2023-10-14T10:37:07.112 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2023-10-14T10:37:07.113 INFO:tasks.cephfs_test_runner: 2023-10-14T10:37:07.113 INFO:tasks.cephfs_test_runner:====================================================================== 2023-10-14T10:37:07.113 INFO:tasks.cephfs_test_runner:ERROR: test_volume_create (tasks.cephfs.test_volumes.TestVolumes) 2023-10-14T10:37:07.113 INFO:tasks.cephfs_test_runner:That the volume can be created and then cleans up 2023-10-14T10:37:07.114 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-10-14T10:37:07.114 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-10-14T10:37:07.114 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/qa/tasks/cephfs/test_volumes.py", line 415, in setUp 2023-10-14T10:37:07.114 INFO:tasks.cephfs_test_runner: super(TestVolumesHelper, self).setUp() 2023-10-14T10:37:07.115 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/qa/tasks/cephfs/cephfs_test_case.py", line 184, in setUp 2023-10-14T10:37:07.115 INFO:tasks.cephfs_test_runner: self.mounts[i].mount_wait() 2023-10-14T10:37:07.115 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/qa/tasks/cephfs/mount.py", line 505, in mount_wait 2023-10-14T10:37:07.115 INFO:tasks.cephfs_test_runner: self.mount(**kwargs) 2023-10-14T10:37:07.117 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/qa/tasks/cephfs/kernel_mount.py", line 59, in mount 2023-10-14T10:37:07.117 INFO:tasks.cephfs_test_runner: retval = self._run_mount_cmd(mntopts, check_status) 2023-10-14T10:37:07.118 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/qa/tasks/cephfs/kernel_mount.py", line 83, in _run_mount_cmd 2023-10-14T10:37:07.118 INFO:tasks.cephfs_test_runner: self.client_remote.run(args=mount_cmd, timeout=300, 2023-10-14T10:37:07.118 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/remote.py", line 522, in run 2023-10-14T10:37:07.118 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2023-10-14T10:37:07.119 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 455, in run 2023-10-14T10:37:07.119 INFO:tasks.cephfs_test_runner: r.wait() 2023-10-14T10:37:07.119 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 161, in wait 2023-10-14T10:37:07.119 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2023-10-14T10:37:07.120 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 181, in _raise_for_status 2023-10-14T10:37:07.120 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2023-10-14T10:37:07.120 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi067 with status 32: 'sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback' 2023-10-14T10:37:07.120 INFO:tasks.cephfs_test_runner: 2023-10-14T10:37:07.121 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_volume_create (tasks.cephfs.test_volumes.TestVolumes) 2023-10-14T10:37:07.186 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=8f19c712116b4ded8e6968d8f7ed2de7 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_volume_create (tasks.cephfs.test_volumes.TestVolumes) 2023-10-14T10:37:07.188 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-10-14T10:37:07.202 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-10-14T10:37:07.210 INFO:tasks.kclient:Unmounting kernel clients... 2023-10-14T10:37:07.211 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2023-10-14T10:37:07.211 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-14T10:37:07.232 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284288k,nr_inodes=4071072,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=43,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20514 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260556k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-10-14T10:37:07.233 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2023-10-14T10:37:07.233 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-14T10:37:07.290 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284288k,nr_inodes=4071072,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=43,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20514 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260556k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-10-14T10:37:07.291 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-10-14T10:37:07.291 DEBUG:teuthology.orchestra.run.smithi072:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-14T10:37:07.312 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284104k,nr_inodes=4071026,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=40,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=12761 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260520k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-10-14T10:37:07.313 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-10-14T10:37:07.313 DEBUG:teuthology.orchestra.run.smithi072:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-14T10:37:07.373 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284104k,nr_inodes=4071026,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=40,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=12761 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260520k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-10-14T10:37:07.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:37:07.374 DEBUG:teuthology.orchestra.run.smithi067:> ip netns list 2023-10-14T10:37:07.391 INFO:teuthology.orchestra.run.smithi067.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-14T10:37:07.391 INFO:teuthology.orchestra.run.smithi067.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-14T10:37:07.392 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:37:07.392 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-14T10:37:07.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:37:07.468 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-14T10:37:07.562 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:37:07.563 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link delete ceph-brx 2023-10-14T10:37:07.639 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:37:07.640 DEBUG:teuthology.orchestra.run.smithi067:> ip netns list 2023-10-14T10:37:07.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:37:07.659 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link delete ceph-brx 2023-10-14T10:37:07.730 INFO:teuthology.orchestra.run.smithi067.stderr:Cannot find device "ceph-brx" 2023-10-14T10:37:07.731 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-14T10:37:07.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:37:07.732 DEBUG:teuthology.orchestra.run.smithi072:> ip netns list 2023-10-14T10:37:07.752 INFO:teuthology.orchestra.run.smithi072.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-14T10:37:07.753 INFO:teuthology.orchestra.run.smithi072.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-14T10:37:07.753 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:37:07.753 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-14T10:37:07.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:37:07.833 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-14T10:37:07.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:37:07.918 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip link delete ceph-brx 2023-10-14T10:37:08.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:37:08.013 DEBUG:teuthology.orchestra.run.smithi072:> ip netns list 2023-10-14T10:37:08.030 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-14T10:37:08.030 DEBUG:teuthology.orchestra.run.smithi072:> sudo ip link delete ceph-brx 2023-10-14T10:37:08.100 INFO:teuthology.orchestra.run.smithi072.stderr:Cannot find device "ceph-brx" 2023-10-14T10:37:08.101 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-14T10:37:08.102 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-10-14T10:37:08.111 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-14T10:37:08.112 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T10:37:08.241 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.238+0000 7f22194ad700 1 Processor -- start 2023-10-14T10:37:08.241 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.239+0000 7f22194ad700 1 -- start start 2023-10-14T10:37:08.243 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.239+0000 7f22194ad700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f22140d0c70 0x7f22140cdfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:08.244 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.239+0000 7f22194ad700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f22140c5110 0x7f22140ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:08.244 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.239+0000 7f22194ad700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22140d23e0 con 0x7f221415b740 2023-10-14T10:37:08.244 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.239+0000 7f22194ad700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f221415bb20 con 0x7f22140c5110 2023-10-14T10:37:08.244 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.239+0000 7f22194ad700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22140cf190 con 0x7f22140d0c70 2023-10-14T10:37:08.245 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.239+0000 7f22137fe700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f22140c5110 0x7f22140ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:08.245 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.240+0000 7f2212ffd700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f221415b740 0x7f22140d1cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:53712/0 (socket says 172.21.15.67:53712) 2023-10-14T10:37:08.245 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.240+0000 7f22137fe700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f22140c5110 0x7f22140ce510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45948/0 (socket says 172.21.15.67:45948) 2023-10-14T10:37:08.246 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.240+0000 7f2212ffd700 1 -- 172.21.15.67:0/1039779186 learned_addr learned my addr 172.21.15.67:0/1039779186 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:37:08.246 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.240+0000 7f22127fc700 1 --2- 172.21.15.67:0/1039779186 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f22140d0c70 0x7f22140cdfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:08.246 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.240+0000 7f22137fe700 1 -- 172.21.15.67:0/1039779186 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f22140d0c70 msgr2=0x7f22140cdfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:37:08.247 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.240+0000 7f22137fe700 1 --2- 172.21.15.67:0/1039779186 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f22140d0c70 0x7f22140cdfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:08.247 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.240+0000 7f22137fe700 1 -- 172.21.15.67:0/1039779186 >> v1:172.21.15.67:6789/0 conn(0x7f221415b740 legacy=0x7f22140d1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:08.247 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.240+0000 7f2211ffb700 1 -- 172.21.15.67:0/1039779186 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1936258433 0 0) 0x7f2204001110 con 0x7f221415b740 2023-10-14T10:37:08.247 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.240+0000 7f22137fe700 1 -- 172.21.15.67:0/1039779186 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22140cbd40 con 0x7f22140c5110 2023-10-14T10:37:08.248 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.240+0000 7f22137fe700 1 --2- 172.21.15.67:0/1039779186 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22140c5110 0x7f22140ce510 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f220c009d90 tx=0x7f220c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a0646523bcce7b2 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:08.248 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.240+0000 7f2211ffb700 1 -- 172.21.15.67:0/1039779186 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f220c00e070 con 0x7f22140c5110 2023-10-14T10:37:08.248 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.240+0000 7f2211ffb700 1 -- 172.21.15.67:0/1039779186 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f220c00bb20 con 0x7f22140c5110 2023-10-14T10:37:08.248 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.240+0000 7f2211ffb700 1 -- 172.21.15.67:0/1039779186 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f220c012430 con 0x7f22140c5110 2023-10-14T10:37:08.249 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.241+0000 7f22194ad700 1 -- 172.21.15.67:0/1039779186 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22140c5110 msgr2=0x7f22140ce510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:08.249 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.241+0000 7f22194ad700 1 --2- 172.21.15.67:0/1039779186 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22140c5110 0x7f22140ce510 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f220c009d90 tx=0x7f220c00b330 comp rx=0 tx=0).stop 2023-10-14T10:37:08.249 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.241+0000 7f22194ad700 1 -- 172.21.15.67:0/1039779186 shutdown_connections 2023-10-14T10:37:08.250 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.241+0000 7f22194ad700 1 --2- 172.21.15.67:0/1039779186 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22140c5110 0x7f22140ce510 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:08.250 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.241+0000 7f22194ad700 1 --2- 172.21.15.67:0/1039779186 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f22140d0c70 0x7f22140cdfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:08.250 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.241+0000 7f22194ad700 1 -- 172.21.15.67:0/1039779186 >> 172.21.15.67:0/1039779186 conn(0x7f22140665a0 msgr2=0x7f2214066980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:37:08.250 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.241+0000 7f22194ad700 1 -- 172.21.15.67:0/1039779186 shutdown_connections 2023-10-14T10:37:08.251 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.241+0000 7f22194ad700 1 -- 172.21.15.67:0/1039779186 wait complete. 2023-10-14T10:37:08.253 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.242+0000 7f22194ad700 1 Processor -- start 2023-10-14T10:37:08.254 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.242+0000 7f22194ad700 1 -- start start 2023-10-14T10:37:08.254 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.243+0000 7f22194ad700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f22140c5110 0x7f221415a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:08.254 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.243+0000 7f22194ad700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f22140d0c70 0x7f22140b4a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:08.255 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.243+0000 7f22194ad700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f221415b740 0x7f22140b6870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:08.255 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.243+0000 7f22194ad700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22140b57d0 con 0x7f221415b740 2023-10-14T10:37:08.255 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.243+0000 7f22194ad700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22140b5ab0 con 0x7f22140c5110 2023-10-14T10:37:08.256 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.243+0000 7f22194ad700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22140b5d90 con 0x7f22140d0c70 2023-10-14T10:37:08.256 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.243+0000 7f22137fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f221415b740 0x7f22140b6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:08.256 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.243+0000 7f22137fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f221415b740 0x7f22140b6870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45954/0 (socket says 172.21.15.67:45954) 2023-10-14T10:37:08.256 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.243+0000 7f22137fe700 1 -- 172.21.15.67:0/677718896 learned_addr learned my addr 172.21.15.67:0/677718896 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:37:08.257 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.243+0000 7f2212ffd700 1 --2- 172.21.15.67:0/677718896 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f22140c5110 0x7f221415a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:08.257 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.243+0000 7f22127fc700 1 --2- 172.21.15.67:0/677718896 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f22140d0c70 0x7f22140b4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:08.257 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.243+0000 7f22137fe700 1 -- 172.21.15.67:0/677718896 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f22140d0c70 msgr2=0x7f22140b4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:08.258 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.243+0000 7f22137fe700 1 --2- 172.21.15.67:0/677718896 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f22140d0c70 0x7f22140b4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:08.258 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.243+0000 7f22137fe700 1 -- 172.21.15.67:0/677718896 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f22140c5110 msgr2=0x7f221415a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:08.258 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.243+0000 7f22137fe700 1 --2- 172.21.15.67:0/677718896 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f22140c5110 0x7f221415a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:08.259 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.243+0000 7f22137fe700 1 -- 172.21.15.67:0/677718896 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f220c009a70 con 0x7f221415b740 2023-10-14T10:37:08.259 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.243+0000 7f22127fc700 1 --2- 172.21.15.67:0/677718896 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f22140d0c70 0x7f22140b4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-14T10:37:08.259 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.244+0000 7f22137fe700 1 --2- 172.21.15.67:0/677718896 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f221415b740 0x7f22140b6870 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f220c014040 tx=0x7f220c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:08.260 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.244+0000 7f220b7fe700 1 -- 172.21.15.67:0/677718896 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f220c012430 con 0x7f221415b740 2023-10-14T10:37:08.260 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.244+0000 7f220b7fe700 1 -- 172.21.15.67:0/677718896 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f220c020070 con 0x7f221415b740 2023-10-14T10:37:08.260 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.244+0000 7f220b7fe700 1 -- 172.21.15.67:0/677718896 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f220c004280 con 0x7f221415b740 2023-10-14T10:37:08.260 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.244+0000 7f22194ad700 1 -- 172.21.15.67:0/677718896 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22140b6070 con 0x7f221415b740 2023-10-14T10:37:08.261 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.244+0000 7f22194ad700 1 -- 172.21.15.67:0/677718896 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f221415a740 con 0x7f221415b740 2023-10-14T10:37:08.261 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.245+0000 7f22194ad700 1 -- 172.21.15.67:0/677718896 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22140b6590 con 0x7f221415b740 2023-10-14T10:37:08.261 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.245+0000 7f220b7fe700 1 -- 172.21.15.67:0/677718896 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f220c0043e0 con 0x7f221415b740 2023-10-14T10:37:08.262 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.246+0000 7f220b7fe700 1 --2- 172.21.15.67:0/677718896 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f21f406af00 0x7f21f406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:08.262 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.246+0000 7f220b7fe700 1 -- 172.21.15.67:0/677718896 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5729+0+0 (secure 0 0 0) 0x7f220c022070 con 0x7f221415b740 2023-10-14T10:37:08.262 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.246+0000 7f2212ffd700 1 --2- 172.21.15.67:0/677718896 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f21f406af00 0x7f21f406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:08.262 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.246+0000 7f2212ffd700 1 --2- 172.21.15.67:0/677718896 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f21f406af00 0x7f21f406d3c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2204000f60 tx=0x7f2204007c90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:08.263 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.250+0000 7f220b7fe700 1 -- 172.21.15.67:0/677718896 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f22140b6590 con 0x7f221415b740 2023-10-14T10:37:08.448 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.446+0000 7f22194ad700 1 -- 172.21.15.67:0/677718896 --> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f22140cc300 con 0x7f21f406af00 2023-10-14T10:37:08.457 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.455+0000 7f220b7fe700 1 -- 172.21.15.67:0/677718896 <== mgr.4100 v2:172.21.15.72:6832/703488382 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289547 (secure 0 0 0) 0x7f2204012380 con 0x7f21f406af00 2023-10-14T10:37:08.460 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-14T10:37:08.463 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.458+0000 7f22194ad700 1 -- 172.21.15.67:0/677718896 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f21f406af00 msgr2=0x7f21f406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:08.463 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.458+0000 7f22194ad700 1 --2- 172.21.15.67:0/677718896 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f21f406af00 0x7f21f406d3c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2204000f60 tx=0x7f2204007c90 comp rx=0 tx=0).stop 2023-10-14T10:37:08.464 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.458+0000 7f22194ad700 1 -- 172.21.15.67:0/677718896 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f221415b740 msgr2=0x7f22140b6870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:08.464 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.458+0000 7f22194ad700 1 --2- 172.21.15.67:0/677718896 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f221415b740 0x7f22140b6870 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f220c014040 tx=0x7f220c004ad0 comp rx=0 tx=0).stop 2023-10-14T10:37:08.464 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.458+0000 7f22194ad700 1 -- 172.21.15.67:0/677718896 shutdown_connections 2023-10-14T10:37:08.464 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.458+0000 7f22194ad700 1 --2- 172.21.15.67:0/677718896 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f21f406af00 0x7f21f406d3c0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:08.465 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.458+0000 7f22194ad700 1 --2- 172.21.15.67:0/677718896 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f221415b740 0x7f22140b6870 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:08.465 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.458+0000 7f22194ad700 1 --2- 172.21.15.67:0/677718896 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f22140c5110 0x7f221415a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:08.465 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.458+0000 7f22194ad700 1 --2- 172.21.15.67:0/677718896 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f22140d0c70 0x7f22140b4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:08.465 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.458+0000 7f22194ad700 1 -- 172.21.15.67:0/677718896 >> 172.21.15.67:0/677718896 conn(0x7f22140665a0 msgr2=0x7f22140c55f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:37:08.466 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.458+0000 7f22194ad700 1 -- 172.21.15.67:0/677718896 shutdown_connections 2023-10-14T10:37:08.466 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.458+0000 7f22194ad700 1 -- 172.21.15.67:0/677718896 wait complete. 2023-10-14T10:37:08.466 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2023-10-14T10:37:08.472 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":62,"stamp":"2023-10-14T10:37:07.770696+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220280,"kb_used_data":4104,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754440,"statfs":{"total":773094113280,"available":772868546560,"internally_reserved":0,"allocated":4202496,"data_stored":2322644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12742,"internal_metadata":221236794},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002093"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.787560+0000","last_change":"2023-10-14T10:37:00.075102+0000","last_active":"2023-10-14T10:37:01.787560+0000","last_peered":"2023-10-14T10:37:01.787560+0000","last_clean":"2023-10-14T10:37:01.787560+0000","last_became_active":"2023-10-14T10:37:00.074851+0000","last_became_peered":"2023-10-14T10:37:00.074851+0000","last_unstale":"2023-10-14T10:37:01.787560+0000","last_undegraded":"2023-10-14T10:37:01.787560+0000","last_fullsized":"2023-10-14T10:37:01.787560+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T21:39:35.587728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788402+0000","last_change":"2023-10-14T10:37:00.068871+0000","last_active":"2023-10-14T10:37:01.788402+0000","last_peered":"2023-10-14T10:37:01.788402+0000","last_clean":"2023-10-14T10:37:01.788402+0000","last_became_active":"2023-10-14T10:37:00.068586+0000","last_became_peered":"2023-10-14T10:37:00.068586+0000","last_unstale":"2023-10-14T10:37:01.788402+0000","last_undegraded":"2023-10-14T10:37:01.788402+0000","last_fullsized":"2023-10-14T10:37:01.788402+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T12:47:24.725885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.062215+0000","last_change":"2023-10-14T10:37:00.080988+0000","last_active":"2023-10-14T10:37:01.062215+0000","last_peered":"2023-10-14T10:37:01.062215+0000","last_clean":"2023-10-14T10:37:01.062215+0000","last_became_active":"2023-10-14T10:37:00.080476+0000","last_became_peered":"2023-10-14T10:37:00.080476+0000","last_unstale":"2023-10-14T10:37:01.062215+0000","last_undegraded":"2023-10-14T10:37:01.062215+0000","last_fullsized":"2023-10-14T10:37:01.062215+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T10:48:03.043383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.059874+0000","last_change":"2023-10-14T10:37:00.077598+0000","last_active":"2023-10-14T10:37:01.059874+0000","last_peered":"2023-10-14T10:37:01.059874+0000","last_clean":"2023-10-14T10:37:01.059874+0000","last_became_active":"2023-10-14T10:37:00.077322+0000","last_became_peered":"2023-10-14T10:37:00.077322+0000","last_unstale":"2023-10-14T10:37:01.059874+0000","last_undegraded":"2023-10-14T10:37:01.059874+0000","last_fullsized":"2023-10-14T10:37:01.059874+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:28:04.536948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788360+0000","last_change":"2023-10-14T10:37:00.068342+0000","last_active":"2023-10-14T10:37:01.788360+0000","last_peered":"2023-10-14T10:37:01.788360+0000","last_clean":"2023-10-14T10:37:01.788360+0000","last_became_active":"2023-10-14T10:37:00.068055+0000","last_became_peered":"2023-10-14T10:37:00.068055+0000","last_unstale":"2023-10-14T10:37:01.788360+0000","last_undegraded":"2023-10-14T10:37:01.788360+0000","last_fullsized":"2023-10-14T10:37:01.788360+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T12:52:16.372161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788917+0000","last_change":"2023-10-14T10:37:00.072629+0000","last_active":"2023-10-14T10:37:01.788917+0000","last_peered":"2023-10-14T10:37:01.788917+0000","last_clean":"2023-10-14T10:37:01.788917+0000","last_became_active":"2023-10-14T10:37:00.072345+0000","last_became_peered":"2023-10-14T10:37:00.072345+0000","last_unstale":"2023-10-14T10:37:01.788917+0000","last_undegraded":"2023-10-14T10:37:01.788917+0000","last_fullsized":"2023-10-14T10:37:01.788917+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T17:09:21.565688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.061635+0000","last_change":"2023-10-14T10:37:00.073019+0000","last_active":"2023-10-14T10:37:01.061635+0000","last_peered":"2023-10-14T10:37:01.061635+0000","last_clean":"2023-10-14T10:37:01.061635+0000","last_became_active":"2023-10-14T10:37:00.072368+0000","last_became_peered":"2023-10-14T10:37:00.072368+0000","last_unstale":"2023-10-14T10:37:01.061635+0000","last_undegraded":"2023-10-14T10:37:01.061635+0000","last_fullsized":"2023-10-14T10:37:01.061635+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:31:16.127752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.062210+0000","last_change":"2023-10-14T10:37:00.074091+0000","last_active":"2023-10-14T10:37:01.062210+0000","last_peered":"2023-10-14T10:37:01.062210+0000","last_clean":"2023-10-14T10:37:01.062210+0000","last_became_active":"2023-10-14T10:37:00.073792+0000","last_became_peered":"2023-10-14T10:37:00.073792+0000","last_unstale":"2023-10-14T10:37:01.062210+0000","last_undegraded":"2023-10-14T10:37:01.062210+0000","last_fullsized":"2023-10-14T10:37:01.062210+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T17:27:50.406084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.489588+0000","last_change":"2023-10-14T10:37:00.081745+0000","last_active":"2023-10-14T10:37:01.489588+0000","last_peered":"2023-10-14T10:37:01.489588+0000","last_clean":"2023-10-14T10:37:01.489588+0000","last_became_active":"2023-10-14T10:37:00.081451+0000","last_became_peered":"2023-10-14T10:37:00.081451+0000","last_unstale":"2023-10-14T10:37:01.489588+0000","last_undegraded":"2023-10-14T10:37:01.489588+0000","last_fullsized":"2023-10-14T10:37:01.489588+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:18:45.297555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.060294+0000","last_change":"2023-10-14T10:37:00.071138+0000","last_active":"2023-10-14T10:37:01.060294+0000","last_peered":"2023-10-14T10:37:01.060294+0000","last_clean":"2023-10-14T10:37:01.060294+0000","last_became_active":"2023-10-14T10:37:00.070667+0000","last_became_peered":"2023-10-14T10:37:00.070667+0000","last_unstale":"2023-10-14T10:37:01.060294+0000","last_undegraded":"2023-10-14T10:37:01.060294+0000","last_fullsized":"2023-10-14T10:37:01.060294+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:52:37.894720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.062228+0000","last_change":"2023-10-14T10:37:00.073322+0000","last_active":"2023-10-14T10:37:01.062228+0000","last_peered":"2023-10-14T10:37:01.062228+0000","last_clean":"2023-10-14T10:37:01.062228+0000","last_became_active":"2023-10-14T10:37:00.072639+0000","last_became_peered":"2023-10-14T10:37:00.072639+0000","last_unstale":"2023-10-14T10:37:01.062228+0000","last_undegraded":"2023-10-14T10:37:01.062228+0000","last_fullsized":"2023-10-14T10:37:01.062228+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T18:57:50.136605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.061858+0000","last_change":"2023-10-14T10:37:00.069681+0000","last_active":"2023-10-14T10:37:01.061858+0000","last_peered":"2023-10-14T10:37:01.061858+0000","last_clean":"2023-10-14T10:37:01.061858+0000","last_became_active":"2023-10-14T10:37:00.069373+0000","last_became_peered":"2023-10-14T10:37:00.069373+0000","last_unstale":"2023-10-14T10:37:01.061858+0000","last_undegraded":"2023-10-14T10:37:01.061858+0000","last_fullsized":"2023-10-14T10:37:01.061858+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:31:16.127752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788687+0000","last_change":"2023-10-14T10:37:00.074917+0000","last_active":"2023-10-14T10:37:01.788687+0000","last_peered":"2023-10-14T10:37:01.788687+0000","last_clean":"2023-10-14T10:37:01.788687+0000","last_became_active":"2023-10-14T10:37:00.074473+0000","last_became_peered":"2023-10-14T10:37:00.074473+0000","last_unstale":"2023-10-14T10:37:01.788687+0000","last_undegraded":"2023-10-14T10:37:01.788687+0000","last_fullsized":"2023-10-14T10:37:01.788687+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T20:20:02.187767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788602+0000","last_change":"2023-10-14T10:37:00.073986+0000","last_active":"2023-10-14T10:37:01.788602+0000","last_peered":"2023-10-14T10:37:01.788602+0000","last_clean":"2023-10-14T10:37:01.788602+0000","last_became_active":"2023-10-14T10:37:00.073651+0000","last_became_peered":"2023-10-14T10:37:00.073651+0000","last_unstale":"2023-10-14T10:37:01.788602+0000","last_undegraded":"2023-10-14T10:37:01.788602+0000","last_fullsized":"2023-10-14T10:37:01.788602+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T10:48:03.043383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.061624+0000","last_change":"2023-10-14T10:37:00.072277+0000","last_active":"2023-10-14T10:37:01.061624+0000","last_peered":"2023-10-14T10:37:01.061624+0000","last_clean":"2023-10-14T10:37:01.061624+0000","last_became_active":"2023-10-14T10:37:00.071999+0000","last_became_peered":"2023-10-14T10:37:00.071999+0000","last_unstale":"2023-10-14T10:37:01.061624+0000","last_undegraded":"2023-10-14T10:37:01.061624+0000","last_fullsized":"2023-10-14T10:37:01.061624+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:18:45.297555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.062458+0000","last_change":"2023-10-14T10:37:00.072492+0000","last_active":"2023-10-14T10:37:01.062458+0000","last_peered":"2023-10-14T10:37:01.062458+0000","last_clean":"2023-10-14T10:37:01.062458+0000","last_became_active":"2023-10-14T10:37:00.071509+0000","last_became_peered":"2023-10-14T10:37:00.071509+0000","last_unstale":"2023-10-14T10:37:01.062458+0000","last_undegraded":"2023-10-14T10:37:01.062458+0000","last_fullsized":"2023-10-14T10:37:01.062458+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:51:00.167498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.062381+0000","last_change":"2023-10-14T10:37:00.072597+0000","last_active":"2023-10-14T10:37:01.062381+0000","last_peered":"2023-10-14T10:37:01.062381+0000","last_clean":"2023-10-14T10:37:01.062381+0000","last_became_active":"2023-10-14T10:37:00.071946+0000","last_became_peered":"2023-10-14T10:37:00.071946+0000","last_unstale":"2023-10-14T10:37:01.062381+0000","last_undegraded":"2023-10-14T10:37:01.062381+0000","last_fullsized":"2023-10-14T10:37:01.062381+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:06:28.391046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788267+0000","last_change":"2023-10-14T10:37:00.080581+0000","last_active":"2023-10-14T10:37:01.788267+0000","last_peered":"2023-10-14T10:37:01.788267+0000","last_clean":"2023-10-14T10:37:01.788267+0000","last_became_active":"2023-10-14T10:37:00.080288+0000","last_became_peered":"2023-10-14T10:37:00.080288+0000","last_unstale":"2023-10-14T10:37:01.788267+0000","last_undegraded":"2023-10-14T10:37:01.788267+0000","last_fullsized":"2023-10-14T10:37:01.788267+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T18:57:50.136605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.787886+0000","last_change":"2023-10-14T10:37:00.080484+0000","last_active":"2023-10-14T10:37:01.787886+0000","last_peered":"2023-10-14T10:37:01.787886+0000","last_clean":"2023-10-14T10:37:01.787886+0000","last_became_active":"2023-10-14T10:37:00.080186+0000","last_became_peered":"2023-10-14T10:37:00.080186+0000","last_unstale":"2023-10-14T10:37:01.787886+0000","last_undegraded":"2023-10-14T10:37:01.787886+0000","last_fullsized":"2023-10-14T10:37:01.787886+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:17:54.730152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788501+0000","last_change":"2023-10-14T10:37:00.072056+0000","last_active":"2023-10-14T10:37:01.788501+0000","last_peered":"2023-10-14T10:37:01.788501+0000","last_clean":"2023-10-14T10:37:01.788501+0000","last_became_active":"2023-10-14T10:37:00.071796+0000","last_became_peered":"2023-10-14T10:37:00.071796+0000","last_unstale":"2023-10-14T10:37:01.788501+0000","last_undegraded":"2023-10-14T10:37:01.788501+0000","last_fullsized":"2023-10-14T10:37:01.788501+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T20:39:07.030985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.489206+0000","last_change":"2023-10-14T10:37:00.080796+0000","last_active":"2023-10-14T10:37:01.489206+0000","last_peered":"2023-10-14T10:37:01.489206+0000","last_clean":"2023-10-14T10:37:01.489206+0000","last_became_active":"2023-10-14T10:37:00.080526+0000","last_became_peered":"2023-10-14T10:37:00.080526+0000","last_unstale":"2023-10-14T10:37:01.489206+0000","last_undegraded":"2023-10-14T10:37:01.489206+0000","last_fullsized":"2023-10-14T10:37:01.489206+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:45:25.303185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.489168+0000","last_change":"2023-10-14T10:37:00.080350+0000","last_active":"2023-10-14T10:37:01.489168+0000","last_peered":"2023-10-14T10:37:01.489168+0000","last_clean":"2023-10-14T10:37:01.489168+0000","last_became_active":"2023-10-14T10:37:00.080062+0000","last_became_peered":"2023-10-14T10:37:00.080062+0000","last_unstale":"2023-10-14T10:37:01.489168+0000","last_undegraded":"2023-10-14T10:37:01.489168+0000","last_fullsized":"2023-10-14T10:37:01.489168+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:10:46.473503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.787611+0000","last_change":"2023-10-14T10:37:00.071237+0000","last_active":"2023-10-14T10:37:01.787611+0000","last_peered":"2023-10-14T10:37:01.787611+0000","last_clean":"2023-10-14T10:37:01.787611+0000","last_became_active":"2023-10-14T10:37:00.070897+0000","last_became_peered":"2023-10-14T10:37:00.070897+0000","last_unstale":"2023-10-14T10:37:01.787611+0000","last_undegraded":"2023-10-14T10:37:01.787611+0000","last_fullsized":"2023-10-14T10:37:01.787611+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:59:07.208582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.061365+0000","last_change":"2023-10-14T10:37:00.069525+0000","last_active":"2023-10-14T10:37:01.061365+0000","last_peered":"2023-10-14T10:37:01.061365+0000","last_clean":"2023-10-14T10:37:01.061365+0000","last_became_active":"2023-10-14T10:37:00.069246+0000","last_became_peered":"2023-10-14T10:37:00.069246+0000","last_unstale":"2023-10-14T10:37:01.061365+0000","last_undegraded":"2023-10-14T10:37:01.061365+0000","last_fullsized":"2023-10-14T10:37:01.061365+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:52:37.894720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.060407+0000","last_change":"2023-10-14T10:37:00.070144+0000","last_active":"2023-10-14T10:37:01.060407+0000","last_peered":"2023-10-14T10:37:01.060407+0000","last_clean":"2023-10-14T10:37:01.060407+0000","last_became_active":"2023-10-14T10:37:00.069871+0000","last_became_peered":"2023-10-14T10:37:00.069871+0000","last_unstale":"2023-10-14T10:37:01.060407+0000","last_undegraded":"2023-10-14T10:37:01.060407+0000","last_fullsized":"2023-10-14T10:37:01.060407+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:22:32.154663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788109+0000","last_change":"2023-10-14T10:37:00.076474+0000","last_active":"2023-10-14T10:37:01.788109+0000","last_peered":"2023-10-14T10:37:01.788109+0000","last_clean":"2023-10-14T10:37:01.788109+0000","last_became_active":"2023-10-14T10:37:00.076169+0000","last_became_peered":"2023-10-14T10:37:00.076169+0000","last_unstale":"2023-10-14T10:37:01.788109+0000","last_undegraded":"2023-10-14T10:37:01.788109+0000","last_fullsized":"2023-10-14T10:37:01.788109+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T14:52:18.060480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.060360+0000","last_change":"2023-10-14T10:37:00.076568+0000","last_active":"2023-10-14T10:37:01.060360+0000","last_peered":"2023-10-14T10:37:01.060360+0000","last_clean":"2023-10-14T10:37:01.060360+0000","last_became_active":"2023-10-14T10:37:00.076263+0000","last_became_peered":"2023-10-14T10:37:00.076263+0000","last_unstale":"2023-10-14T10:37:01.060360+0000","last_undegraded":"2023-10-14T10:37:01.060360+0000","last_fullsized":"2023-10-14T10:37:01.060360+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:18:45.297555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.061994+0000","last_change":"2023-10-14T10:37:00.079439+0000","last_active":"2023-10-14T10:37:01.061994+0000","last_peered":"2023-10-14T10:37:01.061994+0000","last_clean":"2023-10-14T10:37:01.061994+0000","last_became_active":"2023-10-14T10:37:00.079178+0000","last_became_peered":"2023-10-14T10:37:00.079178+0000","last_unstale":"2023-10-14T10:37:01.061994+0000","last_undegraded":"2023-10-14T10:37:01.061994+0000","last_fullsized":"2023-10-14T10:37:01.061994+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:17:54.730152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.489206+0000","last_change":"2023-10-14T10:37:00.080675+0000","last_active":"2023-10-14T10:37:01.489206+0000","last_peered":"2023-10-14T10:37:01.489206+0000","last_clean":"2023-10-14T10:37:01.489206+0000","last_became_active":"2023-10-14T10:37:00.080171+0000","last_became_peered":"2023-10-14T10:37:00.080171+0000","last_unstale":"2023-10-14T10:37:01.489206+0000","last_undegraded":"2023-10-14T10:37:01.489206+0000","last_fullsized":"2023-10-14T10:37:01.489206+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:28:04.536948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.061814+0000","last_change":"2023-10-14T10:37:00.072155+0000","last_active":"2023-10-14T10:37:01.061814+0000","last_peered":"2023-10-14T10:37:01.061814+0000","last_clean":"2023-10-14T10:37:01.061814+0000","last_became_active":"2023-10-14T10:37:00.071882+0000","last_became_peered":"2023-10-14T10:37:00.071882+0000","last_unstale":"2023-10-14T10:37:01.061814+0000","last_undegraded":"2023-10-14T10:37:01.061814+0000","last_fullsized":"2023-10-14T10:37:01.061814+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T10:48:03.043383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.787458+0000","last_change":"2023-10-14T10:37:00.081816+0000","last_active":"2023-10-14T10:37:01.787458+0000","last_peered":"2023-10-14T10:37:01.787458+0000","last_clean":"2023-10-14T10:37:01.787458+0000","last_became_active":"2023-10-14T10:37:00.081312+0000","last_became_peered":"2023-10-14T10:37:00.081312+0000","last_unstale":"2023-10-14T10:37:01.787458+0000","last_undegraded":"2023-10-14T10:37:01.787458+0000","last_fullsized":"2023-10-14T10:37:01.787458+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:58:56.558710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.061373+0000","last_change":"2023-10-14T10:37:00.078914+0000","last_active":"2023-10-14T10:37:01.061373+0000","last_peered":"2023-10-14T10:37:01.061373+0000","last_clean":"2023-10-14T10:37:01.061373+0000","last_became_active":"2023-10-14T10:37:00.078441+0000","last_became_peered":"2023-10-14T10:37:00.078441+0000","last_unstale":"2023-10-14T10:37:01.061373+0000","last_undegraded":"2023-10-14T10:37:01.061373+0000","last_fullsized":"2023-10-14T10:37:01.061373+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:22:32.154663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.787754+0000","last_change":"2023-10-14T10:37:00.071246+0000","last_active":"2023-10-14T10:37:01.787754+0000","last_peered":"2023-10-14T10:37:01.787754+0000","last_clean":"2023-10-14T10:37:01.787754+0000","last_became_active":"2023-10-14T10:37:00.070945+0000","last_became_peered":"2023-10-14T10:37:00.070945+0000","last_unstale":"2023-10-14T10:37:01.787754+0000","last_undegraded":"2023-10-14T10:37:01.787754+0000","last_fullsized":"2023-10-14T10:37:01.787754+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:30:57.784792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.787645+0000","last_change":"2023-10-14T10:36:59.066772+0000","last_active":"2023-10-14T10:37:01.787645+0000","last_peered":"2023-10-14T10:37:01.787645+0000","last_clean":"2023-10-14T10:37:01.787645+0000","last_became_active":"2023-10-14T10:36:59.066488+0000","last_became_peered":"2023-10-14T10:36:59.066488+0000","last_unstale":"2023-10-14T10:37:01.787645+0000","last_undegraded":"2023-10-14T10:37:01.787645+0000","last_fullsized":"2023-10-14T10:37:01.787645+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:39:01.994329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.787569+0000","last_change":"2023-10-14T10:37:00.077997+0000","last_active":"2023-10-14T10:37:01.787569+0000","last_peered":"2023-10-14T10:37:01.787569+0000","last_clean":"2023-10-14T10:37:01.787569+0000","last_became_active":"2023-10-14T10:37:00.077734+0000","last_became_peered":"2023-10-14T10:37:00.077734+0000","last_unstale":"2023-10-14T10:37:01.787569+0000","last_undegraded":"2023-10-14T10:37:01.787569+0000","last_fullsized":"2023-10-14T10:37:01.787569+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:28:45.821377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.787856+0000","last_change":"2023-10-14T10:36:59.070768+0000","last_active":"2023-10-14T10:37:01.787856+0000","last_peered":"2023-10-14T10:37:01.787856+0000","last_clean":"2023-10-14T10:37:01.787856+0000","last_became_active":"2023-10-14T10:36:59.070463+0000","last_became_peered":"2023-10-14T10:36:59.070463+0000","last_unstale":"2023-10-14T10:37:01.787856+0000","last_undegraded":"2023-10-14T10:37:01.787856+0000","last_fullsized":"2023-10-14T10:37:01.787856+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:08:05.335325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.489357+0000","last_change":"2023-10-14T10:37:00.080768+0000","last_active":"2023-10-14T10:37:01.489357+0000","last_peered":"2023-10-14T10:37:01.489357+0000","last_clean":"2023-10-14T10:37:01.489357+0000","last_became_active":"2023-10-14T10:37:00.080160+0000","last_became_peered":"2023-10-14T10:37:00.080160+0000","last_unstale":"2023-10-14T10:37:01.489357+0000","last_undegraded":"2023-10-14T10:37:01.489357+0000","last_fullsized":"2023-10-14T10:37:01.489357+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T20:27:28.798883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788006+0000","last_change":"2023-10-14T10:36:59.067900+0000","last_active":"2023-10-14T10:37:01.788006+0000","last_peered":"2023-10-14T10:37:01.788006+0000","last_clean":"2023-10-14T10:37:01.788006+0000","last_became_active":"2023-10-14T10:36:59.067616+0000","last_became_peered":"2023-10-14T10:36:59.067616+0000","last_unstale":"2023-10-14T10:37:01.788006+0000","last_undegraded":"2023-10-14T10:37:01.788006+0000","last_fullsized":"2023-10-14T10:37:01.788006+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:28:44.815010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.060586+0000","last_change":"2023-10-14T10:37:00.079200+0000","last_active":"2023-10-14T10:37:01.060586+0000","last_peered":"2023-10-14T10:37:01.060586+0000","last_clean":"2023-10-14T10:37:01.060586+0000","last_became_active":"2023-10-14T10:37:00.078829+0000","last_became_peered":"2023-10-14T10:37:00.078829+0000","last_unstale":"2023-10-14T10:37:01.060586+0000","last_undegraded":"2023-10-14T10:37:01.060586+0000","last_fullsized":"2023-10-14T10:37:01.060586+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:31:16.127752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788507+0000","last_change":"2023-10-14T10:36:59.065882+0000","last_active":"2023-10-14T10:37:01.788507+0000","last_peered":"2023-10-14T10:37:01.788507+0000","last_clean":"2023-10-14T10:37:01.788507+0000","last_became_active":"2023-10-14T10:36:59.065608+0000","last_became_peered":"2023-10-14T10:36:59.065608+0000","last_unstale":"2023-10-14T10:37:01.788507+0000","last_undegraded":"2023-10-14T10:37:01.788507+0000","last_fullsized":"2023-10-14T10:37:01.788507+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T21:18:46.547776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.062326+0000","last_change":"2023-10-14T10:37:00.077115+0000","last_active":"2023-10-14T10:37:01.062326+0000","last_peered":"2023-10-14T10:37:01.062326+0000","last_clean":"2023-10-14T10:37:01.062326+0000","last_became_active":"2023-10-14T10:37:00.076840+0000","last_became_peered":"2023-10-14T10:37:00.076840+0000","last_unstale":"2023-10-14T10:37:01.062326+0000","last_undegraded":"2023-10-14T10:37:01.062326+0000","last_fullsized":"2023-10-14T10:37:01.062326+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T12:47:24.725885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.489315+0000","last_change":"2023-10-14T10:36:59.069533+0000","last_active":"2023-10-14T10:37:01.489315+0000","last_peered":"2023-10-14T10:37:01.489315+0000","last_clean":"2023-10-14T10:37:01.489315+0000","last_became_active":"2023-10-14T10:36:59.069192+0000","last_became_peered":"2023-10-14T10:36:59.069192+0000","last_unstale":"2023-10-14T10:37:01.489315+0000","last_undegraded":"2023-10-14T10:37:01.489315+0000","last_fullsized":"2023-10-14T10:37:01.489315+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T21:39:34.581362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.061634+0000","last_change":"2023-10-14T10:37:00.068786+0000","last_active":"2023-10-14T10:37:01.061634+0000","last_peered":"2023-10-14T10:37:01.061634+0000","last_clean":"2023-10-14T10:37:01.061634+0000","last_became_active":"2023-10-14T10:37:00.068517+0000","last_became_peered":"2023-10-14T10:37:00.068517+0000","last_unstale":"2023-10-14T10:37:01.061634+0000","last_undegraded":"2023-10-14T10:37:01.061634+0000","last_fullsized":"2023-10-14T10:37:01.061634+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:22:32.154663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.489303+0000","last_change":"2023-10-14T10:36:59.069445+0000","last_active":"2023-10-14T10:37:01.489303+0000","last_peered":"2023-10-14T10:37:01.489303+0000","last_clean":"2023-10-14T10:37:01.489303+0000","last_became_active":"2023-10-14T10:36:59.069151+0000","last_became_peered":"2023-10-14T10:36:59.069151+0000","last_unstale":"2023-10-14T10:37:01.489303+0000","last_undegraded":"2023-10-14T10:37:01.489303+0000","last_fullsized":"2023-10-14T10:37:01.489303+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:54:15.011628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.488953+0000","last_change":"2023-10-14T10:37:00.076984+0000","last_active":"2023-10-14T10:37:01.488953+0000","last_peered":"2023-10-14T10:37:01.488953+0000","last_clean":"2023-10-14T10:37:01.488953+0000","last_became_active":"2023-10-14T10:37:00.076706+0000","last_became_peered":"2023-10-14T10:37:00.076706+0000","last_unstale":"2023-10-14T10:37:01.488953+0000","last_undegraded":"2023-10-14T10:37:01.488953+0000","last_fullsized":"2023-10-14T10:37:01.488953+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:30:57.784792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.787709+0000","last_change":"2023-10-14T10:36:59.062324+0000","last_active":"2023-10-14T10:37:01.787709+0000","last_peered":"2023-10-14T10:37:01.787709+0000","last_clean":"2023-10-14T10:37:01.787709+0000","last_became_active":"2023-10-14T10:36:59.062075+0000","last_became_peered":"2023-10-14T10:36:59.062075+0000","last_unstale":"2023-10-14T10:37:01.787709+0000","last_undegraded":"2023-10-14T10:37:01.787709+0000","last_fullsized":"2023-10-14T10:37:01.787709+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:45:50.919718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.059848+0000","last_change":"2023-10-14T10:37:00.067486+0000","last_active":"2023-10-14T10:37:01.059848+0000","last_peered":"2023-10-14T10:37:01.059848+0000","last_clean":"2023-10-14T10:37:01.059848+0000","last_became_active":"2023-10-14T10:37:00.067187+0000","last_became_peered":"2023-10-14T10:37:00.067187+0000","last_unstale":"2023-10-14T10:37:01.059848+0000","last_undegraded":"2023-10-14T10:37:01.059848+0000","last_fullsized":"2023-10-14T10:37:01.059848+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:10:46.473503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.488998+0000","last_change":"2023-10-14T10:36:59.066330+0000","last_active":"2023-10-14T10:37:01.488998+0000","last_peered":"2023-10-14T10:37:01.488998+0000","last_clean":"2023-10-14T10:37:01.488998+0000","last_became_active":"2023-10-14T10:36:59.064990+0000","last_became_peered":"2023-10-14T10:36:59.064990+0000","last_unstale":"2023-10-14T10:37:01.488998+0000","last_undegraded":"2023-10-14T10:37:01.488998+0000","last_fullsized":"2023-10-14T10:37:01.488998+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T12:15:16.823069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.061974+0000","last_change":"2023-10-14T10:37:00.077465+0000","last_active":"2023-10-14T10:37:01.061974+0000","last_peered":"2023-10-14T10:37:01.061974+0000","last_clean":"2023-10-14T10:37:01.061974+0000","last_became_active":"2023-10-14T10:37:00.077152+0000","last_became_peered":"2023-10-14T10:37:00.077152+0000","last_unstale":"2023-10-14T10:37:01.061974+0000","last_undegraded":"2023-10-14T10:37:01.061974+0000","last_fullsized":"2023-10-14T10:37:01.061974+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:51:00.167498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788575+0000","last_change":"2023-10-14T10:36:59.061903+0000","last_active":"2023-10-14T10:37:01.788575+0000","last_peered":"2023-10-14T10:37:01.788575+0000","last_clean":"2023-10-14T10:37:01.788575+0000","last_became_active":"2023-10-14T10:36:59.061597+0000","last_became_peered":"2023-10-14T10:36:59.061597+0000","last_unstale":"2023-10-14T10:37:01.788575+0000","last_undegraded":"2023-10-14T10:37:01.788575+0000","last_fullsized":"2023-10-14T10:37:01.788575+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:22:31.148297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788727+0000","last_change":"2023-10-14T10:37:00.076213+0000","last_active":"2023-10-14T10:37:01.788727+0000","last_peered":"2023-10-14T10:37:01.788727+0000","last_clean":"2023-10-14T10:37:01.788727+0000","last_became_active":"2023-10-14T10:37:00.075889+0000","last_became_peered":"2023-10-14T10:37:00.075889+0000","last_unstale":"2023-10-14T10:37:01.788727+0000","last_undegraded":"2023-10-14T10:37:01.788727+0000","last_fullsized":"2023-10-14T10:37:01.788727+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T14:19:22.310866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.787968+0000","last_change":"2023-10-14T10:36:59.070183+0000","last_active":"2023-10-14T10:37:01.787968+0000","last_peered":"2023-10-14T10:37:01.787968+0000","last_clean":"2023-10-14T10:37:01.787968+0000","last_became_active":"2023-10-14T10:36:59.069731+0000","last_became_peered":"2023-10-14T10:36:59.069731+0000","last_unstale":"2023-10-14T10:37:01.787968+0000","last_undegraded":"2023-10-14T10:37:01.787968+0000","last_fullsized":"2023-10-14T10:37:01.787968+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:58:55.552343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788210+0000","last_change":"2023-10-14T10:37:00.072592+0000","last_active":"2023-10-14T10:37:01.788210+0000","last_peered":"2023-10-14T10:37:01.788210+0000","last_clean":"2023-10-14T10:37:01.788210+0000","last_became_active":"2023-10-14T10:37:00.072288+0000","last_became_peered":"2023-10-14T10:37:00.072288+0000","last_unstale":"2023-10-14T10:37:01.788210+0000","last_undegraded":"2023-10-14T10:37:01.788210+0000","last_fullsized":"2023-10-14T10:37:01.788210+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T17:27:50.406084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.062406+0000","last_change":"2023-10-14T10:36:59.062120+0000","last_active":"2023-10-14T10:37:01.062406+0000","last_peered":"2023-10-14T10:37:01.062406+0000","last_clean":"2023-10-14T10:37:01.062406+0000","last_became_active":"2023-10-14T10:36:59.061700+0000","last_became_peered":"2023-10-14T10:36:59.061700+0000","last_unstale":"2023-10-14T10:37:01.062406+0000","last_undegraded":"2023-10-14T10:37:01.062406+0000","last_fullsized":"2023-10-14T10:37:01.062406+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:45:24.296819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.062364+0000","last_change":"2023-10-14T10:37:00.074977+0000","last_active":"2023-10-14T10:37:01.062364+0000","last_peered":"2023-10-14T10:37:01.062364+0000","last_clean":"2023-10-14T10:37:01.062364+0000","last_became_active":"2023-10-14T10:37:00.074700+0000","last_became_peered":"2023-10-14T10:37:00.074700+0000","last_unstale":"2023-10-14T10:37:01.062364+0000","last_undegraded":"2023-10-14T10:37:01.062364+0000","last_fullsized":"2023-10-14T10:37:01.062364+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T18:57:50.136605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788323+0000","last_change":"2023-10-14T10:36:59.069400+0000","last_active":"2023-10-14T10:37:01.788323+0000","last_peered":"2023-10-14T10:37:01.788323+0000","last_clean":"2023-10-14T10:37:01.788323+0000","last_became_active":"2023-10-14T10:36:59.069043+0000","last_became_peered":"2023-10-14T10:36:59.069043+0000","last_unstale":"2023-10-14T10:37:01.788323+0000","last_undegraded":"2023-10-14T10:37:01.788323+0000","last_fullsized":"2023-10-14T10:37:01.788323+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:10:45.467136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.060040+0000","last_change":"2023-10-14T10:37:00.068613+0000","last_active":"2023-10-14T10:37:01.060040+0000","last_peered":"2023-10-14T10:37:01.060040+0000","last_clean":"2023-10-14T10:37:01.060040+0000","last_became_active":"2023-10-14T10:37:00.065345+0000","last_became_peered":"2023-10-14T10:37:00.065345+0000","last_unstale":"2023-10-14T10:37:01.060040+0000","last_undegraded":"2023-10-14T10:37:01.060040+0000","last_fullsized":"2023-10-14T10:37:01.060040+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:45:25.303185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.489387+0000","last_change":"2023-10-14T10:36:59.065846+0000","last_active":"2023-10-14T10:37:01.489387+0000","last_peered":"2023-10-14T10:37:01.489387+0000","last_clean":"2023-10-14T10:37:01.489387+0000","last_became_active":"2023-10-14T10:36:59.065569+0000","last_became_peered":"2023-10-14T10:36:59.065569+0000","last_unstale":"2023-10-14T10:37:01.489387+0000","last_undegraded":"2023-10-14T10:37:01.489387+0000","last_fullsized":"2023-10-14T10:37:01.489387+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:28:44.815010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788692+0000","last_change":"2023-10-14T10:37:00.079137+0000","last_active":"2023-10-14T10:37:01.788692+0000","last_peered":"2023-10-14T10:37:01.788692+0000","last_clean":"2023-10-14T10:37:01.788692+0000","last_became_active":"2023-10-14T10:37:00.078844+0000","last_became_peered":"2023-10-14T10:37:00.078844+0000","last_unstale":"2023-10-14T10:37:01.788692+0000","last_undegraded":"2023-10-14T10:37:01.788692+0000","last_fullsized":"2023-10-14T10:37:01.788692+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:30:51.914014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.059920+0000","last_change":"2023-10-14T10:36:59.067245+0000","last_active":"2023-10-14T10:37:01.059920+0000","last_peered":"2023-10-14T10:37:01.059920+0000","last_clean":"2023-10-14T10:37:01.059920+0000","last_became_active":"2023-10-14T10:36:59.066952+0000","last_became_peered":"2023-10-14T10:36:59.066952+0000","last_unstale":"2023-10-14T10:37:01.059920+0000","last_undegraded":"2023-10-14T10:37:01.059920+0000","last_fullsized":"2023-10-14T10:37:01.059920+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T12:08:55.620308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.062454+0000","last_change":"2023-10-14T10:37:00.078646+0000","last_active":"2023-10-14T10:37:01.062454+0000","last_peered":"2023-10-14T10:37:01.062454+0000","last_clean":"2023-10-14T10:37:01.062454+0000","last_became_active":"2023-10-14T10:37:00.078353+0000","last_became_peered":"2023-10-14T10:37:00.078353+0000","last_unstale":"2023-10-14T10:37:01.062454+0000","last_undegraded":"2023-10-14T10:37:01.062454+0000","last_fullsized":"2023-10-14T10:37:01.062454+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:30:51.914014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788938+0000","last_change":"2023-10-14T10:36:59.063118+0000","last_active":"2023-10-14T10:37:01.788938+0000","last_peered":"2023-10-14T10:37:01.788938+0000","last_clean":"2023-10-14T10:37:01.788938+0000","last_became_active":"2023-10-14T10:36:59.062380+0000","last_became_peered":"2023-10-14T10:36:59.062380+0000","last_unstale":"2023-10-14T10:37:01.788938+0000","last_undegraded":"2023-10-14T10:37:01.788938+0000","last_fullsized":"2023-10-14T10:37:01.788938+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T21:39:34.581362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.787507+0000","last_change":"2023-10-14T10:37:00.078077+0000","last_active":"2023-10-14T10:37:01.787507+0000","last_peered":"2023-10-14T10:37:01.787507+0000","last_clean":"2023-10-14T10:37:01.787507+0000","last_became_active":"2023-10-14T10:37:00.077806+0000","last_became_peered":"2023-10-14T10:37:00.077806+0000","last_unstale":"2023-10-14T10:37:01.787507+0000","last_undegraded":"2023-10-14T10:37:01.787507+0000","last_fullsized":"2023-10-14T10:37:01.787507+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:45:51.926085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788260+0000","last_change":"2023-10-14T10:36:59.068012+0000","last_active":"2023-10-14T10:37:01.788260+0000","last_peered":"2023-10-14T10:37:01.788260+0000","last_clean":"2023-10-14T10:37:01.788260+0000","last_became_active":"2023-10-14T10:36:59.067727+0000","last_became_peered":"2023-10-14T10:36:59.067727+0000","last_unstale":"2023-10-14T10:37:01.788260+0000","last_undegraded":"2023-10-14T10:37:01.788260+0000","last_fullsized":"2023-10-14T10:37:01.788260+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:44:49.671921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.489522+0000","last_change":"2023-10-14T10:37:00.076926+0000","last_active":"2023-10-14T10:37:01.489522+0000","last_peered":"2023-10-14T10:37:01.489522+0000","last_clean":"2023-10-14T10:37:01.489522+0000","last_became_active":"2023-10-14T10:37:00.076658+0000","last_became_peered":"2023-10-14T10:37:00.076658+0000","last_unstale":"2023-10-14T10:37:01.489522+0000","last_undegraded":"2023-10-14T10:37:01.489522+0000","last_fullsized":"2023-10-14T10:37:01.489522+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:52:37.894720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.060490+0000","last_change":"2023-10-14T10:36:59.071714+0000","last_active":"2023-10-14T10:37:01.060490+0000","last_peered":"2023-10-14T10:37:01.060490+0000","last_clean":"2023-10-14T10:37:01.060490+0000","last_became_active":"2023-10-14T10:36:59.071406+0000","last_became_peered":"2023-10-14T10:36:59.071406+0000","last_unstale":"2023-10-14T10:37:01.060490+0000","last_undegraded":"2023-10-14T10:37:01.060490+0000","last_fullsized":"2023-10-14T10:37:01.060490+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:59:06.202215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.062345+0000","last_change":"2023-10-14T10:37:00.080060+0000","last_active":"2023-10-14T10:37:01.062345+0000","last_peered":"2023-10-14T10:37:01.062345+0000","last_clean":"2023-10-14T10:37:01.062345+0000","last_became_active":"2023-10-14T10:37:00.079795+0000","last_became_peered":"2023-10-14T10:37:00.079795+0000","last_unstale":"2023-10-14T10:37:01.062345+0000","last_undegraded":"2023-10-14T10:37:01.062345+0000","last_fullsized":"2023-10-14T10:37:01.062345+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T17:27:50.406084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.489554+0000","last_change":"2023-10-14T10:36:59.065271+0000","last_active":"2023-10-14T10:37:01.489554+0000","last_peered":"2023-10-14T10:37:01.489554+0000","last_clean":"2023-10-14T10:37:01.489554+0000","last_became_active":"2023-10-14T10:36:59.064959+0000","last_became_peered":"2023-10-14T10:36:59.064959+0000","last_unstale":"2023-10-14T10:37:01.489554+0000","last_undegraded":"2023-10-14T10:37:01.489554+0000","last_fullsized":"2023-10-14T10:37:01.489554+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T21:56:55.137838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.060192+0000","last_change":"2023-10-14T10:37:00.077751+0000","last_active":"2023-10-14T10:37:01.060192+0000","last_peered":"2023-10-14T10:37:01.060192+0000","last_clean":"2023-10-14T10:37:01.060192+0000","last_became_active":"2023-10-14T10:37:00.077459+0000","last_became_peered":"2023-10-14T10:37:00.077459+0000","last_unstale":"2023-10-14T10:37:01.060192+0000","last_undegraded":"2023-10-14T10:37:01.060192+0000","last_fullsized":"2023-10-14T10:37:01.060192+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T20:27:28.798883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.489473+0000","last_change":"2023-10-14T10:36:59.058893+0000","last_active":"2023-10-14T10:37:01.489473+0000","last_peered":"2023-10-14T10:37:01.489473+0000","last_clean":"2023-10-14T10:37:01.489473+0000","last_became_active":"2023-10-14T10:36:59.058558+0000","last_became_peered":"2023-10-14T10:36:59.058558+0000","last_unstale":"2023-10-14T10:37:01.489473+0000","last_undegraded":"2023-10-14T10:37:01.489473+0000","last_fullsized":"2023-10-14T10:37:01.489473+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:45:50.919718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788693+0000","last_change":"2023-10-14T10:37:00.071398+0000","last_active":"2023-10-14T10:37:01.788693+0000","last_peered":"2023-10-14T10:37:01.788693+0000","last_clean":"2023-10-14T10:37:01.788693+0000","last_became_active":"2023-10-14T10:37:00.071167+0000","last_became_peered":"2023-10-14T10:37:00.071167+0000","last_unstale":"2023-10-14T10:37:01.788693+0000","last_undegraded":"2023-10-14T10:37:01.788693+0000","last_fullsized":"2023-10-14T10:37:01.788693+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:31:12.123956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.489473+0000","last_change":"2023-10-14T10:36:59.069420+0000","last_active":"2023-10-14T10:37:01.489473+0000","last_peered":"2023-10-14T10:37:01.489473+0000","last_clean":"2023-10-14T10:37:01.489473+0000","last_became_active":"2023-10-14T10:36:59.069110+0000","last_became_peered":"2023-10-14T10:36:59.069110+0000","last_unstale":"2023-10-14T10:37:01.489473+0000","last_undegraded":"2023-10-14T10:37:01.489473+0000","last_fullsized":"2023-10-14T10:37:01.489473+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:09:59.812757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788586+0000","last_change":"2023-10-14T10:37:00.077550+0000","last_active":"2023-10-14T10:37:01.788586+0000","last_peered":"2023-10-14T10:37:01.788586+0000","last_clean":"2023-10-14T10:37:01.788586+0000","last_became_active":"2023-10-14T10:37:00.077175+0000","last_became_peered":"2023-10-14T10:37:00.077175+0000","last_unstale":"2023-10-14T10:37:01.788586+0000","last_undegraded":"2023-10-14T10:37:01.788586+0000","last_fullsized":"2023-10-14T10:37:01.788586+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T14:51:19.968923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.787861+0000","last_change":"2023-10-14T10:36:59.067621+0000","last_active":"2023-10-14T10:37:01.787861+0000","last_peered":"2023-10-14T10:37:01.787861+0000","last_clean":"2023-10-14T10:37:01.787861+0000","last_became_active":"2023-10-14T10:36:59.067345+0000","last_became_peered":"2023-10-14T10:36:59.067345+0000","last_unstale":"2023-10-14T10:37:01.787861+0000","last_undegraded":"2023-10-14T10:37:01.787861+0000","last_fullsized":"2023-10-14T10:37:01.787861+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:54:15.011628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788501+0000","last_change":"2023-10-14T10:37:00.077744+0000","last_active":"2023-10-14T10:37:01.788501+0000","last_peered":"2023-10-14T10:37:01.788501+0000","last_clean":"2023-10-14T10:37:01.788501+0000","last_became_active":"2023-10-14T10:37:00.077465+0000","last_became_peered":"2023-10-14T10:37:00.077465+0000","last_unstale":"2023-10-14T10:37:01.788501+0000","last_undegraded":"2023-10-14T10:37:01.788501+0000","last_fullsized":"2023-10-14T10:37:01.788501+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T20:36:23.971821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.787432+0000","last_change":"2023-10-14T10:36:59.069707+0000","last_active":"2023-10-14T10:37:01.787432+0000","last_peered":"2023-10-14T10:37:01.787432+0000","last_clean":"2023-10-14T10:37:01.787432+0000","last_became_active":"2023-10-14T10:36:59.069269+0000","last_became_peered":"2023-10-14T10:36:59.069269+0000","last_unstale":"2023-10-14T10:37:01.787432+0000","last_undegraded":"2023-10-14T10:37:01.787432+0000","last_fullsized":"2023-10-14T10:37:01.787432+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T12:15:16.823069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788221+0000","last_change":"2023-10-14T10:37:00.068887+0000","last_active":"2023-10-14T10:37:01.788221+0000","last_peered":"2023-10-14T10:37:01.788221+0000","last_clean":"2023-10-14T10:37:01.788221+0000","last_became_active":"2023-10-14T10:37:00.068620+0000","last_became_peered":"2023-10-14T10:37:00.068620+0000","last_unstale":"2023-10-14T10:37:01.788221+0000","last_undegraded":"2023-10-14T10:37:01.788221+0000","last_fullsized":"2023-10-14T10:37:01.788221+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T18:04:59.338050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.489058+0000","last_change":"2023-10-14T10:36:59.061947+0000","last_active":"2023-10-14T10:37:01.489058+0000","last_peered":"2023-10-14T10:37:01.489058+0000","last_clean":"2023-10-14T10:37:01.489058+0000","last_became_active":"2023-10-14T10:36:59.061678+0000","last_became_peered":"2023-10-14T10:36:59.061678+0000","last_unstale":"2023-10-14T10:37:01.489058+0000","last_undegraded":"2023-10-14T10:37:01.489058+0000","last_fullsized":"2023-10-14T10:37:01.489058+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:39:01.994329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.061998+0000","last_change":"2023-10-14T10:37:00.073204+0000","last_active":"2023-10-14T10:37:01.061998+0000","last_peered":"2023-10-14T10:37:01.061998+0000","last_clean":"2023-10-14T10:37:01.061998+0000","last_became_active":"2023-10-14T10:37:00.071692+0000","last_became_peered":"2023-10-14T10:37:00.071692+0000","last_unstale":"2023-10-14T10:37:01.061998+0000","last_undegraded":"2023-10-14T10:37:01.061998+0000","last_fullsized":"2023-10-14T10:37:01.061998+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:17:54.730152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.061467+0000","last_change":"2023-10-14T10:36:59.063545+0000","last_active":"2023-10-14T10:37:01.061467+0000","last_peered":"2023-10-14T10:37:01.061467+0000","last_clean":"2023-10-14T10:37:01.061467+0000","last_became_active":"2023-10-14T10:36:59.063332+0000","last_became_peered":"2023-10-14T10:36:59.063332+0000","last_unstale":"2023-10-14T10:37:01.061467+0000","last_undegraded":"2023-10-14T10:37:01.061467+0000","last_fullsized":"2023-10-14T10:37:01.061467+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:10:45.467136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788558+0000","last_change":"2023-10-14T10:37:00.075249+0000","last_active":"2023-10-14T10:37:01.788558+0000","last_peered":"2023-10-14T10:37:01.788558+0000","last_clean":"2023-10-14T10:37:01.788558+0000","last_became_active":"2023-10-14T10:37:00.074859+0000","last_became_peered":"2023-10-14T10:37:00.074859+0000","last_unstale":"2023-10-14T10:37:01.788558+0000","last_undegraded":"2023-10-14T10:37:01.788558+0000","last_fullsized":"2023-10-14T10:37:01.788558+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:04:51.088268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.060369+0000","last_change":"2023-10-14T10:36:59.069418+0000","last_active":"2023-10-14T10:37:01.060369+0000","last_peered":"2023-10-14T10:37:01.060369+0000","last_clean":"2023-10-14T10:37:01.060369+0000","last_became_active":"2023-10-14T10:36:59.069037+0000","last_became_peered":"2023-10-14T10:36:59.069037+0000","last_unstale":"2023-10-14T10:37:01.060369+0000","last_undegraded":"2023-10-14T10:37:01.060369+0000","last_fullsized":"2023-10-14T10:37:01.060369+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:45:50.919718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.062253+0000","last_change":"2023-10-14T10:37:00.075731+0000","last_active":"2023-10-14T10:37:01.062253+0000","last_peered":"2023-10-14T10:37:01.062253+0000","last_clean":"2023-10-14T10:37:01.062253+0000","last_became_active":"2023-10-14T10:37:00.075491+0000","last_became_peered":"2023-10-14T10:37:00.075491+0000","last_unstale":"2023-10-14T10:37:01.062253+0000","last_undegraded":"2023-10-14T10:37:01.062253+0000","last_fullsized":"2023-10-14T10:37:01.062253+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T14:52:18.060480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"33'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.821131+0000","last_change":"2023-10-14T10:36:59.066461+0000","last_active":"2023-10-14T10:37:01.821131+0000","last_peered":"2023-10-14T10:37:01.821131+0000","last_clean":"2023-10-14T10:37:01.821131+0000","last_became_active":"2023-10-14T10:36:59.066146+0000","last_became_peered":"2023-10-14T10:36:59.066146+0000","last_unstale":"2023-10-14T10:37:01.821131+0000","last_undegraded":"2023-10-14T10:37:01.821131+0000","last_fullsized":"2023-10-14T10:37:01.821131+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:45:50.919718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.787608+0000","last_change":"2023-10-14T10:37:00.082851+0000","last_active":"2023-10-14T10:37:01.787608+0000","last_peered":"2023-10-14T10:37:01.787608+0000","last_clean":"2023-10-14T10:37:01.787608+0000","last_became_active":"2023-10-14T10:37:00.082570+0000","last_became_peered":"2023-10-14T10:37:00.082570+0000","last_unstale":"2023-10-14T10:37:01.787608+0000","last_undegraded":"2023-10-14T10:37:01.787608+0000","last_fullsized":"2023-10-14T10:37:01.787608+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:28:45.821377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.059981+0000","last_change":"2023-10-14T10:36:59.061936+0000","last_active":"2023-10-14T10:37:01.059981+0000","last_peered":"2023-10-14T10:37:01.059981+0000","last_clean":"2023-10-14T10:37:01.059981+0000","last_became_active":"2023-10-14T10:36:59.061642+0000","last_became_peered":"2023-10-14T10:36:59.061642+0000","last_unstale":"2023-10-14T10:37:01.059981+0000","last_undegraded":"2023-10-14T10:37:01.059981+0000","last_fullsized":"2023-10-14T10:37:01.059981+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:54:15.011628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.787449+0000","last_change":"2023-10-14T10:37:00.071551+0000","last_active":"2023-10-14T10:37:01.787449+0000","last_peered":"2023-10-14T10:37:01.787449+0000","last_clean":"2023-10-14T10:37:01.787449+0000","last_became_active":"2023-10-14T10:37:00.071244+0000","last_became_peered":"2023-10-14T10:37:00.071244+0000","last_unstale":"2023-10-14T10:37:01.787449+0000","last_undegraded":"2023-10-14T10:37:01.787449+0000","last_fullsized":"2023-10-14T10:37:01.787449+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:08:06.341692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.787778+0000","last_change":"2023-10-14T10:36:59.066839+0000","last_active":"2023-10-14T10:37:01.787778+0000","last_peered":"2023-10-14T10:37:01.787778+0000","last_clean":"2023-10-14T10:37:01.787778+0000","last_became_active":"2023-10-14T10:36:59.066592+0000","last_became_peered":"2023-10-14T10:36:59.066592+0000","last_unstale":"2023-10-14T10:37:01.787778+0000","last_undegraded":"2023-10-14T10:37:01.787778+0000","last_fullsized":"2023-10-14T10:37:01.787778+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T12:15:16.823069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788013+0000","last_change":"2023-10-14T10:37:00.076076+0000","last_active":"2023-10-14T10:37:01.788013+0000","last_peered":"2023-10-14T10:37:01.788013+0000","last_clean":"2023-10-14T10:37:01.788013+0000","last_became_active":"2023-10-14T10:37:00.075794+0000","last_became_peered":"2023-10-14T10:37:00.075794+0000","last_unstale":"2023-10-14T10:37:01.788013+0000","last_undegraded":"2023-10-14T10:37:01.788013+0000","last_fullsized":"2023-10-14T10:37:01.788013+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:44:50.678288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.060021+0000","last_change":"2023-10-14T10:36:59.065626+0000","last_active":"2023-10-14T10:37:01.060021+0000","last_peered":"2023-10-14T10:37:01.060021+0000","last_clean":"2023-10-14T10:37:01.060021+0000","last_became_active":"2023-10-14T10:36:59.065163+0000","last_became_peered":"2023-10-14T10:36:59.065163+0000","last_unstale":"2023-10-14T10:37:01.060021+0000","last_undegraded":"2023-10-14T10:37:01.060021+0000","last_fullsized":"2023-10-14T10:37:01.060021+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:08:05.335325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788427+0000","last_change":"2023-10-14T10:37:00.078024+0000","last_active":"2023-10-14T10:37:01.788427+0000","last_peered":"2023-10-14T10:37:01.788427+0000","last_clean":"2023-10-14T10:37:01.788427+0000","last_became_active":"2023-10-14T10:37:00.077621+0000","last_became_peered":"2023-10-14T10:37:00.077621+0000","last_unstale":"2023-10-14T10:37:01.788427+0000","last_undegraded":"2023-10-14T10:37:01.788427+0000","last_fullsized":"2023-10-14T10:37:01.788427+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:06:28.391046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.489094+0000","last_change":"2023-10-14T10:36:59.064632+0000","last_active":"2023-10-14T10:37:01.489094+0000","last_peered":"2023-10-14T10:37:01.489094+0000","last_clean":"2023-10-14T10:37:01.489094+0000","last_became_active":"2023-10-14T10:36:59.064336+0000","last_became_peered":"2023-10-14T10:36:59.064336+0000","last_unstale":"2023-10-14T10:37:01.489094+0000","last_undegraded":"2023-10-14T10:37:01.489094+0000","last_fullsized":"2023-10-14T10:37:01.489094+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:08:05.335325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.488971+0000","last_change":"2023-10-14T10:37:00.081042+0000","last_active":"2023-10-14T10:37:01.488971+0000","last_peered":"2023-10-14T10:37:01.488971+0000","last_clean":"2023-10-14T10:37:01.488971+0000","last_became_active":"2023-10-14T10:37:00.080340+0000","last_became_peered":"2023-10-14T10:37:00.080340+0000","last_unstale":"2023-10-14T10:37:01.488971+0000","last_undegraded":"2023-10-14T10:37:01.488971+0000","last_fullsized":"2023-10-14T10:37:01.488971+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:44:50.678288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788029+0000","last_change":"2023-10-14T10:36:59.063568+0000","last_active":"2023-10-14T10:37:01.788029+0000","last_peered":"2023-10-14T10:37:01.788029+0000","last_clean":"2023-10-14T10:37:01.788029+0000","last_became_active":"2023-10-14T10:36:59.063130+0000","last_became_peered":"2023-10-14T10:36:59.063130+0000","last_unstale":"2023-10-14T10:37:01.788029+0000","last_undegraded":"2023-10-14T10:37:01.788029+0000","last_fullsized":"2023-10-14T10:37:01.788029+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:59:06.202215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.061998+0000","last_change":"2023-10-14T10:37:00.068870+0000","last_active":"2023-10-14T10:37:01.061998+0000","last_peered":"2023-10-14T10:37:01.061998+0000","last_clean":"2023-10-14T10:37:01.061998+0000","last_became_active":"2023-10-14T10:37:00.068556+0000","last_became_peered":"2023-10-14T10:37:00.068556+0000","last_unstale":"2023-10-14T10:37:01.061998+0000","last_undegraded":"2023-10-14T10:37:01.061998+0000","last_fullsized":"2023-10-14T10:37:01.061998+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T14:52:18.060480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.787484+0000","last_change":"2023-10-14T10:36:59.066703+0000","last_active":"2023-10-14T10:37:01.787484+0000","last_peered":"2023-10-14T10:37:01.787484+0000","last_clean":"2023-10-14T10:37:01.787484+0000","last_became_active":"2023-10-14T10:36:59.066424+0000","last_became_peered":"2023-10-14T10:36:59.066424+0000","last_unstale":"2023-10-14T10:37:01.787484+0000","last_undegraded":"2023-10-14T10:37:01.787484+0000","last_fullsized":"2023-10-14T10:37:01.787484+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:54:15.011628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":176,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-14T10:37:00.060617+0000","last_change":"2023-10-14T10:35:59.770163+0000","last_active":"2023-10-14T10:37:00.060617+0000","last_peered":"2023-10-14T10:37:00.060617+0000","last_clean":"2023-10-14T10:37:00.060617+0000","last_became_active":"2023-10-14T10:35:59.769199+0000","last_became_peered":"2023-10-14T10:35:59.769199+0000","last_unstale":"2023-10-14T10:37:00.060617+0000","last_undegraded":"2023-10-14T10:37:00.060617+0000","last_fullsized":"2023-10-14T10:37:00.060617+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:35:58.675572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:35:58.675572+0000","last_clean_scrub_stamp":"2023-10-14T10:35:58.675572+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:03:25.281187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542162,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27972,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343868,"statfs":{"total":96636764160,"available":96608120832,"internally_reserved":0,"allocated":966656,"data_stored":732594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 10:36:59 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":2.4649999999999999,"5min":2.4649999999999999,"15min":2.4649999999999999},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.45500000000000002}]},{"osd":1,"last update":"Sat Oct 14 10:36:59 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":2.4289999999999998,"5min":2.4289999999999998,"15min":2.4289999999999998},"last":0.60199999999999998}]},{"osd":2,"last update":"Sat Oct 14 10:36:59 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.59099999999999997}]},{"osd":3,"last update":"Sat Oct 14 10:36:59 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.405,"5min":1.405,"15min":1.405},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.47699999999999998}]},{"osd":4,"last update":"Sat Oct 14 10:36:59 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.57799999999999996}]},{"osd":5,"last update":"Sat Oct 14 10:36:59 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.51900000000000002}]},{"osd":6,"last update":"Sat Oct 14 10:36:59 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.42499999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542163,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27388,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344452,"statfs":{"total":96636764160,"available":96608718848,"internally_reserved":0,"allocated":376832,"data_stored":142752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 10:36:57 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.58499999999999996}]},{"osd":1,"last update":"Sat Oct 14 10:36:57 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.389,"5min":1.389,"15min":1.389},"last":0.68899999999999995}]},{"osd":2,"last update":"Sat Oct 14 10:36:57 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.74299999999999999}]},{"osd":3,"last update":"Sat Oct 14 10:36:57 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.65100000000000002}]},{"osd":4,"last update":"Sat Oct 14 10:36:57 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.499}]},{"osd":5,"last update":"Sat Oct 14 10:36:57 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.52700000000000002}]},{"osd":7,"last update":"Sat Oct 14 10:36:57 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.435},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.41299999999999998}]}]},{"osd":1,"up_from":14,"seq":60129542162,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":376,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":385024,"data_stored":143770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.254,"15min":1.254},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":16.855,"5min":16.855,"15min":16.855},"last":0.28399999999999997},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":19.353000000000002,"5min":19.353000000000002,"15min":19.353000000000002},"last":0.36399999999999999}]},{"osd":2,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":3.0670000000000002,"5min":3.0670000000000002,"15min":3.0670000000000002},"last":0.26100000000000001},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.32500000000000001}]},{"osd":3,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":10.005000000000001,"5min":10.005000000000001,"15min":10.005000000000001},"last":0.30299999999999999}]},{"osd":4,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":2.9500000000000002,"5min":2.9500000000000002,"15min":2.9500000000000002},"last":0.495},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":5.7270000000000003,"5min":5.7270000000000003,"15min":5.7270000000000003},"last":0.65400000000000003}]},{"osd":5,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.129,"15min":1.129},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":9.8800000000000008,"5min":9.8800000000000008,"15min":9.8800000000000008},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":3.0350000000000001,"5min":3.0350000000000001,"15min":3.0350000000000001},"last":0.66700000000000004}]},{"osd":6,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":5.681,"5min":5.681,"15min":5.681},"last":0.54600000000000004},{"interface":"front","average":{"1min":1.173,"5min":1.173,"15min":1.173},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":9.9380000000000006,"5min":9.9380000000000006,"15min":9.9380000000000006},"last":0.68700000000000006}]},{"osd":7,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":9.9730000000000008,"5min":9.9730000000000008,"15min":9.9730000000000008},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":3.0009999999999999,"5min":3.0009999999999999,"15min":3.0009999999999999},"last":0.58899999999999997}]}]},{"osd":0,"up_from":14,"seq":60129542161,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27960,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343880,"statfs":{"total":96636764160,"available":96608133120,"internally_reserved":0,"allocated":962560,"data_stored":732504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.62,"5min":0.62,"15min":0.62},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.495,"5min":0.495,"15min":0.495},"last":0.48199999999999998}]},{"osd":2,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"last":0.378},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"last":0.40400000000000003}]},{"osd":3,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"last":0.42699999999999999}]},{"osd":4,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.66500000000000004}]},{"osd":5,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.55800000000000005}]},{"osd":6,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.57299999999999995}]},{"osd":7,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.69699999999999995}]}]},{"osd":2,"up_from":14,"seq":60129542162,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":372736,"data_stored":142206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":1.429,"5min":1.429,"15min":1.429},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.36199999999999999}]},{"osd":1,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":1.5269999999999999,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.40500000000000003}]},{"osd":3,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":4.0629999999999997,"5min":4.0629999999999997,"15min":4.0629999999999997},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.381,"5min":1.381,"15min":1.381},"last":0.42899999999999999}]},{"osd":4,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.58399999999999996}]},{"osd":5,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":3.8599999999999999,"5min":3.8599999999999999,"15min":3.8599999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.464,"5min":1.464,"15min":1.464},"last":0.68899999999999995}]},{"osd":6,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.753},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":3.9820000000000002,"5min":3.9820000000000002,"15min":3.9820000000000002},"last":0.63600000000000001}]},{"osd":7,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":3.9049999999999998,"5min":3.9049999999999998,"15min":3.9049999999999998},"last":0.61699999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542162,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":376,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":385024,"data_stored":143770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"last":0.34300000000000003}]},{"osd":1,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"last":0.373}]},{"osd":2,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"last":0.26600000000000001},{"interface":"front","average":{"1min":0.437,"5min":0.437,"15min":0.437},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.40300000000000002}]},{"osd":4,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":6.0350000000000001,"5min":6.0350000000000001,"15min":6.0350000000000001},"last":0.71499999999999997}]},{"osd":5,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":6.0540000000000003,"5min":6.0540000000000003,"15min":6.0540000000000003},"last":0.65700000000000003}]},{"osd":6,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.621}]},{"osd":7,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":6.0819999999999999,"5min":6.0819999999999999,"15min":6.0819999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.61199999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542161,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27388,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344452,"statfs":{"total":96636764160,"available":96608718848,"internally_reserved":0,"allocated":376832,"data_stored":142240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 10:36:56 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.47699999999999998}]},{"osd":1,"last update":"Sat Oct 14 10:36:56 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.39000000000000001}]},{"osd":2,"last update":"Sat Oct 14 10:36:56 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.42699999999999999}]},{"osd":3,"last update":"Sat Oct 14 10:36:56 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.48999999999999999}]},{"osd":5,"last update":"Sat Oct 14 10:36:56 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.60299999999999998}]},{"osd":6,"last update":"Sat Oct 14 10:36:56 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.61899999999999999}]},{"osd":7,"last update":"Sat Oct 14 10:36:56 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.66300000000000003}]}]},{"osd":5,"up_from":14,"seq":60129542162,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27388,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344452,"statfs":{"total":96636764160,"available":96608718848,"internally_reserved":0,"allocated":376832,"data_stored":142808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.64600000000000002}]},{"osd":1,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.72099999999999997}]},{"osd":2,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.70799999999999996}]},{"osd":3,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.66200000000000003}]},{"osd":4,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.372},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.41099999999999998}]},{"osd":6,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.439}]},{"osd":7,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.51900000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-14T10:37:08.479 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T10:37:08.638 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.635+0000 7f876a9d6700 1 Processor -- start 2023-10-14T10:37:08.638 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.635+0000 7f876a9d6700 1 -- start start 2023-10-14T10:37:08.641 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.636+0000 7f876a9d6700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f876415f130 0x7f87640cede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:08.641 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.636+0000 7f876a9d6700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f87640b28b0 0x7f876406a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:08.641 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.636+0000 7f876a9d6700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f876406ad80 0x7f8764071af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:08.642 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.636+0000 7f876a9d6700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87640cde00 con 0x7f876406ad80 2023-10-14T10:37:08.642 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.636+0000 7f876a9d6700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87640b3530 con 0x7f87640b28b0 2023-10-14T10:37:08.642 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.636+0000 7f876a9d6700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f876406b1b0 con 0x7f876415f130 2023-10-14T10:37:08.642 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.636+0000 7f8768f73700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f876406ad80 0x7f8764071af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:08.643 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.636+0000 7f8768f73700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f876406ad80 0x7f8764071af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45960/0 (socket says 172.21.15.67:45960) 2023-10-14T10:37:08.643 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.636+0000 7f8768f73700 1 -- 172.21.15.67:0/681748441 learned_addr learned my addr 172.21.15.67:0/681748441 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:37:08.643 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.636+0000 7f87637fe700 1 --2- 172.21.15.67:0/681748441 >> v2:172.21.15.72:3300/0 conn(0x7f87640b28b0 0x7f876406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:08.644 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.637+0000 7f8763fff700 1 --2- 172.21.15.67:0/681748441 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f876415f130 0x7f87640cede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:08.644 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.637+0000 7f8768f73700 1 -- 172.21.15.67:0/681748441 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f876415f130 msgr2=0x7f87640cede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:37:08.644 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.637+0000 7f8768f73700 1 --2- 172.21.15.67:0/681748441 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f876415f130 0x7f87640cede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:08.644 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.637+0000 7f8768f73700 1 -- 172.21.15.67:0/681748441 >> v2:172.21.15.72:3300/0 conn(0x7f87640b28b0 msgr2=0x7f876406a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:37:08.645 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.637+0000 7f8768f73700 1 --2- 172.21.15.67:0/681748441 >> v2:172.21.15.72:3300/0 conn(0x7f87640b28b0 0x7f876406a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:08.645 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.637+0000 7f8768f73700 1 -- 172.21.15.67:0/681748441 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87640cd180 con 0x7f876406ad80 2023-10-14T10:37:08.645 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.637+0000 7f8768f73700 1 --2- 172.21.15.67:0/681748441 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f876406ad80 0x7f8764071af0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f87640d1560 tx=0x7f875c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6083660423313466 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:08.645 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.637+0000 7f8762ffd700 1 -- 172.21.15.67:0/681748441 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f875c00e070 con 0x7f876406ad80 2023-10-14T10:37:08.646 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.637+0000 7f8762ffd700 1 -- 172.21.15.67:0/681748441 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f875c00b910 con 0x7f876406ad80 2023-10-14T10:37:08.646 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.637+0000 7f8762ffd700 1 -- 172.21.15.67:0/681748441 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f875c00bcc0 con 0x7f876406ad80 2023-10-14T10:37:08.646 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.638+0000 7f876a9d6700 1 -- 172.21.15.67:0/681748441 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f876406ad80 msgr2=0x7f8764071af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:08.647 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.638+0000 7f876a9d6700 1 --2- 172.21.15.67:0/681748441 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f876406ad80 0x7f8764071af0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f87640d1560 tx=0x7f875c00b330 comp rx=0 tx=0).stop 2023-10-14T10:37:08.647 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.638+0000 7f876a9d6700 1 -- 172.21.15.67:0/681748441 shutdown_connections 2023-10-14T10:37:08.647 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.638+0000 7f876a9d6700 1 --2- 172.21.15.67:0/681748441 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f876406ad80 0x7f8764071af0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:08.647 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.638+0000 7f876a9d6700 1 --2- 172.21.15.67:0/681748441 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f876415f130 0x7f87640cede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:08.648 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.638+0000 7f876a9d6700 1 --2- 172.21.15.67:0/681748441 >> v2:172.21.15.72:3300/0 conn(0x7f87640b28b0 0x7f876406a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:08.648 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.638+0000 7f876a9d6700 1 -- 172.21.15.67:0/681748441 >> 172.21.15.67:0/681748441 conn(0x7f876406dae0 msgr2=0x7f876406def0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:37:08.648 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.638+0000 7f876a9d6700 1 -- 172.21.15.67:0/681748441 shutdown_connections 2023-10-14T10:37:08.648 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.638+0000 7f876a9d6700 1 -- 172.21.15.67:0/681748441 wait complete. 2023-10-14T10:37:08.651 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.639+0000 7f876a9d6700 1 Processor -- start 2023-10-14T10:37:08.652 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.640+0000 7f876a9d6700 1 -- start start 2023-10-14T10:37:08.652 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.640+0000 7f876a9d6700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f876406ad80 0x7f8764153ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:08.652 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.640+0000 7f876a9d6700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f87640b28b0 0x7f87640c9f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:08.653 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.640+0000 7f876a9d6700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f876415f130 0x7f87640ca490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:08.653 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.640+0000 7f876a9d6700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87640c99e0 con 0x7f876415f130 2023-10-14T10:37:08.653 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.640+0000 7f876a9d6700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87641541e0 con 0x7f876406ad80 2023-10-14T10:37:08.653 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.640+0000 7f876a9d6700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87640c4be0 con 0x7f87640b28b0 2023-10-14T10:37:08.654 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.640+0000 7f8768f73700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f876415f130 0x7f87640ca490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:08.654 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.641+0000 7f8768f73700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f876415f130 0x7f87640ca490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45968/0 (socket says 172.21.15.67:45968) 2023-10-14T10:37:08.654 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.641+0000 7f8768f73700 1 -- 172.21.15.67:0/2471647668 learned_addr learned my addr 172.21.15.67:0/2471647668 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:37:08.654 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.641+0000 7f87637fe700 1 --2- 172.21.15.67:0/2471647668 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f87640b28b0 0x7f87640c9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:08.655 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.641+0000 7f8763fff700 1 --2- 172.21.15.67:0/2471647668 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f876406ad80 0x7f8764153ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:08.655 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.641+0000 7f8768f73700 1 -- 172.21.15.67:0/2471647668 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f87640b28b0 msgr2=0x7f87640c9f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:08.655 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.641+0000 7f8768f73700 1 --2- 172.21.15.67:0/2471647668 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f87640b28b0 0x7f87640c9f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:08.655 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.641+0000 7f8768f73700 1 -- 172.21.15.67:0/2471647668 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f876406ad80 msgr2=0x7f8764153ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:08.656 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.641+0000 7f8768f73700 1 --2- 172.21.15.67:0/2471647668 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f876406ad80 0x7f8764153ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:08.656 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.641+0000 7f8768f73700 1 -- 172.21.15.67:0/2471647668 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f875c009b30 con 0x7f876415f130 2023-10-14T10:37:08.656 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.641+0000 7f8768f73700 1 --2- 172.21.15.67:0/2471647668 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f876415f130 0x7f87640ca490 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f87640ce050 tx=0x7f875c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:08.657 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.641+0000 7f8760ff9700 1 -- 172.21.15.67:0/2471647668 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f875c0099b0 con 0x7f876415f130 2023-10-14T10:37:08.657 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.641+0000 7f8760ff9700 1 -- 172.21.15.67:0/2471647668 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f875c00ce80 con 0x7f876415f130 2023-10-14T10:37:08.657 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.641+0000 7f876a9d6700 1 -- 172.21.15.67:0/2471647668 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87640c4ec0 con 0x7f876415f130 2023-10-14T10:37:08.657 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.641+0000 7f8760ff9700 1 -- 172.21.15.67:0/2471647668 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f875c004490 con 0x7f876415f130 2023-10-14T10:37:08.658 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.641+0000 7f876a9d6700 1 -- 172.21.15.67:0/2471647668 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87640c5480 con 0x7f876415f130 2023-10-14T10:37:08.658 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.642+0000 7f876a9d6700 1 -- 172.21.15.67:0/2471647668 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8764067d20 con 0x7f876415f130 2023-10-14T10:37:08.658 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.643+0000 7f8760ff9700 1 -- 172.21.15.67:0/2471647668 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f875c0045f0 con 0x7f876415f130 2023-10-14T10:37:08.659 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.643+0000 7f8760ff9700 1 --2- 172.21.15.67:0/2471647668 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f874406af00 0x7f874406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:08.659 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.643+0000 7f8760ff9700 1 -- 172.21.15.67:0/2471647668 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5729+0+0 (secure 0 0 0) 0x7f875c08a530 con 0x7f876415f130 2023-10-14T10:37:08.659 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.643+0000 7f8763fff700 1 --2- 172.21.15.67:0/2471647668 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f874406af00 0x7f874406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:08.659 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.644+0000 7f8763fff700 1 --2- 172.21.15.67:0/2471647668 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f874406af00 0x7f874406d3c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8754009be0 tx=0x7f8754009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:08.660 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.648+0000 7f8760ff9700 1 -- 172.21.15.67:0/2471647668 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f875c055650 con 0x7f876415f130 2023-10-14T10:37:08.845 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.843+0000 7f876a9d6700 1 -- 172.21.15.67:0/2471647668 --> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f87640cd740 con 0x7f874406af00 2023-10-14T10:37:08.853 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.851+0000 7f8760ff9700 1 -- 172.21.15.67:0/2471647668 <== mgr.4100 v2:172.21.15.72:6832/703488382 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289547 (secure 0 0 0) 0x7f8754011860 con 0x7f874406af00 2023-10-14T10:37:08.857 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-14T10:37:08.859 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.855+0000 7f876a9d6700 1 -- 172.21.15.67:0/2471647668 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f874406af00 msgr2=0x7f874406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:08.860 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.855+0000 7f876a9d6700 1 --2- 172.21.15.67:0/2471647668 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f874406af00 0x7f874406d3c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8754009be0 tx=0x7f8754009210 comp rx=0 tx=0).stop 2023-10-14T10:37:08.860 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.855+0000 7f876a9d6700 1 -- 172.21.15.67:0/2471647668 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f876415f130 msgr2=0x7f87640ca490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:08.860 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.855+0000 7f876a9d6700 1 --2- 172.21.15.67:0/2471647668 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f876415f130 0x7f87640ca490 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f87640ce050 tx=0x7f875c004ad0 comp rx=0 tx=0).stop 2023-10-14T10:37:08.860 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.855+0000 7f876a9d6700 1 -- 172.21.15.67:0/2471647668 shutdown_connections 2023-10-14T10:37:08.861 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.855+0000 7f876a9d6700 1 --2- 172.21.15.67:0/2471647668 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f874406af00 0x7f874406d3c0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:08.861 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.855+0000 7f876a9d6700 1 --2- 172.21.15.67:0/2471647668 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f876415f130 0x7f87640ca490 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:08.861 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.855+0000 7f876a9d6700 1 --2- 172.21.15.67:0/2471647668 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f876406ad80 0x7f8764153ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:08.861 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.855+0000 7f876a9d6700 1 --2- 172.21.15.67:0/2471647668 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f87640b28b0 0x7f87640c9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:08.862 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.855+0000 7f876a9d6700 1 -- 172.21.15.67:0/2471647668 >> 172.21.15.67:0/2471647668 conn(0x7f876406dae0 msgr2=0x7f8764154410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:37:08.862 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.855+0000 7f876a9d6700 1 -- 172.21.15.67:0/2471647668 shutdown_connections 2023-10-14T10:37:08.862 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:08.855+0000 7f876a9d6700 1 -- 172.21.15.67:0/2471647668 wait complete. 2023-10-14T10:37:08.863 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2023-10-14T10:37:08.869 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":62,"stamp":"2023-10-14T10:37:07.770696+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220280,"kb_used_data":4104,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754440,"statfs":{"total":773094113280,"available":772868546560,"internally_reserved":0,"allocated":4202496,"data_stored":2322644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12742,"internal_metadata":221236794},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002093"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.787560+0000","last_change":"2023-10-14T10:37:00.075102+0000","last_active":"2023-10-14T10:37:01.787560+0000","last_peered":"2023-10-14T10:37:01.787560+0000","last_clean":"2023-10-14T10:37:01.787560+0000","last_became_active":"2023-10-14T10:37:00.074851+0000","last_became_peered":"2023-10-14T10:37:00.074851+0000","last_unstale":"2023-10-14T10:37:01.787560+0000","last_undegraded":"2023-10-14T10:37:01.787560+0000","last_fullsized":"2023-10-14T10:37:01.787560+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T21:39:35.587728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788402+0000","last_change":"2023-10-14T10:37:00.068871+0000","last_active":"2023-10-14T10:37:01.788402+0000","last_peered":"2023-10-14T10:37:01.788402+0000","last_clean":"2023-10-14T10:37:01.788402+0000","last_became_active":"2023-10-14T10:37:00.068586+0000","last_became_peered":"2023-10-14T10:37:00.068586+0000","last_unstale":"2023-10-14T10:37:01.788402+0000","last_undegraded":"2023-10-14T10:37:01.788402+0000","last_fullsized":"2023-10-14T10:37:01.788402+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T12:47:24.725885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.062215+0000","last_change":"2023-10-14T10:37:00.080988+0000","last_active":"2023-10-14T10:37:01.062215+0000","last_peered":"2023-10-14T10:37:01.062215+0000","last_clean":"2023-10-14T10:37:01.062215+0000","last_became_active":"2023-10-14T10:37:00.080476+0000","last_became_peered":"2023-10-14T10:37:00.080476+0000","last_unstale":"2023-10-14T10:37:01.062215+0000","last_undegraded":"2023-10-14T10:37:01.062215+0000","last_fullsized":"2023-10-14T10:37:01.062215+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T10:48:03.043383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.059874+0000","last_change":"2023-10-14T10:37:00.077598+0000","last_active":"2023-10-14T10:37:01.059874+0000","last_peered":"2023-10-14T10:37:01.059874+0000","last_clean":"2023-10-14T10:37:01.059874+0000","last_became_active":"2023-10-14T10:37:00.077322+0000","last_became_peered":"2023-10-14T10:37:00.077322+0000","last_unstale":"2023-10-14T10:37:01.059874+0000","last_undegraded":"2023-10-14T10:37:01.059874+0000","last_fullsized":"2023-10-14T10:37:01.059874+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:28:04.536948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788360+0000","last_change":"2023-10-14T10:37:00.068342+0000","last_active":"2023-10-14T10:37:01.788360+0000","last_peered":"2023-10-14T10:37:01.788360+0000","last_clean":"2023-10-14T10:37:01.788360+0000","last_became_active":"2023-10-14T10:37:00.068055+0000","last_became_peered":"2023-10-14T10:37:00.068055+0000","last_unstale":"2023-10-14T10:37:01.788360+0000","last_undegraded":"2023-10-14T10:37:01.788360+0000","last_fullsized":"2023-10-14T10:37:01.788360+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T12:52:16.372161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788917+0000","last_change":"2023-10-14T10:37:00.072629+0000","last_active":"2023-10-14T10:37:01.788917+0000","last_peered":"2023-10-14T10:37:01.788917+0000","last_clean":"2023-10-14T10:37:01.788917+0000","last_became_active":"2023-10-14T10:37:00.072345+0000","last_became_peered":"2023-10-14T10:37:00.072345+0000","last_unstale":"2023-10-14T10:37:01.788917+0000","last_undegraded":"2023-10-14T10:37:01.788917+0000","last_fullsized":"2023-10-14T10:37:01.788917+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T17:09:21.565688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.061635+0000","last_change":"2023-10-14T10:37:00.073019+0000","last_active":"2023-10-14T10:37:01.061635+0000","last_peered":"2023-10-14T10:37:01.061635+0000","last_clean":"2023-10-14T10:37:01.061635+0000","last_became_active":"2023-10-14T10:37:00.072368+0000","last_became_peered":"2023-10-14T10:37:00.072368+0000","last_unstale":"2023-10-14T10:37:01.061635+0000","last_undegraded":"2023-10-14T10:37:01.061635+0000","last_fullsized":"2023-10-14T10:37:01.061635+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:31:16.127752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.062210+0000","last_change":"2023-10-14T10:37:00.074091+0000","last_active":"2023-10-14T10:37:01.062210+0000","last_peered":"2023-10-14T10:37:01.062210+0000","last_clean":"2023-10-14T10:37:01.062210+0000","last_became_active":"2023-10-14T10:37:00.073792+0000","last_became_peered":"2023-10-14T10:37:00.073792+0000","last_unstale":"2023-10-14T10:37:01.062210+0000","last_undegraded":"2023-10-14T10:37:01.062210+0000","last_fullsized":"2023-10-14T10:37:01.062210+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T17:27:50.406084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.489588+0000","last_change":"2023-10-14T10:37:00.081745+0000","last_active":"2023-10-14T10:37:01.489588+0000","last_peered":"2023-10-14T10:37:01.489588+0000","last_clean":"2023-10-14T10:37:01.489588+0000","last_became_active":"2023-10-14T10:37:00.081451+0000","last_became_peered":"2023-10-14T10:37:00.081451+0000","last_unstale":"2023-10-14T10:37:01.489588+0000","last_undegraded":"2023-10-14T10:37:01.489588+0000","last_fullsized":"2023-10-14T10:37:01.489588+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:18:45.297555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.060294+0000","last_change":"2023-10-14T10:37:00.071138+0000","last_active":"2023-10-14T10:37:01.060294+0000","last_peered":"2023-10-14T10:37:01.060294+0000","last_clean":"2023-10-14T10:37:01.060294+0000","last_became_active":"2023-10-14T10:37:00.070667+0000","last_became_peered":"2023-10-14T10:37:00.070667+0000","last_unstale":"2023-10-14T10:37:01.060294+0000","last_undegraded":"2023-10-14T10:37:01.060294+0000","last_fullsized":"2023-10-14T10:37:01.060294+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:52:37.894720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.062228+0000","last_change":"2023-10-14T10:37:00.073322+0000","last_active":"2023-10-14T10:37:01.062228+0000","last_peered":"2023-10-14T10:37:01.062228+0000","last_clean":"2023-10-14T10:37:01.062228+0000","last_became_active":"2023-10-14T10:37:00.072639+0000","last_became_peered":"2023-10-14T10:37:00.072639+0000","last_unstale":"2023-10-14T10:37:01.062228+0000","last_undegraded":"2023-10-14T10:37:01.062228+0000","last_fullsized":"2023-10-14T10:37:01.062228+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T18:57:50.136605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.061858+0000","last_change":"2023-10-14T10:37:00.069681+0000","last_active":"2023-10-14T10:37:01.061858+0000","last_peered":"2023-10-14T10:37:01.061858+0000","last_clean":"2023-10-14T10:37:01.061858+0000","last_became_active":"2023-10-14T10:37:00.069373+0000","last_became_peered":"2023-10-14T10:37:00.069373+0000","last_unstale":"2023-10-14T10:37:01.061858+0000","last_undegraded":"2023-10-14T10:37:01.061858+0000","last_fullsized":"2023-10-14T10:37:01.061858+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:31:16.127752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788687+0000","last_change":"2023-10-14T10:37:00.074917+0000","last_active":"2023-10-14T10:37:01.788687+0000","last_peered":"2023-10-14T10:37:01.788687+0000","last_clean":"2023-10-14T10:37:01.788687+0000","last_became_active":"2023-10-14T10:37:00.074473+0000","last_became_peered":"2023-10-14T10:37:00.074473+0000","last_unstale":"2023-10-14T10:37:01.788687+0000","last_undegraded":"2023-10-14T10:37:01.788687+0000","last_fullsized":"2023-10-14T10:37:01.788687+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T20:20:02.187767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788602+0000","last_change":"2023-10-14T10:37:00.073986+0000","last_active":"2023-10-14T10:37:01.788602+0000","last_peered":"2023-10-14T10:37:01.788602+0000","last_clean":"2023-10-14T10:37:01.788602+0000","last_became_active":"2023-10-14T10:37:00.073651+0000","last_became_peered":"2023-10-14T10:37:00.073651+0000","last_unstale":"2023-10-14T10:37:01.788602+0000","last_undegraded":"2023-10-14T10:37:01.788602+0000","last_fullsized":"2023-10-14T10:37:01.788602+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T10:48:03.043383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.061624+0000","last_change":"2023-10-14T10:37:00.072277+0000","last_active":"2023-10-14T10:37:01.061624+0000","last_peered":"2023-10-14T10:37:01.061624+0000","last_clean":"2023-10-14T10:37:01.061624+0000","last_became_active":"2023-10-14T10:37:00.071999+0000","last_became_peered":"2023-10-14T10:37:00.071999+0000","last_unstale":"2023-10-14T10:37:01.061624+0000","last_undegraded":"2023-10-14T10:37:01.061624+0000","last_fullsized":"2023-10-14T10:37:01.061624+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:18:45.297555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.062458+0000","last_change":"2023-10-14T10:37:00.072492+0000","last_active":"2023-10-14T10:37:01.062458+0000","last_peered":"2023-10-14T10:37:01.062458+0000","last_clean":"2023-10-14T10:37:01.062458+0000","last_became_active":"2023-10-14T10:37:00.071509+0000","last_became_peered":"2023-10-14T10:37:00.071509+0000","last_unstale":"2023-10-14T10:37:01.062458+0000","last_undegraded":"2023-10-14T10:37:01.062458+0000","last_fullsized":"2023-10-14T10:37:01.062458+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:51:00.167498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.062381+0000","last_change":"2023-10-14T10:37:00.072597+0000","last_active":"2023-10-14T10:37:01.062381+0000","last_peered":"2023-10-14T10:37:01.062381+0000","last_clean":"2023-10-14T10:37:01.062381+0000","last_became_active":"2023-10-14T10:37:00.071946+0000","last_became_peered":"2023-10-14T10:37:00.071946+0000","last_unstale":"2023-10-14T10:37:01.062381+0000","last_undegraded":"2023-10-14T10:37:01.062381+0000","last_fullsized":"2023-10-14T10:37:01.062381+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:06:28.391046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788267+0000","last_change":"2023-10-14T10:37:00.080581+0000","last_active":"2023-10-14T10:37:01.788267+0000","last_peered":"2023-10-14T10:37:01.788267+0000","last_clean":"2023-10-14T10:37:01.788267+0000","last_became_active":"2023-10-14T10:37:00.080288+0000","last_became_peered":"2023-10-14T10:37:00.080288+0000","last_unstale":"2023-10-14T10:37:01.788267+0000","last_undegraded":"2023-10-14T10:37:01.788267+0000","last_fullsized":"2023-10-14T10:37:01.788267+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T18:57:50.136605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.787886+0000","last_change":"2023-10-14T10:37:00.080484+0000","last_active":"2023-10-14T10:37:01.787886+0000","last_peered":"2023-10-14T10:37:01.787886+0000","last_clean":"2023-10-14T10:37:01.787886+0000","last_became_active":"2023-10-14T10:37:00.080186+0000","last_became_peered":"2023-10-14T10:37:00.080186+0000","last_unstale":"2023-10-14T10:37:01.787886+0000","last_undegraded":"2023-10-14T10:37:01.787886+0000","last_fullsized":"2023-10-14T10:37:01.787886+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:17:54.730152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788501+0000","last_change":"2023-10-14T10:37:00.072056+0000","last_active":"2023-10-14T10:37:01.788501+0000","last_peered":"2023-10-14T10:37:01.788501+0000","last_clean":"2023-10-14T10:37:01.788501+0000","last_became_active":"2023-10-14T10:37:00.071796+0000","last_became_peered":"2023-10-14T10:37:00.071796+0000","last_unstale":"2023-10-14T10:37:01.788501+0000","last_undegraded":"2023-10-14T10:37:01.788501+0000","last_fullsized":"2023-10-14T10:37:01.788501+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T20:39:07.030985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.489206+0000","last_change":"2023-10-14T10:37:00.080796+0000","last_active":"2023-10-14T10:37:01.489206+0000","last_peered":"2023-10-14T10:37:01.489206+0000","last_clean":"2023-10-14T10:37:01.489206+0000","last_became_active":"2023-10-14T10:37:00.080526+0000","last_became_peered":"2023-10-14T10:37:00.080526+0000","last_unstale":"2023-10-14T10:37:01.489206+0000","last_undegraded":"2023-10-14T10:37:01.489206+0000","last_fullsized":"2023-10-14T10:37:01.489206+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:45:25.303185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.489168+0000","last_change":"2023-10-14T10:37:00.080350+0000","last_active":"2023-10-14T10:37:01.489168+0000","last_peered":"2023-10-14T10:37:01.489168+0000","last_clean":"2023-10-14T10:37:01.489168+0000","last_became_active":"2023-10-14T10:37:00.080062+0000","last_became_peered":"2023-10-14T10:37:00.080062+0000","last_unstale":"2023-10-14T10:37:01.489168+0000","last_undegraded":"2023-10-14T10:37:01.489168+0000","last_fullsized":"2023-10-14T10:37:01.489168+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:10:46.473503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.787611+0000","last_change":"2023-10-14T10:37:00.071237+0000","last_active":"2023-10-14T10:37:01.787611+0000","last_peered":"2023-10-14T10:37:01.787611+0000","last_clean":"2023-10-14T10:37:01.787611+0000","last_became_active":"2023-10-14T10:37:00.070897+0000","last_became_peered":"2023-10-14T10:37:00.070897+0000","last_unstale":"2023-10-14T10:37:01.787611+0000","last_undegraded":"2023-10-14T10:37:01.787611+0000","last_fullsized":"2023-10-14T10:37:01.787611+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:59:07.208582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.061365+0000","last_change":"2023-10-14T10:37:00.069525+0000","last_active":"2023-10-14T10:37:01.061365+0000","last_peered":"2023-10-14T10:37:01.061365+0000","last_clean":"2023-10-14T10:37:01.061365+0000","last_became_active":"2023-10-14T10:37:00.069246+0000","last_became_peered":"2023-10-14T10:37:00.069246+0000","last_unstale":"2023-10-14T10:37:01.061365+0000","last_undegraded":"2023-10-14T10:37:01.061365+0000","last_fullsized":"2023-10-14T10:37:01.061365+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:52:37.894720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.060407+0000","last_change":"2023-10-14T10:37:00.070144+0000","last_active":"2023-10-14T10:37:01.060407+0000","last_peered":"2023-10-14T10:37:01.060407+0000","last_clean":"2023-10-14T10:37:01.060407+0000","last_became_active":"2023-10-14T10:37:00.069871+0000","last_became_peered":"2023-10-14T10:37:00.069871+0000","last_unstale":"2023-10-14T10:37:01.060407+0000","last_undegraded":"2023-10-14T10:37:01.060407+0000","last_fullsized":"2023-10-14T10:37:01.060407+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:22:32.154663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788109+0000","last_change":"2023-10-14T10:37:00.076474+0000","last_active":"2023-10-14T10:37:01.788109+0000","last_peered":"2023-10-14T10:37:01.788109+0000","last_clean":"2023-10-14T10:37:01.788109+0000","last_became_active":"2023-10-14T10:37:00.076169+0000","last_became_peered":"2023-10-14T10:37:00.076169+0000","last_unstale":"2023-10-14T10:37:01.788109+0000","last_undegraded":"2023-10-14T10:37:01.788109+0000","last_fullsized":"2023-10-14T10:37:01.788109+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T14:52:18.060480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.060360+0000","last_change":"2023-10-14T10:37:00.076568+0000","last_active":"2023-10-14T10:37:01.060360+0000","last_peered":"2023-10-14T10:37:01.060360+0000","last_clean":"2023-10-14T10:37:01.060360+0000","last_became_active":"2023-10-14T10:37:00.076263+0000","last_became_peered":"2023-10-14T10:37:00.076263+0000","last_unstale":"2023-10-14T10:37:01.060360+0000","last_undegraded":"2023-10-14T10:37:01.060360+0000","last_fullsized":"2023-10-14T10:37:01.060360+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:18:45.297555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.061994+0000","last_change":"2023-10-14T10:37:00.079439+0000","last_active":"2023-10-14T10:37:01.061994+0000","last_peered":"2023-10-14T10:37:01.061994+0000","last_clean":"2023-10-14T10:37:01.061994+0000","last_became_active":"2023-10-14T10:37:00.079178+0000","last_became_peered":"2023-10-14T10:37:00.079178+0000","last_unstale":"2023-10-14T10:37:01.061994+0000","last_undegraded":"2023-10-14T10:37:01.061994+0000","last_fullsized":"2023-10-14T10:37:01.061994+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:17:54.730152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.489206+0000","last_change":"2023-10-14T10:37:00.080675+0000","last_active":"2023-10-14T10:37:01.489206+0000","last_peered":"2023-10-14T10:37:01.489206+0000","last_clean":"2023-10-14T10:37:01.489206+0000","last_became_active":"2023-10-14T10:37:00.080171+0000","last_became_peered":"2023-10-14T10:37:00.080171+0000","last_unstale":"2023-10-14T10:37:01.489206+0000","last_undegraded":"2023-10-14T10:37:01.489206+0000","last_fullsized":"2023-10-14T10:37:01.489206+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:28:04.536948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.061814+0000","last_change":"2023-10-14T10:37:00.072155+0000","last_active":"2023-10-14T10:37:01.061814+0000","last_peered":"2023-10-14T10:37:01.061814+0000","last_clean":"2023-10-14T10:37:01.061814+0000","last_became_active":"2023-10-14T10:37:00.071882+0000","last_became_peered":"2023-10-14T10:37:00.071882+0000","last_unstale":"2023-10-14T10:37:01.061814+0000","last_undegraded":"2023-10-14T10:37:01.061814+0000","last_fullsized":"2023-10-14T10:37:01.061814+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T10:48:03.043383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.787458+0000","last_change":"2023-10-14T10:37:00.081816+0000","last_active":"2023-10-14T10:37:01.787458+0000","last_peered":"2023-10-14T10:37:01.787458+0000","last_clean":"2023-10-14T10:37:01.787458+0000","last_became_active":"2023-10-14T10:37:00.081312+0000","last_became_peered":"2023-10-14T10:37:00.081312+0000","last_unstale":"2023-10-14T10:37:01.787458+0000","last_undegraded":"2023-10-14T10:37:01.787458+0000","last_fullsized":"2023-10-14T10:37:01.787458+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:58:56.558710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.061373+0000","last_change":"2023-10-14T10:37:00.078914+0000","last_active":"2023-10-14T10:37:01.061373+0000","last_peered":"2023-10-14T10:37:01.061373+0000","last_clean":"2023-10-14T10:37:01.061373+0000","last_became_active":"2023-10-14T10:37:00.078441+0000","last_became_peered":"2023-10-14T10:37:00.078441+0000","last_unstale":"2023-10-14T10:37:01.061373+0000","last_undegraded":"2023-10-14T10:37:01.061373+0000","last_fullsized":"2023-10-14T10:37:01.061373+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:22:32.154663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.787754+0000","last_change":"2023-10-14T10:37:00.071246+0000","last_active":"2023-10-14T10:37:01.787754+0000","last_peered":"2023-10-14T10:37:01.787754+0000","last_clean":"2023-10-14T10:37:01.787754+0000","last_became_active":"2023-10-14T10:37:00.070945+0000","last_became_peered":"2023-10-14T10:37:00.070945+0000","last_unstale":"2023-10-14T10:37:01.787754+0000","last_undegraded":"2023-10-14T10:37:01.787754+0000","last_fullsized":"2023-10-14T10:37:01.787754+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:30:57.784792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.787645+0000","last_change":"2023-10-14T10:36:59.066772+0000","last_active":"2023-10-14T10:37:01.787645+0000","last_peered":"2023-10-14T10:37:01.787645+0000","last_clean":"2023-10-14T10:37:01.787645+0000","last_became_active":"2023-10-14T10:36:59.066488+0000","last_became_peered":"2023-10-14T10:36:59.066488+0000","last_unstale":"2023-10-14T10:37:01.787645+0000","last_undegraded":"2023-10-14T10:37:01.787645+0000","last_fullsized":"2023-10-14T10:37:01.787645+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:39:01.994329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.787569+0000","last_change":"2023-10-14T10:37:00.077997+0000","last_active":"2023-10-14T10:37:01.787569+0000","last_peered":"2023-10-14T10:37:01.787569+0000","last_clean":"2023-10-14T10:37:01.787569+0000","last_became_active":"2023-10-14T10:37:00.077734+0000","last_became_peered":"2023-10-14T10:37:00.077734+0000","last_unstale":"2023-10-14T10:37:01.787569+0000","last_undegraded":"2023-10-14T10:37:01.787569+0000","last_fullsized":"2023-10-14T10:37:01.787569+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:28:45.821377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.787856+0000","last_change":"2023-10-14T10:36:59.070768+0000","last_active":"2023-10-14T10:37:01.787856+0000","last_peered":"2023-10-14T10:37:01.787856+0000","last_clean":"2023-10-14T10:37:01.787856+0000","last_became_active":"2023-10-14T10:36:59.070463+0000","last_became_peered":"2023-10-14T10:36:59.070463+0000","last_unstale":"2023-10-14T10:37:01.787856+0000","last_undegraded":"2023-10-14T10:37:01.787856+0000","last_fullsized":"2023-10-14T10:37:01.787856+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:08:05.335325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.489357+0000","last_change":"2023-10-14T10:37:00.080768+0000","last_active":"2023-10-14T10:37:01.489357+0000","last_peered":"2023-10-14T10:37:01.489357+0000","last_clean":"2023-10-14T10:37:01.489357+0000","last_became_active":"2023-10-14T10:37:00.080160+0000","last_became_peered":"2023-10-14T10:37:00.080160+0000","last_unstale":"2023-10-14T10:37:01.489357+0000","last_undegraded":"2023-10-14T10:37:01.489357+0000","last_fullsized":"2023-10-14T10:37:01.489357+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T20:27:28.798883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788006+0000","last_change":"2023-10-14T10:36:59.067900+0000","last_active":"2023-10-14T10:37:01.788006+0000","last_peered":"2023-10-14T10:37:01.788006+0000","last_clean":"2023-10-14T10:37:01.788006+0000","last_became_active":"2023-10-14T10:36:59.067616+0000","last_became_peered":"2023-10-14T10:36:59.067616+0000","last_unstale":"2023-10-14T10:37:01.788006+0000","last_undegraded":"2023-10-14T10:37:01.788006+0000","last_fullsized":"2023-10-14T10:37:01.788006+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:28:44.815010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.060586+0000","last_change":"2023-10-14T10:37:00.079200+0000","last_active":"2023-10-14T10:37:01.060586+0000","last_peered":"2023-10-14T10:37:01.060586+0000","last_clean":"2023-10-14T10:37:01.060586+0000","last_became_active":"2023-10-14T10:37:00.078829+0000","last_became_peered":"2023-10-14T10:37:00.078829+0000","last_unstale":"2023-10-14T10:37:01.060586+0000","last_undegraded":"2023-10-14T10:37:01.060586+0000","last_fullsized":"2023-10-14T10:37:01.060586+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:31:16.127752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788507+0000","last_change":"2023-10-14T10:36:59.065882+0000","last_active":"2023-10-14T10:37:01.788507+0000","last_peered":"2023-10-14T10:37:01.788507+0000","last_clean":"2023-10-14T10:37:01.788507+0000","last_became_active":"2023-10-14T10:36:59.065608+0000","last_became_peered":"2023-10-14T10:36:59.065608+0000","last_unstale":"2023-10-14T10:37:01.788507+0000","last_undegraded":"2023-10-14T10:37:01.788507+0000","last_fullsized":"2023-10-14T10:37:01.788507+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T21:18:46.547776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.062326+0000","last_change":"2023-10-14T10:37:00.077115+0000","last_active":"2023-10-14T10:37:01.062326+0000","last_peered":"2023-10-14T10:37:01.062326+0000","last_clean":"2023-10-14T10:37:01.062326+0000","last_became_active":"2023-10-14T10:37:00.076840+0000","last_became_peered":"2023-10-14T10:37:00.076840+0000","last_unstale":"2023-10-14T10:37:01.062326+0000","last_undegraded":"2023-10-14T10:37:01.062326+0000","last_fullsized":"2023-10-14T10:37:01.062326+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T12:47:24.725885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.489315+0000","last_change":"2023-10-14T10:36:59.069533+0000","last_active":"2023-10-14T10:37:01.489315+0000","last_peered":"2023-10-14T10:37:01.489315+0000","last_clean":"2023-10-14T10:37:01.489315+0000","last_became_active":"2023-10-14T10:36:59.069192+0000","last_became_peered":"2023-10-14T10:36:59.069192+0000","last_unstale":"2023-10-14T10:37:01.489315+0000","last_undegraded":"2023-10-14T10:37:01.489315+0000","last_fullsized":"2023-10-14T10:37:01.489315+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T21:39:34.581362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.061634+0000","last_change":"2023-10-14T10:37:00.068786+0000","last_active":"2023-10-14T10:37:01.061634+0000","last_peered":"2023-10-14T10:37:01.061634+0000","last_clean":"2023-10-14T10:37:01.061634+0000","last_became_active":"2023-10-14T10:37:00.068517+0000","last_became_peered":"2023-10-14T10:37:00.068517+0000","last_unstale":"2023-10-14T10:37:01.061634+0000","last_undegraded":"2023-10-14T10:37:01.061634+0000","last_fullsized":"2023-10-14T10:37:01.061634+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:22:32.154663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.489303+0000","last_change":"2023-10-14T10:36:59.069445+0000","last_active":"2023-10-14T10:37:01.489303+0000","last_peered":"2023-10-14T10:37:01.489303+0000","last_clean":"2023-10-14T10:37:01.489303+0000","last_became_active":"2023-10-14T10:36:59.069151+0000","last_became_peered":"2023-10-14T10:36:59.069151+0000","last_unstale":"2023-10-14T10:37:01.489303+0000","last_undegraded":"2023-10-14T10:37:01.489303+0000","last_fullsized":"2023-10-14T10:37:01.489303+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:54:15.011628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.488953+0000","last_change":"2023-10-14T10:37:00.076984+0000","last_active":"2023-10-14T10:37:01.488953+0000","last_peered":"2023-10-14T10:37:01.488953+0000","last_clean":"2023-10-14T10:37:01.488953+0000","last_became_active":"2023-10-14T10:37:00.076706+0000","last_became_peered":"2023-10-14T10:37:00.076706+0000","last_unstale":"2023-10-14T10:37:01.488953+0000","last_undegraded":"2023-10-14T10:37:01.488953+0000","last_fullsized":"2023-10-14T10:37:01.488953+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:30:57.784792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.787709+0000","last_change":"2023-10-14T10:36:59.062324+0000","last_active":"2023-10-14T10:37:01.787709+0000","last_peered":"2023-10-14T10:37:01.787709+0000","last_clean":"2023-10-14T10:37:01.787709+0000","last_became_active":"2023-10-14T10:36:59.062075+0000","last_became_peered":"2023-10-14T10:36:59.062075+0000","last_unstale":"2023-10-14T10:37:01.787709+0000","last_undegraded":"2023-10-14T10:37:01.787709+0000","last_fullsized":"2023-10-14T10:37:01.787709+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:45:50.919718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.059848+0000","last_change":"2023-10-14T10:37:00.067486+0000","last_active":"2023-10-14T10:37:01.059848+0000","last_peered":"2023-10-14T10:37:01.059848+0000","last_clean":"2023-10-14T10:37:01.059848+0000","last_became_active":"2023-10-14T10:37:00.067187+0000","last_became_peered":"2023-10-14T10:37:00.067187+0000","last_unstale":"2023-10-14T10:37:01.059848+0000","last_undegraded":"2023-10-14T10:37:01.059848+0000","last_fullsized":"2023-10-14T10:37:01.059848+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:10:46.473503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.488998+0000","last_change":"2023-10-14T10:36:59.066330+0000","last_active":"2023-10-14T10:37:01.488998+0000","last_peered":"2023-10-14T10:37:01.488998+0000","last_clean":"2023-10-14T10:37:01.488998+0000","last_became_active":"2023-10-14T10:36:59.064990+0000","last_became_peered":"2023-10-14T10:36:59.064990+0000","last_unstale":"2023-10-14T10:37:01.488998+0000","last_undegraded":"2023-10-14T10:37:01.488998+0000","last_fullsized":"2023-10-14T10:37:01.488998+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T12:15:16.823069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.061974+0000","last_change":"2023-10-14T10:37:00.077465+0000","last_active":"2023-10-14T10:37:01.061974+0000","last_peered":"2023-10-14T10:37:01.061974+0000","last_clean":"2023-10-14T10:37:01.061974+0000","last_became_active":"2023-10-14T10:37:00.077152+0000","last_became_peered":"2023-10-14T10:37:00.077152+0000","last_unstale":"2023-10-14T10:37:01.061974+0000","last_undegraded":"2023-10-14T10:37:01.061974+0000","last_fullsized":"2023-10-14T10:37:01.061974+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:51:00.167498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788575+0000","last_change":"2023-10-14T10:36:59.061903+0000","last_active":"2023-10-14T10:37:01.788575+0000","last_peered":"2023-10-14T10:37:01.788575+0000","last_clean":"2023-10-14T10:37:01.788575+0000","last_became_active":"2023-10-14T10:36:59.061597+0000","last_became_peered":"2023-10-14T10:36:59.061597+0000","last_unstale":"2023-10-14T10:37:01.788575+0000","last_undegraded":"2023-10-14T10:37:01.788575+0000","last_fullsized":"2023-10-14T10:37:01.788575+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:22:31.148297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788727+0000","last_change":"2023-10-14T10:37:00.076213+0000","last_active":"2023-10-14T10:37:01.788727+0000","last_peered":"2023-10-14T10:37:01.788727+0000","last_clean":"2023-10-14T10:37:01.788727+0000","last_became_active":"2023-10-14T10:37:00.075889+0000","last_became_peered":"2023-10-14T10:37:00.075889+0000","last_unstale":"2023-10-14T10:37:01.788727+0000","last_undegraded":"2023-10-14T10:37:01.788727+0000","last_fullsized":"2023-10-14T10:37:01.788727+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T14:19:22.310866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.787968+0000","last_change":"2023-10-14T10:36:59.070183+0000","last_active":"2023-10-14T10:37:01.787968+0000","last_peered":"2023-10-14T10:37:01.787968+0000","last_clean":"2023-10-14T10:37:01.787968+0000","last_became_active":"2023-10-14T10:36:59.069731+0000","last_became_peered":"2023-10-14T10:36:59.069731+0000","last_unstale":"2023-10-14T10:37:01.787968+0000","last_undegraded":"2023-10-14T10:37:01.787968+0000","last_fullsized":"2023-10-14T10:37:01.787968+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:58:55.552343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788210+0000","last_change":"2023-10-14T10:37:00.072592+0000","last_active":"2023-10-14T10:37:01.788210+0000","last_peered":"2023-10-14T10:37:01.788210+0000","last_clean":"2023-10-14T10:37:01.788210+0000","last_became_active":"2023-10-14T10:37:00.072288+0000","last_became_peered":"2023-10-14T10:37:00.072288+0000","last_unstale":"2023-10-14T10:37:01.788210+0000","last_undegraded":"2023-10-14T10:37:01.788210+0000","last_fullsized":"2023-10-14T10:37:01.788210+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T17:27:50.406084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.062406+0000","last_change":"2023-10-14T10:36:59.062120+0000","last_active":"2023-10-14T10:37:01.062406+0000","last_peered":"2023-10-14T10:37:01.062406+0000","last_clean":"2023-10-14T10:37:01.062406+0000","last_became_active":"2023-10-14T10:36:59.061700+0000","last_became_peered":"2023-10-14T10:36:59.061700+0000","last_unstale":"2023-10-14T10:37:01.062406+0000","last_undegraded":"2023-10-14T10:37:01.062406+0000","last_fullsized":"2023-10-14T10:37:01.062406+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:45:24.296819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.062364+0000","last_change":"2023-10-14T10:37:00.074977+0000","last_active":"2023-10-14T10:37:01.062364+0000","last_peered":"2023-10-14T10:37:01.062364+0000","last_clean":"2023-10-14T10:37:01.062364+0000","last_became_active":"2023-10-14T10:37:00.074700+0000","last_became_peered":"2023-10-14T10:37:00.074700+0000","last_unstale":"2023-10-14T10:37:01.062364+0000","last_undegraded":"2023-10-14T10:37:01.062364+0000","last_fullsized":"2023-10-14T10:37:01.062364+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T18:57:50.136605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788323+0000","last_change":"2023-10-14T10:36:59.069400+0000","last_active":"2023-10-14T10:37:01.788323+0000","last_peered":"2023-10-14T10:37:01.788323+0000","last_clean":"2023-10-14T10:37:01.788323+0000","last_became_active":"2023-10-14T10:36:59.069043+0000","last_became_peered":"2023-10-14T10:36:59.069043+0000","last_unstale":"2023-10-14T10:37:01.788323+0000","last_undegraded":"2023-10-14T10:37:01.788323+0000","last_fullsized":"2023-10-14T10:37:01.788323+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:10:45.467136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.060040+0000","last_change":"2023-10-14T10:37:00.068613+0000","last_active":"2023-10-14T10:37:01.060040+0000","last_peered":"2023-10-14T10:37:01.060040+0000","last_clean":"2023-10-14T10:37:01.060040+0000","last_became_active":"2023-10-14T10:37:00.065345+0000","last_became_peered":"2023-10-14T10:37:00.065345+0000","last_unstale":"2023-10-14T10:37:01.060040+0000","last_undegraded":"2023-10-14T10:37:01.060040+0000","last_fullsized":"2023-10-14T10:37:01.060040+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:45:25.303185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.489387+0000","last_change":"2023-10-14T10:36:59.065846+0000","last_active":"2023-10-14T10:37:01.489387+0000","last_peered":"2023-10-14T10:37:01.489387+0000","last_clean":"2023-10-14T10:37:01.489387+0000","last_became_active":"2023-10-14T10:36:59.065569+0000","last_became_peered":"2023-10-14T10:36:59.065569+0000","last_unstale":"2023-10-14T10:37:01.489387+0000","last_undegraded":"2023-10-14T10:37:01.489387+0000","last_fullsized":"2023-10-14T10:37:01.489387+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:28:44.815010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788692+0000","last_change":"2023-10-14T10:37:00.079137+0000","last_active":"2023-10-14T10:37:01.788692+0000","last_peered":"2023-10-14T10:37:01.788692+0000","last_clean":"2023-10-14T10:37:01.788692+0000","last_became_active":"2023-10-14T10:37:00.078844+0000","last_became_peered":"2023-10-14T10:37:00.078844+0000","last_unstale":"2023-10-14T10:37:01.788692+0000","last_undegraded":"2023-10-14T10:37:01.788692+0000","last_fullsized":"2023-10-14T10:37:01.788692+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:30:51.914014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.059920+0000","last_change":"2023-10-14T10:36:59.067245+0000","last_active":"2023-10-14T10:37:01.059920+0000","last_peered":"2023-10-14T10:37:01.059920+0000","last_clean":"2023-10-14T10:37:01.059920+0000","last_became_active":"2023-10-14T10:36:59.066952+0000","last_became_peered":"2023-10-14T10:36:59.066952+0000","last_unstale":"2023-10-14T10:37:01.059920+0000","last_undegraded":"2023-10-14T10:37:01.059920+0000","last_fullsized":"2023-10-14T10:37:01.059920+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T12:08:55.620308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.062454+0000","last_change":"2023-10-14T10:37:00.078646+0000","last_active":"2023-10-14T10:37:01.062454+0000","last_peered":"2023-10-14T10:37:01.062454+0000","last_clean":"2023-10-14T10:37:01.062454+0000","last_became_active":"2023-10-14T10:37:00.078353+0000","last_became_peered":"2023-10-14T10:37:00.078353+0000","last_unstale":"2023-10-14T10:37:01.062454+0000","last_undegraded":"2023-10-14T10:37:01.062454+0000","last_fullsized":"2023-10-14T10:37:01.062454+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:30:51.914014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788938+0000","last_change":"2023-10-14T10:36:59.063118+0000","last_active":"2023-10-14T10:37:01.788938+0000","last_peered":"2023-10-14T10:37:01.788938+0000","last_clean":"2023-10-14T10:37:01.788938+0000","last_became_active":"2023-10-14T10:36:59.062380+0000","last_became_peered":"2023-10-14T10:36:59.062380+0000","last_unstale":"2023-10-14T10:37:01.788938+0000","last_undegraded":"2023-10-14T10:37:01.788938+0000","last_fullsized":"2023-10-14T10:37:01.788938+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T21:39:34.581362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.787507+0000","last_change":"2023-10-14T10:37:00.078077+0000","last_active":"2023-10-14T10:37:01.787507+0000","last_peered":"2023-10-14T10:37:01.787507+0000","last_clean":"2023-10-14T10:37:01.787507+0000","last_became_active":"2023-10-14T10:37:00.077806+0000","last_became_peered":"2023-10-14T10:37:00.077806+0000","last_unstale":"2023-10-14T10:37:01.787507+0000","last_undegraded":"2023-10-14T10:37:01.787507+0000","last_fullsized":"2023-10-14T10:37:01.787507+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:45:51.926085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788260+0000","last_change":"2023-10-14T10:36:59.068012+0000","last_active":"2023-10-14T10:37:01.788260+0000","last_peered":"2023-10-14T10:37:01.788260+0000","last_clean":"2023-10-14T10:37:01.788260+0000","last_became_active":"2023-10-14T10:36:59.067727+0000","last_became_peered":"2023-10-14T10:36:59.067727+0000","last_unstale":"2023-10-14T10:37:01.788260+0000","last_undegraded":"2023-10-14T10:37:01.788260+0000","last_fullsized":"2023-10-14T10:37:01.788260+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:44:49.671921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.489522+0000","last_change":"2023-10-14T10:37:00.076926+0000","last_active":"2023-10-14T10:37:01.489522+0000","last_peered":"2023-10-14T10:37:01.489522+0000","last_clean":"2023-10-14T10:37:01.489522+0000","last_became_active":"2023-10-14T10:37:00.076658+0000","last_became_peered":"2023-10-14T10:37:00.076658+0000","last_unstale":"2023-10-14T10:37:01.489522+0000","last_undegraded":"2023-10-14T10:37:01.489522+0000","last_fullsized":"2023-10-14T10:37:01.489522+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:52:37.894720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.060490+0000","last_change":"2023-10-14T10:36:59.071714+0000","last_active":"2023-10-14T10:37:01.060490+0000","last_peered":"2023-10-14T10:37:01.060490+0000","last_clean":"2023-10-14T10:37:01.060490+0000","last_became_active":"2023-10-14T10:36:59.071406+0000","last_became_peered":"2023-10-14T10:36:59.071406+0000","last_unstale":"2023-10-14T10:37:01.060490+0000","last_undegraded":"2023-10-14T10:37:01.060490+0000","last_fullsized":"2023-10-14T10:37:01.060490+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:59:06.202215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.062345+0000","last_change":"2023-10-14T10:37:00.080060+0000","last_active":"2023-10-14T10:37:01.062345+0000","last_peered":"2023-10-14T10:37:01.062345+0000","last_clean":"2023-10-14T10:37:01.062345+0000","last_became_active":"2023-10-14T10:37:00.079795+0000","last_became_peered":"2023-10-14T10:37:00.079795+0000","last_unstale":"2023-10-14T10:37:01.062345+0000","last_undegraded":"2023-10-14T10:37:01.062345+0000","last_fullsized":"2023-10-14T10:37:01.062345+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T17:27:50.406084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.489554+0000","last_change":"2023-10-14T10:36:59.065271+0000","last_active":"2023-10-14T10:37:01.489554+0000","last_peered":"2023-10-14T10:37:01.489554+0000","last_clean":"2023-10-14T10:37:01.489554+0000","last_became_active":"2023-10-14T10:36:59.064959+0000","last_became_peered":"2023-10-14T10:36:59.064959+0000","last_unstale":"2023-10-14T10:37:01.489554+0000","last_undegraded":"2023-10-14T10:37:01.489554+0000","last_fullsized":"2023-10-14T10:37:01.489554+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T21:56:55.137838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.060192+0000","last_change":"2023-10-14T10:37:00.077751+0000","last_active":"2023-10-14T10:37:01.060192+0000","last_peered":"2023-10-14T10:37:01.060192+0000","last_clean":"2023-10-14T10:37:01.060192+0000","last_became_active":"2023-10-14T10:37:00.077459+0000","last_became_peered":"2023-10-14T10:37:00.077459+0000","last_unstale":"2023-10-14T10:37:01.060192+0000","last_undegraded":"2023-10-14T10:37:01.060192+0000","last_fullsized":"2023-10-14T10:37:01.060192+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T20:27:28.798883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.489473+0000","last_change":"2023-10-14T10:36:59.058893+0000","last_active":"2023-10-14T10:37:01.489473+0000","last_peered":"2023-10-14T10:37:01.489473+0000","last_clean":"2023-10-14T10:37:01.489473+0000","last_became_active":"2023-10-14T10:36:59.058558+0000","last_became_peered":"2023-10-14T10:36:59.058558+0000","last_unstale":"2023-10-14T10:37:01.489473+0000","last_undegraded":"2023-10-14T10:37:01.489473+0000","last_fullsized":"2023-10-14T10:37:01.489473+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:45:50.919718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788693+0000","last_change":"2023-10-14T10:37:00.071398+0000","last_active":"2023-10-14T10:37:01.788693+0000","last_peered":"2023-10-14T10:37:01.788693+0000","last_clean":"2023-10-14T10:37:01.788693+0000","last_became_active":"2023-10-14T10:37:00.071167+0000","last_became_peered":"2023-10-14T10:37:00.071167+0000","last_unstale":"2023-10-14T10:37:01.788693+0000","last_undegraded":"2023-10-14T10:37:01.788693+0000","last_fullsized":"2023-10-14T10:37:01.788693+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:31:12.123956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.489473+0000","last_change":"2023-10-14T10:36:59.069420+0000","last_active":"2023-10-14T10:37:01.489473+0000","last_peered":"2023-10-14T10:37:01.489473+0000","last_clean":"2023-10-14T10:37:01.489473+0000","last_became_active":"2023-10-14T10:36:59.069110+0000","last_became_peered":"2023-10-14T10:36:59.069110+0000","last_unstale":"2023-10-14T10:37:01.489473+0000","last_undegraded":"2023-10-14T10:37:01.489473+0000","last_fullsized":"2023-10-14T10:37:01.489473+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:09:59.812757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788586+0000","last_change":"2023-10-14T10:37:00.077550+0000","last_active":"2023-10-14T10:37:01.788586+0000","last_peered":"2023-10-14T10:37:01.788586+0000","last_clean":"2023-10-14T10:37:01.788586+0000","last_became_active":"2023-10-14T10:37:00.077175+0000","last_became_peered":"2023-10-14T10:37:00.077175+0000","last_unstale":"2023-10-14T10:37:01.788586+0000","last_undegraded":"2023-10-14T10:37:01.788586+0000","last_fullsized":"2023-10-14T10:37:01.788586+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T14:51:19.968923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.787861+0000","last_change":"2023-10-14T10:36:59.067621+0000","last_active":"2023-10-14T10:37:01.787861+0000","last_peered":"2023-10-14T10:37:01.787861+0000","last_clean":"2023-10-14T10:37:01.787861+0000","last_became_active":"2023-10-14T10:36:59.067345+0000","last_became_peered":"2023-10-14T10:36:59.067345+0000","last_unstale":"2023-10-14T10:37:01.787861+0000","last_undegraded":"2023-10-14T10:37:01.787861+0000","last_fullsized":"2023-10-14T10:37:01.787861+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:54:15.011628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788501+0000","last_change":"2023-10-14T10:37:00.077744+0000","last_active":"2023-10-14T10:37:01.788501+0000","last_peered":"2023-10-14T10:37:01.788501+0000","last_clean":"2023-10-14T10:37:01.788501+0000","last_became_active":"2023-10-14T10:37:00.077465+0000","last_became_peered":"2023-10-14T10:37:00.077465+0000","last_unstale":"2023-10-14T10:37:01.788501+0000","last_undegraded":"2023-10-14T10:37:01.788501+0000","last_fullsized":"2023-10-14T10:37:01.788501+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T20:36:23.971821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.787432+0000","last_change":"2023-10-14T10:36:59.069707+0000","last_active":"2023-10-14T10:37:01.787432+0000","last_peered":"2023-10-14T10:37:01.787432+0000","last_clean":"2023-10-14T10:37:01.787432+0000","last_became_active":"2023-10-14T10:36:59.069269+0000","last_became_peered":"2023-10-14T10:36:59.069269+0000","last_unstale":"2023-10-14T10:37:01.787432+0000","last_undegraded":"2023-10-14T10:37:01.787432+0000","last_fullsized":"2023-10-14T10:37:01.787432+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T12:15:16.823069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788221+0000","last_change":"2023-10-14T10:37:00.068887+0000","last_active":"2023-10-14T10:37:01.788221+0000","last_peered":"2023-10-14T10:37:01.788221+0000","last_clean":"2023-10-14T10:37:01.788221+0000","last_became_active":"2023-10-14T10:37:00.068620+0000","last_became_peered":"2023-10-14T10:37:00.068620+0000","last_unstale":"2023-10-14T10:37:01.788221+0000","last_undegraded":"2023-10-14T10:37:01.788221+0000","last_fullsized":"2023-10-14T10:37:01.788221+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T18:04:59.338050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.489058+0000","last_change":"2023-10-14T10:36:59.061947+0000","last_active":"2023-10-14T10:37:01.489058+0000","last_peered":"2023-10-14T10:37:01.489058+0000","last_clean":"2023-10-14T10:37:01.489058+0000","last_became_active":"2023-10-14T10:36:59.061678+0000","last_became_peered":"2023-10-14T10:36:59.061678+0000","last_unstale":"2023-10-14T10:37:01.489058+0000","last_undegraded":"2023-10-14T10:37:01.489058+0000","last_fullsized":"2023-10-14T10:37:01.489058+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:39:01.994329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.061998+0000","last_change":"2023-10-14T10:37:00.073204+0000","last_active":"2023-10-14T10:37:01.061998+0000","last_peered":"2023-10-14T10:37:01.061998+0000","last_clean":"2023-10-14T10:37:01.061998+0000","last_became_active":"2023-10-14T10:37:00.071692+0000","last_became_peered":"2023-10-14T10:37:00.071692+0000","last_unstale":"2023-10-14T10:37:01.061998+0000","last_undegraded":"2023-10-14T10:37:01.061998+0000","last_fullsized":"2023-10-14T10:37:01.061998+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:17:54.730152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.061467+0000","last_change":"2023-10-14T10:36:59.063545+0000","last_active":"2023-10-14T10:37:01.061467+0000","last_peered":"2023-10-14T10:37:01.061467+0000","last_clean":"2023-10-14T10:37:01.061467+0000","last_became_active":"2023-10-14T10:36:59.063332+0000","last_became_peered":"2023-10-14T10:36:59.063332+0000","last_unstale":"2023-10-14T10:37:01.061467+0000","last_undegraded":"2023-10-14T10:37:01.061467+0000","last_fullsized":"2023-10-14T10:37:01.061467+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:10:45.467136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788558+0000","last_change":"2023-10-14T10:37:00.075249+0000","last_active":"2023-10-14T10:37:01.788558+0000","last_peered":"2023-10-14T10:37:01.788558+0000","last_clean":"2023-10-14T10:37:01.788558+0000","last_became_active":"2023-10-14T10:37:00.074859+0000","last_became_peered":"2023-10-14T10:37:00.074859+0000","last_unstale":"2023-10-14T10:37:01.788558+0000","last_undegraded":"2023-10-14T10:37:01.788558+0000","last_fullsized":"2023-10-14T10:37:01.788558+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:04:51.088268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.060369+0000","last_change":"2023-10-14T10:36:59.069418+0000","last_active":"2023-10-14T10:37:01.060369+0000","last_peered":"2023-10-14T10:37:01.060369+0000","last_clean":"2023-10-14T10:37:01.060369+0000","last_became_active":"2023-10-14T10:36:59.069037+0000","last_became_peered":"2023-10-14T10:36:59.069037+0000","last_unstale":"2023-10-14T10:37:01.060369+0000","last_undegraded":"2023-10-14T10:37:01.060369+0000","last_fullsized":"2023-10-14T10:37:01.060369+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:45:50.919718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.062253+0000","last_change":"2023-10-14T10:37:00.075731+0000","last_active":"2023-10-14T10:37:01.062253+0000","last_peered":"2023-10-14T10:37:01.062253+0000","last_clean":"2023-10-14T10:37:01.062253+0000","last_became_active":"2023-10-14T10:37:00.075491+0000","last_became_peered":"2023-10-14T10:37:00.075491+0000","last_unstale":"2023-10-14T10:37:01.062253+0000","last_undegraded":"2023-10-14T10:37:01.062253+0000","last_fullsized":"2023-10-14T10:37:01.062253+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T14:52:18.060480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"33'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.821131+0000","last_change":"2023-10-14T10:36:59.066461+0000","last_active":"2023-10-14T10:37:01.821131+0000","last_peered":"2023-10-14T10:37:01.821131+0000","last_clean":"2023-10-14T10:37:01.821131+0000","last_became_active":"2023-10-14T10:36:59.066146+0000","last_became_peered":"2023-10-14T10:36:59.066146+0000","last_unstale":"2023-10-14T10:37:01.821131+0000","last_undegraded":"2023-10-14T10:37:01.821131+0000","last_fullsized":"2023-10-14T10:37:01.821131+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:45:50.919718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.787608+0000","last_change":"2023-10-14T10:37:00.082851+0000","last_active":"2023-10-14T10:37:01.787608+0000","last_peered":"2023-10-14T10:37:01.787608+0000","last_clean":"2023-10-14T10:37:01.787608+0000","last_became_active":"2023-10-14T10:37:00.082570+0000","last_became_peered":"2023-10-14T10:37:00.082570+0000","last_unstale":"2023-10-14T10:37:01.787608+0000","last_undegraded":"2023-10-14T10:37:01.787608+0000","last_fullsized":"2023-10-14T10:37:01.787608+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:28:45.821377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.059981+0000","last_change":"2023-10-14T10:36:59.061936+0000","last_active":"2023-10-14T10:37:01.059981+0000","last_peered":"2023-10-14T10:37:01.059981+0000","last_clean":"2023-10-14T10:37:01.059981+0000","last_became_active":"2023-10-14T10:36:59.061642+0000","last_became_peered":"2023-10-14T10:36:59.061642+0000","last_unstale":"2023-10-14T10:37:01.059981+0000","last_undegraded":"2023-10-14T10:37:01.059981+0000","last_fullsized":"2023-10-14T10:37:01.059981+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:54:15.011628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.787449+0000","last_change":"2023-10-14T10:37:00.071551+0000","last_active":"2023-10-14T10:37:01.787449+0000","last_peered":"2023-10-14T10:37:01.787449+0000","last_clean":"2023-10-14T10:37:01.787449+0000","last_became_active":"2023-10-14T10:37:00.071244+0000","last_became_peered":"2023-10-14T10:37:00.071244+0000","last_unstale":"2023-10-14T10:37:01.787449+0000","last_undegraded":"2023-10-14T10:37:01.787449+0000","last_fullsized":"2023-10-14T10:37:01.787449+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:08:06.341692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.787778+0000","last_change":"2023-10-14T10:36:59.066839+0000","last_active":"2023-10-14T10:37:01.787778+0000","last_peered":"2023-10-14T10:37:01.787778+0000","last_clean":"2023-10-14T10:37:01.787778+0000","last_became_active":"2023-10-14T10:36:59.066592+0000","last_became_peered":"2023-10-14T10:36:59.066592+0000","last_unstale":"2023-10-14T10:37:01.787778+0000","last_undegraded":"2023-10-14T10:37:01.787778+0000","last_fullsized":"2023-10-14T10:37:01.787778+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T12:15:16.823069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788013+0000","last_change":"2023-10-14T10:37:00.076076+0000","last_active":"2023-10-14T10:37:01.788013+0000","last_peered":"2023-10-14T10:37:01.788013+0000","last_clean":"2023-10-14T10:37:01.788013+0000","last_became_active":"2023-10-14T10:37:00.075794+0000","last_became_peered":"2023-10-14T10:37:00.075794+0000","last_unstale":"2023-10-14T10:37:01.788013+0000","last_undegraded":"2023-10-14T10:37:01.788013+0000","last_fullsized":"2023-10-14T10:37:01.788013+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:44:50.678288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.060021+0000","last_change":"2023-10-14T10:36:59.065626+0000","last_active":"2023-10-14T10:37:01.060021+0000","last_peered":"2023-10-14T10:37:01.060021+0000","last_clean":"2023-10-14T10:37:01.060021+0000","last_became_active":"2023-10-14T10:36:59.065163+0000","last_became_peered":"2023-10-14T10:36:59.065163+0000","last_unstale":"2023-10-14T10:37:01.060021+0000","last_undegraded":"2023-10-14T10:37:01.060021+0000","last_fullsized":"2023-10-14T10:37:01.060021+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:08:05.335325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788427+0000","last_change":"2023-10-14T10:37:00.078024+0000","last_active":"2023-10-14T10:37:01.788427+0000","last_peered":"2023-10-14T10:37:01.788427+0000","last_clean":"2023-10-14T10:37:01.788427+0000","last_became_active":"2023-10-14T10:37:00.077621+0000","last_became_peered":"2023-10-14T10:37:00.077621+0000","last_unstale":"2023-10-14T10:37:01.788427+0000","last_undegraded":"2023-10-14T10:37:01.788427+0000","last_fullsized":"2023-10-14T10:37:01.788427+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:06:28.391046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.489094+0000","last_change":"2023-10-14T10:36:59.064632+0000","last_active":"2023-10-14T10:37:01.489094+0000","last_peered":"2023-10-14T10:37:01.489094+0000","last_clean":"2023-10-14T10:37:01.489094+0000","last_became_active":"2023-10-14T10:36:59.064336+0000","last_became_peered":"2023-10-14T10:36:59.064336+0000","last_unstale":"2023-10-14T10:37:01.489094+0000","last_undegraded":"2023-10-14T10:37:01.489094+0000","last_fullsized":"2023-10-14T10:37:01.489094+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:08:05.335325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.488971+0000","last_change":"2023-10-14T10:37:00.081042+0000","last_active":"2023-10-14T10:37:01.488971+0000","last_peered":"2023-10-14T10:37:01.488971+0000","last_clean":"2023-10-14T10:37:01.488971+0000","last_became_active":"2023-10-14T10:37:00.080340+0000","last_became_peered":"2023-10-14T10:37:00.080340+0000","last_unstale":"2023-10-14T10:37:01.488971+0000","last_undegraded":"2023-10-14T10:37:01.488971+0000","last_fullsized":"2023-10-14T10:37:01.488971+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:44:50.678288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788029+0000","last_change":"2023-10-14T10:36:59.063568+0000","last_active":"2023-10-14T10:37:01.788029+0000","last_peered":"2023-10-14T10:37:01.788029+0000","last_clean":"2023-10-14T10:37:01.788029+0000","last_became_active":"2023-10-14T10:36:59.063130+0000","last_became_peered":"2023-10-14T10:36:59.063130+0000","last_unstale":"2023-10-14T10:37:01.788029+0000","last_undegraded":"2023-10-14T10:37:01.788029+0000","last_fullsized":"2023-10-14T10:37:01.788029+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:59:06.202215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.061998+0000","last_change":"2023-10-14T10:37:00.068870+0000","last_active":"2023-10-14T10:37:01.061998+0000","last_peered":"2023-10-14T10:37:01.061998+0000","last_clean":"2023-10-14T10:37:01.061998+0000","last_became_active":"2023-10-14T10:37:00.068556+0000","last_became_peered":"2023-10-14T10:37:00.068556+0000","last_unstale":"2023-10-14T10:37:01.061998+0000","last_undegraded":"2023-10-14T10:37:01.061998+0000","last_fullsized":"2023-10-14T10:37:01.061998+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T14:52:18.060480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.787484+0000","last_change":"2023-10-14T10:36:59.066703+0000","last_active":"2023-10-14T10:37:01.787484+0000","last_peered":"2023-10-14T10:37:01.787484+0000","last_clean":"2023-10-14T10:37:01.787484+0000","last_became_active":"2023-10-14T10:36:59.066424+0000","last_became_peered":"2023-10-14T10:36:59.066424+0000","last_unstale":"2023-10-14T10:37:01.787484+0000","last_undegraded":"2023-10-14T10:37:01.787484+0000","last_fullsized":"2023-10-14T10:37:01.787484+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:54:15.011628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":176,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-14T10:37:00.060617+0000","last_change":"2023-10-14T10:35:59.770163+0000","last_active":"2023-10-14T10:37:00.060617+0000","last_peered":"2023-10-14T10:37:00.060617+0000","last_clean":"2023-10-14T10:37:00.060617+0000","last_became_active":"2023-10-14T10:35:59.769199+0000","last_became_peered":"2023-10-14T10:35:59.769199+0000","last_unstale":"2023-10-14T10:37:00.060617+0000","last_undegraded":"2023-10-14T10:37:00.060617+0000","last_fullsized":"2023-10-14T10:37:00.060617+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:35:58.675572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:35:58.675572+0000","last_clean_scrub_stamp":"2023-10-14T10:35:58.675572+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:03:25.281187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542162,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27972,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343868,"statfs":{"total":96636764160,"available":96608120832,"internally_reserved":0,"allocated":966656,"data_stored":732594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 10:36:59 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":2.4649999999999999,"5min":2.4649999999999999,"15min":2.4649999999999999},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.45500000000000002}]},{"osd":1,"last update":"Sat Oct 14 10:36:59 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":2.4289999999999998,"5min":2.4289999999999998,"15min":2.4289999999999998},"last":0.60199999999999998}]},{"osd":2,"last update":"Sat Oct 14 10:36:59 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.59099999999999997}]},{"osd":3,"last update":"Sat Oct 14 10:36:59 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.405,"5min":1.405,"15min":1.405},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.47699999999999998}]},{"osd":4,"last update":"Sat Oct 14 10:36:59 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.57799999999999996}]},{"osd":5,"last update":"Sat Oct 14 10:36:59 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.51900000000000002}]},{"osd":6,"last update":"Sat Oct 14 10:36:59 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.42499999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542163,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27388,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344452,"statfs":{"total":96636764160,"available":96608718848,"internally_reserved":0,"allocated":376832,"data_stored":142752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 10:36:57 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.58499999999999996}]},{"osd":1,"last update":"Sat Oct 14 10:36:57 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.389,"5min":1.389,"15min":1.389},"last":0.68899999999999995}]},{"osd":2,"last update":"Sat Oct 14 10:36:57 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.74299999999999999}]},{"osd":3,"last update":"Sat Oct 14 10:36:57 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.65100000000000002}]},{"osd":4,"last update":"Sat Oct 14 10:36:57 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.499}]},{"osd":5,"last update":"Sat Oct 14 10:36:57 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.52700000000000002}]},{"osd":7,"last update":"Sat Oct 14 10:36:57 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.435},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.41299999999999998}]}]},{"osd":1,"up_from":14,"seq":60129542162,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":376,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":385024,"data_stored":143770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.254,"15min":1.254},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":16.855,"5min":16.855,"15min":16.855},"last":0.28399999999999997},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":19.353000000000002,"5min":19.353000000000002,"15min":19.353000000000002},"last":0.36399999999999999}]},{"osd":2,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":3.0670000000000002,"5min":3.0670000000000002,"15min":3.0670000000000002},"last":0.26100000000000001},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.32500000000000001}]},{"osd":3,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":10.005000000000001,"5min":10.005000000000001,"15min":10.005000000000001},"last":0.30299999999999999}]},{"osd":4,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":2.9500000000000002,"5min":2.9500000000000002,"15min":2.9500000000000002},"last":0.495},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":5.7270000000000003,"5min":5.7270000000000003,"15min":5.7270000000000003},"last":0.65400000000000003}]},{"osd":5,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.129,"15min":1.129},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":9.8800000000000008,"5min":9.8800000000000008,"15min":9.8800000000000008},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":3.0350000000000001,"5min":3.0350000000000001,"15min":3.0350000000000001},"last":0.66700000000000004}]},{"osd":6,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":5.681,"5min":5.681,"15min":5.681},"last":0.54600000000000004},{"interface":"front","average":{"1min":1.173,"5min":1.173,"15min":1.173},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":9.9380000000000006,"5min":9.9380000000000006,"15min":9.9380000000000006},"last":0.68700000000000006}]},{"osd":7,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":9.9730000000000008,"5min":9.9730000000000008,"15min":9.9730000000000008},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":3.0009999999999999,"5min":3.0009999999999999,"15min":3.0009999999999999},"last":0.58899999999999997}]}]},{"osd":0,"up_from":14,"seq":60129542161,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27960,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343880,"statfs":{"total":96636764160,"available":96608133120,"internally_reserved":0,"allocated":962560,"data_stored":732504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.62,"5min":0.62,"15min":0.62},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.495,"5min":0.495,"15min":0.495},"last":0.48199999999999998}]},{"osd":2,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"last":0.378},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"last":0.40400000000000003}]},{"osd":3,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"last":0.42699999999999999}]},{"osd":4,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.66500000000000004}]},{"osd":5,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.55800000000000005}]},{"osd":6,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.57299999999999995}]},{"osd":7,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.69699999999999995}]}]},{"osd":2,"up_from":14,"seq":60129542162,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":372736,"data_stored":142206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":1.429,"5min":1.429,"15min":1.429},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.36199999999999999}]},{"osd":1,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":1.5269999999999999,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.40500000000000003}]},{"osd":3,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":4.0629999999999997,"5min":4.0629999999999997,"15min":4.0629999999999997},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.381,"5min":1.381,"15min":1.381},"last":0.42899999999999999}]},{"osd":4,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.58399999999999996}]},{"osd":5,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":3.8599999999999999,"5min":3.8599999999999999,"15min":3.8599999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.464,"5min":1.464,"15min":1.464},"last":0.68899999999999995}]},{"osd":6,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.753},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":3.9820000000000002,"5min":3.9820000000000002,"15min":3.9820000000000002},"last":0.63600000000000001}]},{"osd":7,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":3.9049999999999998,"5min":3.9049999999999998,"15min":3.9049999999999998},"last":0.61699999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542162,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":376,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":385024,"data_stored":143770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"last":0.34300000000000003}]},{"osd":1,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"last":0.373}]},{"osd":2,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"last":0.26600000000000001},{"interface":"front","average":{"1min":0.437,"5min":0.437,"15min":0.437},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.40300000000000002}]},{"osd":4,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":6.0350000000000001,"5min":6.0350000000000001,"15min":6.0350000000000001},"last":0.71499999999999997}]},{"osd":5,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":6.0540000000000003,"5min":6.0540000000000003,"15min":6.0540000000000003},"last":0.65700000000000003}]},{"osd":6,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.621}]},{"osd":7,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":6.0819999999999999,"5min":6.0819999999999999,"15min":6.0819999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.61199999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542161,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27388,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344452,"statfs":{"total":96636764160,"available":96608718848,"internally_reserved":0,"allocated":376832,"data_stored":142240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 10:36:56 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.47699999999999998}]},{"osd":1,"last update":"Sat Oct 14 10:36:56 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.39000000000000001}]},{"osd":2,"last update":"Sat Oct 14 10:36:56 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.42699999999999999}]},{"osd":3,"last update":"Sat Oct 14 10:36:56 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.48999999999999999}]},{"osd":5,"last update":"Sat Oct 14 10:36:56 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.60299999999999998}]},{"osd":6,"last update":"Sat Oct 14 10:36:56 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.61899999999999999}]},{"osd":7,"last update":"Sat Oct 14 10:36:56 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.66300000000000003}]}]},{"osd":5,"up_from":14,"seq":60129542162,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27388,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344452,"statfs":{"total":96636764160,"available":96608718848,"internally_reserved":0,"allocated":376832,"data_stored":142808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.64600000000000002}]},{"osd":1,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.72099999999999997}]},{"osd":2,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.70799999999999996}]},{"osd":3,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.66200000000000003}]},{"osd":4,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.372},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.41099999999999998}]},{"osd":6,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.439}]},{"osd":7,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.51900000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-14T10:37:08.875 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-14T10:37:08.876 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T10:37:09.034 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.032+0000 7f81a9502700 1 Processor -- start 2023-10-14T10:37:09.034 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.032+0000 7f81a9502700 1 -- start start 2023-10-14T10:37:09.038 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.032+0000 7f81a9502700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f81a40b2c60 0x7f81a40cde60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:09.038 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.032+0000 7f81a9502700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81a40d0d00 con 0x7f81a40b2880 2023-10-14T10:37:09.039 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.032+0000 7f81a9502700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81a40ce7a0 con 0x7f81a40ce3a0 2023-10-14T10:37:09.039 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.032+0000 7f81a9502700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81a406cd70 con 0x7f81a40b2c60 2023-10-14T10:37:09.039 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.033+0000 7f81a2ffd700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f81a40b2880 0x7f81a406c660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:53740/0 (socket says 172.21.15.67:53740) 2023-10-14T10:37:09.040 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.033+0000 7f81a2ffd700 1 -- 172.21.15.67:0/2751609862 learned_addr learned my addr 172.21.15.67:0/2751609862 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:37:09.040 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.033+0000 7f81a27fc700 1 --2- 172.21.15.67:0/2751609862 >> v2:172.21.15.72:3300/0 conn(0x7f81a40b2c60 0x7f81a40cde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:09.040 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.033+0000 7f81a1ffb700 1 -- 172.21.15.67:0/2751609862 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3777105191 0 0) 0x7f81a40d0d00 con 0x7f81a40b2880 2023-10-14T10:37:09.041 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.033+0000 7f81a1ffb700 1 -- 172.21.15.67:0/2751609862 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81900035b0 con 0x7f81a40b2880 2023-10-14T10:37:09.041 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.033+0000 7f81a1ffb700 1 -- 172.21.15.67:0/2751609862 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 313336398 0 0) 0x7f81900035b0 con 0x7f81a40b2880 2023-10-14T10:37:09.041 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.033+0000 7f81a1ffb700 1 -- 172.21.15.67:0/2751609862 >> v2:172.21.15.72:3300/0 conn(0x7f81a40b2c60 msgr2=0x7f81a40cde60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:37:09.041 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.033+0000 7f81a1ffb700 1 --2- 172.21.15.67:0/2751609862 >> v2:172.21.15.72:3300/0 conn(0x7f81a40b2c60 0x7f81a40cde60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:09.042 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.033+0000 7f81a1ffb700 1 -- 172.21.15.67:0/2751609862 >> v1:172.21.15.72:6789/0 conn(0x7f81a40ce3a0 legacy=0x7f81a415d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:09.042 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.033+0000 7f81a1ffb700 1 -- 172.21.15.67:0/2751609862 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81a40d2450 con 0x7f81a40b2880 2023-10-14T10:37:09.042 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.033+0000 7f81a1ffb700 1 -- 172.21.15.67:0/2751609862 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3180043755 0 0) 0x7f81940017f0 con 0x7f81a40b2880 2023-10-14T10:37:09.043 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.033+0000 7f81a1ffb700 1 -- 172.21.15.67:0/2751609862 >> v1:172.21.15.67:6789/0 conn(0x7f81a40b2880 legacy=0x7f81a406c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:09.043 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.033+0000 7f81a1ffb700 1 --2- 172.21.15.67:0/2751609862 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8190003a00 0x7f8190003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:09.043 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.033+0000 7f81a1ffb700 1 --2- 172.21.15.67:0/2751609862 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8190004850 0x7f8190004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:09.043 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.033+0000 7f81a1ffb700 1 --2- 172.21.15.67:0/2751609862 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f8190005210 0x7f819000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:09.043 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.033+0000 7f81a1ffb700 1 -- 172.21.15.67:0/2751609862 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81a406cd70 con 0x7f8190003a00 2023-10-14T10:37:09.044 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.033+0000 7f81a1ffb700 1 -- 172.21.15.67:0/2751609862 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f819000c0c0 con 0x7f8190004850 2023-10-14T10:37:09.044 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.033+0000 7f81a1ffb700 1 -- 172.21.15.67:0/2751609862 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f819000c280 con 0x7f8190005210 2023-10-14T10:37:09.044 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.033+0000 7f81a2ffd700 1 --2- 172.21.15.67:0/2751609862 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8190003a00 0x7f8190003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:09.044 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.033+0000 7f81a27fc700 1 --2- 172.21.15.67:0/2751609862 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8190004850 0x7f8190004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:09.045 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.033+0000 7f81a37fe700 1 --2- 172.21.15.67:0/2751609862 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f8190005210 0x7f819000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:09.045 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.034+0000 7f81a2ffd700 1 -- 172.21.15.67:0/2751609862 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f8190005210 msgr2=0x7f819000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:37:09.045 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.034+0000 7f81a2ffd700 1 --2- 172.21.15.67:0/2751609862 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f8190005210 0x7f819000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:09.046 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.034+0000 7f81a2ffd700 1 -- 172.21.15.67:0/2751609862 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8190004850 msgr2=0x7f8190004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:37:09.046 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.034+0000 7f81a2ffd700 1 --2- 172.21.15.67:0/2751609862 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8190004850 0x7f8190004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:09.046 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.034+0000 7f81a37fe700 1 -- 172.21.15.67:0/2751609862 reap_dead start 2023-10-14T10:37:09.046 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.034+0000 7f81a2ffd700 1 -- 172.21.15.67:0/2751609862 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81900035b0 con 0x7f8190003a00 2023-10-14T10:37:09.047 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.034+0000 7f81a2ffd700 1 --2- 172.21.15.67:0/2751609862 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8190003a00 0x7f8190003e20 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f819400c240 tx=0x7f8190005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=947793431aee3e33 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:09.047 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.034+0000 7f81a1ffb700 1 -- 172.21.15.67:0/2751609862 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f819400cda0 con 0x7f8190003a00 2023-10-14T10:37:09.047 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.034+0000 7f81a1ffb700 1 -- 172.21.15.67:0/2751609862 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8194007b80 con 0x7f8190003a00 2023-10-14T10:37:09.047 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.034+0000 7f81a1ffb700 1 -- 172.21.15.67:0/2751609862 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81940072d0 con 0x7f8190003a00 2023-10-14T10:37:09.048 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.034+0000 7f81a9502700 1 -- 172.21.15.67:0/2751609862 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8190003a00 msgr2=0x7f8190003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:09.048 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.034+0000 7f81a9502700 1 --2- 172.21.15.67:0/2751609862 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8190003a00 0x7f8190003e20 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f819400c240 tx=0x7f8190005880 comp rx=0 tx=0).stop 2023-10-14T10:37:09.048 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.035+0000 7f81a9502700 1 -- 172.21.15.67:0/2751609862 shutdown_connections 2023-10-14T10:37:09.048 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.035+0000 7f81a9502700 1 --2- 172.21.15.67:0/2751609862 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8190003a00 0x7f8190003e20 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:09.049 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.035+0000 7f81a9502700 1 -- 172.21.15.67:0/2751609862 >> 172.21.15.67:0/2751609862 conn(0x7f81a40d15e0 msgr2=0x7f81a40d19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:37:09.049 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.035+0000 7f81a9502700 1 -- 172.21.15.67:0/2751609862 shutdown_connections 2023-10-14T10:37:09.049 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.035+0000 7f81a9502700 1 -- 172.21.15.67:0/2751609862 wait complete. 2023-10-14T10:37:09.050 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.036+0000 7f81a9502700 1 Processor -- start 2023-10-14T10:37:09.050 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.036+0000 7f81a9502700 1 -- start start 2023-10-14T10:37:09.053 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.037+0000 7f81a9502700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f8190003a00 0x7f81a415cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:09.053 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.037+0000 7f81a9502700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81a41f4640 0x7f81a41f3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:09.053 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.037+0000 7f81a9502700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f81a40b69f0 0x7f81a41595f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:09.053 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.037+0000 7f81a9502700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81a415ab70 con 0x7f81a41f4640 2023-10-14T10:37:09.054 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.037+0000 7f81a9502700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81a41f4ac0 con 0x7f81a40b69f0 2023-10-14T10:37:09.054 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.037+0000 7f81a9502700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81a415a540 con 0x7f8190003a00 2023-10-14T10:37:09.054 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.037+0000 7f81a27fc700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81a41f4640 0x7f81a41f3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:09.055 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.037+0000 7f81a27fc700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81a41f4640 0x7f81a41f3ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45988/0 (socket says 172.21.15.67:45988) 2023-10-14T10:37:09.055 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.037+0000 7f81a27fc700 1 -- 172.21.15.67:0/3909129499 learned_addr learned my addr 172.21.15.67:0/3909129499 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:37:09.055 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.038+0000 7f81a2ffd700 1 --2- 172.21.15.67:0/3909129499 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f8190003a00 0x7f81a415cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:09.055 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.038+0000 7f81a37fe700 1 --2- 172.21.15.67:0/3909129499 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f81a40b69f0 0x7f81a41595f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:09.056 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.038+0000 7f81a27fc700 1 -- 172.21.15.67:0/3909129499 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f8190003a00 msgr2=0x7f81a415cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:09.056 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.038+0000 7f81a27fc700 1 --2- 172.21.15.67:0/3909129499 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f8190003a00 0x7f81a415cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:09.056 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.038+0000 7f81a27fc700 1 -- 172.21.15.67:0/3909129499 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f81a40b69f0 msgr2=0x7f81a41595f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:09.056 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.038+0000 7f81a27fc700 1 --2- 172.21.15.67:0/3909129499 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f81a40b69f0 0x7f81a41595f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:09.057 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.038+0000 7f81a27fc700 1 -- 172.21.15.67:0/3909129499 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81a415d190 con 0x7f81a41f4640 2023-10-14T10:37:09.057 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.038+0000 7f81a27fc700 1 --2- 172.21.15.67:0/3909129499 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81a41f4640 0x7f81a41f3ea0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f818c0119d0 tx=0x7f818c011ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:09.057 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.038+0000 7f819b7fe700 1 -- 172.21.15.67:0/3909129499 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f818c004030 con 0x7f81a41f4640 2023-10-14T10:37:09.058 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.038+0000 7f819b7fe700 1 -- 172.21.15.67:0/3909129499 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f818c004190 con 0x7f81a41f4640 2023-10-14T10:37:09.058 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.038+0000 7f81a9502700 1 -- 172.21.15.67:0/3909129499 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81a415d470 con 0x7f81a41f4640 2023-10-14T10:37:09.058 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.038+0000 7f819b7fe700 1 -- 172.21.15.67:0/3909129499 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f818c004520 con 0x7f81a41f4640 2023-10-14T10:37:09.058 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.038+0000 7f81a9502700 1 -- 172.21.15.67:0/3909129499 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81a41f43e0 con 0x7f81a41f4640 2023-10-14T10:37:09.059 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.039+0000 7f81a9502700 1 -- 172.21.15.67:0/3909129499 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81a415acb0 con 0x7f81a41f4640 2023-10-14T10:37:09.059 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.039+0000 7f819b7fe700 1 -- 172.21.15.67:0/3909129499 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f818c00f730 con 0x7f81a41f4640 2023-10-14T10:37:09.059 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.040+0000 7f819b7fe700 1 --2- 172.21.15.67:0/3909129499 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f818406af00 0x7f818406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:09.059 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.040+0000 7f819b7fe700 1 -- 172.21.15.67:0/3909129499 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5729+0+0 (secure 0 0 0) 0x7f818c089900 con 0x7f81a41f4640 2023-10-14T10:37:09.060 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.040+0000 7f81a2ffd700 1 --2- 172.21.15.67:0/3909129499 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f818406af00 0x7f818406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:09.060 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.041+0000 7f81a2ffd700 1 --2- 172.21.15.67:0/3909129499 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f818406af00 0x7f818406d3c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8194007d30 tx=0x7f8194007000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:09.060 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.044+0000 7f819b7fe700 1 -- 172.21.15.67:0/3909129499 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f81a415acb0 con 0x7f81a41f4640 2023-10-14T10:37:09.247 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.245+0000 7f81a9502700 1 -- 172.21.15.67:0/3909129499 --> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f81a40cff40 con 0x7f818406af00 2023-10-14T10:37:09.255 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.253+0000 7f819b7fe700 1 -- 172.21.15.67:0/3909129499 <== mgr.4100 v2:172.21.15.72:6832/703488382 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289547 (secure 0 0 0) 0x7f8194000f70 con 0x7f818406af00 2023-10-14T10:37:09.259 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-14T10:37:09.261 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.257+0000 7f81a9502700 1 -- 172.21.15.67:0/3909129499 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f818406af00 msgr2=0x7f818406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:09.262 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.257+0000 7f81a9502700 1 --2- 172.21.15.67:0/3909129499 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f818406af00 0x7f818406d3c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8194007d30 tx=0x7f8194007000 comp rx=0 tx=0).stop 2023-10-14T10:37:09.262 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.257+0000 7f81a9502700 1 -- 172.21.15.67:0/3909129499 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81a41f4640 msgr2=0x7f81a41f3ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:09.262 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.257+0000 7f81a9502700 1 --2- 172.21.15.67:0/3909129499 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81a41f4640 0x7f81a41f3ea0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f818c0119d0 tx=0x7f818c011ce0 comp rx=0 tx=0).stop 2023-10-14T10:37:09.263 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.257+0000 7f81a9502700 1 -- 172.21.15.67:0/3909129499 shutdown_connections 2023-10-14T10:37:09.263 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.257+0000 7f81a9502700 1 --2- 172.21.15.67:0/3909129499 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f818406af00 0x7f818406d3c0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:09.263 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.257+0000 7f81a9502700 1 --2- 172.21.15.67:0/3909129499 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f81a40b69f0 0x7f81a41595f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:09.263 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.257+0000 7f81a9502700 1 --2- 172.21.15.67:0/3909129499 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f8190003a00 0x7f81a415cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:09.264 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.257+0000 7f81a9502700 1 --2- 172.21.15.67:0/3909129499 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81a41f4640 0x7f81a41f3ea0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:09.264 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.257+0000 7f81a9502700 1 -- 172.21.15.67:0/3909129499 >> 172.21.15.67:0/3909129499 conn(0x7f81a40d15e0 msgr2=0x7f81a406b950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:37:09.264 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.257+0000 7f81a9502700 1 -- 172.21.15.67:0/3909129499 shutdown_connections 2023-10-14T10:37:09.265 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.257+0000 7f81a9502700 1 -- 172.21.15.67:0/3909129499 wait complete. 2023-10-14T10:37:09.265 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2023-10-14T10:37:09.271 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":62,"stamp":"2023-10-14T10:37:07.770696+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220280,"kb_used_data":4104,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754440,"statfs":{"total":773094113280,"available":772868546560,"internally_reserved":0,"allocated":4202496,"data_stored":2322644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12742,"internal_metadata":221236794},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002093"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.787560+0000","last_change":"2023-10-14T10:37:00.075102+0000","last_active":"2023-10-14T10:37:01.787560+0000","last_peered":"2023-10-14T10:37:01.787560+0000","last_clean":"2023-10-14T10:37:01.787560+0000","last_became_active":"2023-10-14T10:37:00.074851+0000","last_became_peered":"2023-10-14T10:37:00.074851+0000","last_unstale":"2023-10-14T10:37:01.787560+0000","last_undegraded":"2023-10-14T10:37:01.787560+0000","last_fullsized":"2023-10-14T10:37:01.787560+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T21:39:35.587728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788402+0000","last_change":"2023-10-14T10:37:00.068871+0000","last_active":"2023-10-14T10:37:01.788402+0000","last_peered":"2023-10-14T10:37:01.788402+0000","last_clean":"2023-10-14T10:37:01.788402+0000","last_became_active":"2023-10-14T10:37:00.068586+0000","last_became_peered":"2023-10-14T10:37:00.068586+0000","last_unstale":"2023-10-14T10:37:01.788402+0000","last_undegraded":"2023-10-14T10:37:01.788402+0000","last_fullsized":"2023-10-14T10:37:01.788402+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T12:47:24.725885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.062215+0000","last_change":"2023-10-14T10:37:00.080988+0000","last_active":"2023-10-14T10:37:01.062215+0000","last_peered":"2023-10-14T10:37:01.062215+0000","last_clean":"2023-10-14T10:37:01.062215+0000","last_became_active":"2023-10-14T10:37:00.080476+0000","last_became_peered":"2023-10-14T10:37:00.080476+0000","last_unstale":"2023-10-14T10:37:01.062215+0000","last_undegraded":"2023-10-14T10:37:01.062215+0000","last_fullsized":"2023-10-14T10:37:01.062215+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T10:48:03.043383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.059874+0000","last_change":"2023-10-14T10:37:00.077598+0000","last_active":"2023-10-14T10:37:01.059874+0000","last_peered":"2023-10-14T10:37:01.059874+0000","last_clean":"2023-10-14T10:37:01.059874+0000","last_became_active":"2023-10-14T10:37:00.077322+0000","last_became_peered":"2023-10-14T10:37:00.077322+0000","last_unstale":"2023-10-14T10:37:01.059874+0000","last_undegraded":"2023-10-14T10:37:01.059874+0000","last_fullsized":"2023-10-14T10:37:01.059874+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:28:04.536948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788360+0000","last_change":"2023-10-14T10:37:00.068342+0000","last_active":"2023-10-14T10:37:01.788360+0000","last_peered":"2023-10-14T10:37:01.788360+0000","last_clean":"2023-10-14T10:37:01.788360+0000","last_became_active":"2023-10-14T10:37:00.068055+0000","last_became_peered":"2023-10-14T10:37:00.068055+0000","last_unstale":"2023-10-14T10:37:01.788360+0000","last_undegraded":"2023-10-14T10:37:01.788360+0000","last_fullsized":"2023-10-14T10:37:01.788360+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T12:52:16.372161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788917+0000","last_change":"2023-10-14T10:37:00.072629+0000","last_active":"2023-10-14T10:37:01.788917+0000","last_peered":"2023-10-14T10:37:01.788917+0000","last_clean":"2023-10-14T10:37:01.788917+0000","last_became_active":"2023-10-14T10:37:00.072345+0000","last_became_peered":"2023-10-14T10:37:00.072345+0000","last_unstale":"2023-10-14T10:37:01.788917+0000","last_undegraded":"2023-10-14T10:37:01.788917+0000","last_fullsized":"2023-10-14T10:37:01.788917+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T17:09:21.565688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.061635+0000","last_change":"2023-10-14T10:37:00.073019+0000","last_active":"2023-10-14T10:37:01.061635+0000","last_peered":"2023-10-14T10:37:01.061635+0000","last_clean":"2023-10-14T10:37:01.061635+0000","last_became_active":"2023-10-14T10:37:00.072368+0000","last_became_peered":"2023-10-14T10:37:00.072368+0000","last_unstale":"2023-10-14T10:37:01.061635+0000","last_undegraded":"2023-10-14T10:37:01.061635+0000","last_fullsized":"2023-10-14T10:37:01.061635+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:31:16.127752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.062210+0000","last_change":"2023-10-14T10:37:00.074091+0000","last_active":"2023-10-14T10:37:01.062210+0000","last_peered":"2023-10-14T10:37:01.062210+0000","last_clean":"2023-10-14T10:37:01.062210+0000","last_became_active":"2023-10-14T10:37:00.073792+0000","last_became_peered":"2023-10-14T10:37:00.073792+0000","last_unstale":"2023-10-14T10:37:01.062210+0000","last_undegraded":"2023-10-14T10:37:01.062210+0000","last_fullsized":"2023-10-14T10:37:01.062210+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T17:27:50.406084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.489588+0000","last_change":"2023-10-14T10:37:00.081745+0000","last_active":"2023-10-14T10:37:01.489588+0000","last_peered":"2023-10-14T10:37:01.489588+0000","last_clean":"2023-10-14T10:37:01.489588+0000","last_became_active":"2023-10-14T10:37:00.081451+0000","last_became_peered":"2023-10-14T10:37:00.081451+0000","last_unstale":"2023-10-14T10:37:01.489588+0000","last_undegraded":"2023-10-14T10:37:01.489588+0000","last_fullsized":"2023-10-14T10:37:01.489588+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:18:45.297555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.060294+0000","last_change":"2023-10-14T10:37:00.071138+0000","last_active":"2023-10-14T10:37:01.060294+0000","last_peered":"2023-10-14T10:37:01.060294+0000","last_clean":"2023-10-14T10:37:01.060294+0000","last_became_active":"2023-10-14T10:37:00.070667+0000","last_became_peered":"2023-10-14T10:37:00.070667+0000","last_unstale":"2023-10-14T10:37:01.060294+0000","last_undegraded":"2023-10-14T10:37:01.060294+0000","last_fullsized":"2023-10-14T10:37:01.060294+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:52:37.894720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.062228+0000","last_change":"2023-10-14T10:37:00.073322+0000","last_active":"2023-10-14T10:37:01.062228+0000","last_peered":"2023-10-14T10:37:01.062228+0000","last_clean":"2023-10-14T10:37:01.062228+0000","last_became_active":"2023-10-14T10:37:00.072639+0000","last_became_peered":"2023-10-14T10:37:00.072639+0000","last_unstale":"2023-10-14T10:37:01.062228+0000","last_undegraded":"2023-10-14T10:37:01.062228+0000","last_fullsized":"2023-10-14T10:37:01.062228+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T18:57:50.136605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.061858+0000","last_change":"2023-10-14T10:37:00.069681+0000","last_active":"2023-10-14T10:37:01.061858+0000","last_peered":"2023-10-14T10:37:01.061858+0000","last_clean":"2023-10-14T10:37:01.061858+0000","last_became_active":"2023-10-14T10:37:00.069373+0000","last_became_peered":"2023-10-14T10:37:00.069373+0000","last_unstale":"2023-10-14T10:37:01.061858+0000","last_undegraded":"2023-10-14T10:37:01.061858+0000","last_fullsized":"2023-10-14T10:37:01.061858+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:31:16.127752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788687+0000","last_change":"2023-10-14T10:37:00.074917+0000","last_active":"2023-10-14T10:37:01.788687+0000","last_peered":"2023-10-14T10:37:01.788687+0000","last_clean":"2023-10-14T10:37:01.788687+0000","last_became_active":"2023-10-14T10:37:00.074473+0000","last_became_peered":"2023-10-14T10:37:00.074473+0000","last_unstale":"2023-10-14T10:37:01.788687+0000","last_undegraded":"2023-10-14T10:37:01.788687+0000","last_fullsized":"2023-10-14T10:37:01.788687+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T20:20:02.187767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788602+0000","last_change":"2023-10-14T10:37:00.073986+0000","last_active":"2023-10-14T10:37:01.788602+0000","last_peered":"2023-10-14T10:37:01.788602+0000","last_clean":"2023-10-14T10:37:01.788602+0000","last_became_active":"2023-10-14T10:37:00.073651+0000","last_became_peered":"2023-10-14T10:37:00.073651+0000","last_unstale":"2023-10-14T10:37:01.788602+0000","last_undegraded":"2023-10-14T10:37:01.788602+0000","last_fullsized":"2023-10-14T10:37:01.788602+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T10:48:03.043383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.061624+0000","last_change":"2023-10-14T10:37:00.072277+0000","last_active":"2023-10-14T10:37:01.061624+0000","last_peered":"2023-10-14T10:37:01.061624+0000","last_clean":"2023-10-14T10:37:01.061624+0000","last_became_active":"2023-10-14T10:37:00.071999+0000","last_became_peered":"2023-10-14T10:37:00.071999+0000","last_unstale":"2023-10-14T10:37:01.061624+0000","last_undegraded":"2023-10-14T10:37:01.061624+0000","last_fullsized":"2023-10-14T10:37:01.061624+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:18:45.297555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.062458+0000","last_change":"2023-10-14T10:37:00.072492+0000","last_active":"2023-10-14T10:37:01.062458+0000","last_peered":"2023-10-14T10:37:01.062458+0000","last_clean":"2023-10-14T10:37:01.062458+0000","last_became_active":"2023-10-14T10:37:00.071509+0000","last_became_peered":"2023-10-14T10:37:00.071509+0000","last_unstale":"2023-10-14T10:37:01.062458+0000","last_undegraded":"2023-10-14T10:37:01.062458+0000","last_fullsized":"2023-10-14T10:37:01.062458+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:51:00.167498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.062381+0000","last_change":"2023-10-14T10:37:00.072597+0000","last_active":"2023-10-14T10:37:01.062381+0000","last_peered":"2023-10-14T10:37:01.062381+0000","last_clean":"2023-10-14T10:37:01.062381+0000","last_became_active":"2023-10-14T10:37:00.071946+0000","last_became_peered":"2023-10-14T10:37:00.071946+0000","last_unstale":"2023-10-14T10:37:01.062381+0000","last_undegraded":"2023-10-14T10:37:01.062381+0000","last_fullsized":"2023-10-14T10:37:01.062381+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:06:28.391046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788267+0000","last_change":"2023-10-14T10:37:00.080581+0000","last_active":"2023-10-14T10:37:01.788267+0000","last_peered":"2023-10-14T10:37:01.788267+0000","last_clean":"2023-10-14T10:37:01.788267+0000","last_became_active":"2023-10-14T10:37:00.080288+0000","last_became_peered":"2023-10-14T10:37:00.080288+0000","last_unstale":"2023-10-14T10:37:01.788267+0000","last_undegraded":"2023-10-14T10:37:01.788267+0000","last_fullsized":"2023-10-14T10:37:01.788267+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T18:57:50.136605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.787886+0000","last_change":"2023-10-14T10:37:00.080484+0000","last_active":"2023-10-14T10:37:01.787886+0000","last_peered":"2023-10-14T10:37:01.787886+0000","last_clean":"2023-10-14T10:37:01.787886+0000","last_became_active":"2023-10-14T10:37:00.080186+0000","last_became_peered":"2023-10-14T10:37:00.080186+0000","last_unstale":"2023-10-14T10:37:01.787886+0000","last_undegraded":"2023-10-14T10:37:01.787886+0000","last_fullsized":"2023-10-14T10:37:01.787886+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:17:54.730152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788501+0000","last_change":"2023-10-14T10:37:00.072056+0000","last_active":"2023-10-14T10:37:01.788501+0000","last_peered":"2023-10-14T10:37:01.788501+0000","last_clean":"2023-10-14T10:37:01.788501+0000","last_became_active":"2023-10-14T10:37:00.071796+0000","last_became_peered":"2023-10-14T10:37:00.071796+0000","last_unstale":"2023-10-14T10:37:01.788501+0000","last_undegraded":"2023-10-14T10:37:01.788501+0000","last_fullsized":"2023-10-14T10:37:01.788501+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T20:39:07.030985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.489206+0000","last_change":"2023-10-14T10:37:00.080796+0000","last_active":"2023-10-14T10:37:01.489206+0000","last_peered":"2023-10-14T10:37:01.489206+0000","last_clean":"2023-10-14T10:37:01.489206+0000","last_became_active":"2023-10-14T10:37:00.080526+0000","last_became_peered":"2023-10-14T10:37:00.080526+0000","last_unstale":"2023-10-14T10:37:01.489206+0000","last_undegraded":"2023-10-14T10:37:01.489206+0000","last_fullsized":"2023-10-14T10:37:01.489206+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:45:25.303185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.489168+0000","last_change":"2023-10-14T10:37:00.080350+0000","last_active":"2023-10-14T10:37:01.489168+0000","last_peered":"2023-10-14T10:37:01.489168+0000","last_clean":"2023-10-14T10:37:01.489168+0000","last_became_active":"2023-10-14T10:37:00.080062+0000","last_became_peered":"2023-10-14T10:37:00.080062+0000","last_unstale":"2023-10-14T10:37:01.489168+0000","last_undegraded":"2023-10-14T10:37:01.489168+0000","last_fullsized":"2023-10-14T10:37:01.489168+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:10:46.473503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.787611+0000","last_change":"2023-10-14T10:37:00.071237+0000","last_active":"2023-10-14T10:37:01.787611+0000","last_peered":"2023-10-14T10:37:01.787611+0000","last_clean":"2023-10-14T10:37:01.787611+0000","last_became_active":"2023-10-14T10:37:00.070897+0000","last_became_peered":"2023-10-14T10:37:00.070897+0000","last_unstale":"2023-10-14T10:37:01.787611+0000","last_undegraded":"2023-10-14T10:37:01.787611+0000","last_fullsized":"2023-10-14T10:37:01.787611+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:59:07.208582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.061365+0000","last_change":"2023-10-14T10:37:00.069525+0000","last_active":"2023-10-14T10:37:01.061365+0000","last_peered":"2023-10-14T10:37:01.061365+0000","last_clean":"2023-10-14T10:37:01.061365+0000","last_became_active":"2023-10-14T10:37:00.069246+0000","last_became_peered":"2023-10-14T10:37:00.069246+0000","last_unstale":"2023-10-14T10:37:01.061365+0000","last_undegraded":"2023-10-14T10:37:01.061365+0000","last_fullsized":"2023-10-14T10:37:01.061365+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:52:37.894720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.060407+0000","last_change":"2023-10-14T10:37:00.070144+0000","last_active":"2023-10-14T10:37:01.060407+0000","last_peered":"2023-10-14T10:37:01.060407+0000","last_clean":"2023-10-14T10:37:01.060407+0000","last_became_active":"2023-10-14T10:37:00.069871+0000","last_became_peered":"2023-10-14T10:37:00.069871+0000","last_unstale":"2023-10-14T10:37:01.060407+0000","last_undegraded":"2023-10-14T10:37:01.060407+0000","last_fullsized":"2023-10-14T10:37:01.060407+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:22:32.154663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788109+0000","last_change":"2023-10-14T10:37:00.076474+0000","last_active":"2023-10-14T10:37:01.788109+0000","last_peered":"2023-10-14T10:37:01.788109+0000","last_clean":"2023-10-14T10:37:01.788109+0000","last_became_active":"2023-10-14T10:37:00.076169+0000","last_became_peered":"2023-10-14T10:37:00.076169+0000","last_unstale":"2023-10-14T10:37:01.788109+0000","last_undegraded":"2023-10-14T10:37:01.788109+0000","last_fullsized":"2023-10-14T10:37:01.788109+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T14:52:18.060480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.060360+0000","last_change":"2023-10-14T10:37:00.076568+0000","last_active":"2023-10-14T10:37:01.060360+0000","last_peered":"2023-10-14T10:37:01.060360+0000","last_clean":"2023-10-14T10:37:01.060360+0000","last_became_active":"2023-10-14T10:37:00.076263+0000","last_became_peered":"2023-10-14T10:37:00.076263+0000","last_unstale":"2023-10-14T10:37:01.060360+0000","last_undegraded":"2023-10-14T10:37:01.060360+0000","last_fullsized":"2023-10-14T10:37:01.060360+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:18:45.297555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.061994+0000","last_change":"2023-10-14T10:37:00.079439+0000","last_active":"2023-10-14T10:37:01.061994+0000","last_peered":"2023-10-14T10:37:01.061994+0000","last_clean":"2023-10-14T10:37:01.061994+0000","last_became_active":"2023-10-14T10:37:00.079178+0000","last_became_peered":"2023-10-14T10:37:00.079178+0000","last_unstale":"2023-10-14T10:37:01.061994+0000","last_undegraded":"2023-10-14T10:37:01.061994+0000","last_fullsized":"2023-10-14T10:37:01.061994+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:17:54.730152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.489206+0000","last_change":"2023-10-14T10:37:00.080675+0000","last_active":"2023-10-14T10:37:01.489206+0000","last_peered":"2023-10-14T10:37:01.489206+0000","last_clean":"2023-10-14T10:37:01.489206+0000","last_became_active":"2023-10-14T10:37:00.080171+0000","last_became_peered":"2023-10-14T10:37:00.080171+0000","last_unstale":"2023-10-14T10:37:01.489206+0000","last_undegraded":"2023-10-14T10:37:01.489206+0000","last_fullsized":"2023-10-14T10:37:01.489206+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:28:04.536948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.061814+0000","last_change":"2023-10-14T10:37:00.072155+0000","last_active":"2023-10-14T10:37:01.061814+0000","last_peered":"2023-10-14T10:37:01.061814+0000","last_clean":"2023-10-14T10:37:01.061814+0000","last_became_active":"2023-10-14T10:37:00.071882+0000","last_became_peered":"2023-10-14T10:37:00.071882+0000","last_unstale":"2023-10-14T10:37:01.061814+0000","last_undegraded":"2023-10-14T10:37:01.061814+0000","last_fullsized":"2023-10-14T10:37:01.061814+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T10:48:03.043383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.787458+0000","last_change":"2023-10-14T10:37:00.081816+0000","last_active":"2023-10-14T10:37:01.787458+0000","last_peered":"2023-10-14T10:37:01.787458+0000","last_clean":"2023-10-14T10:37:01.787458+0000","last_became_active":"2023-10-14T10:37:00.081312+0000","last_became_peered":"2023-10-14T10:37:00.081312+0000","last_unstale":"2023-10-14T10:37:01.787458+0000","last_undegraded":"2023-10-14T10:37:01.787458+0000","last_fullsized":"2023-10-14T10:37:01.787458+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:58:56.558710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.061373+0000","last_change":"2023-10-14T10:37:00.078914+0000","last_active":"2023-10-14T10:37:01.061373+0000","last_peered":"2023-10-14T10:37:01.061373+0000","last_clean":"2023-10-14T10:37:01.061373+0000","last_became_active":"2023-10-14T10:37:00.078441+0000","last_became_peered":"2023-10-14T10:37:00.078441+0000","last_unstale":"2023-10-14T10:37:01.061373+0000","last_undegraded":"2023-10-14T10:37:01.061373+0000","last_fullsized":"2023-10-14T10:37:01.061373+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:22:32.154663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.787754+0000","last_change":"2023-10-14T10:37:00.071246+0000","last_active":"2023-10-14T10:37:01.787754+0000","last_peered":"2023-10-14T10:37:01.787754+0000","last_clean":"2023-10-14T10:37:01.787754+0000","last_became_active":"2023-10-14T10:37:00.070945+0000","last_became_peered":"2023-10-14T10:37:00.070945+0000","last_unstale":"2023-10-14T10:37:01.787754+0000","last_undegraded":"2023-10-14T10:37:01.787754+0000","last_fullsized":"2023-10-14T10:37:01.787754+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:30:57.784792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.787645+0000","last_change":"2023-10-14T10:36:59.066772+0000","last_active":"2023-10-14T10:37:01.787645+0000","last_peered":"2023-10-14T10:37:01.787645+0000","last_clean":"2023-10-14T10:37:01.787645+0000","last_became_active":"2023-10-14T10:36:59.066488+0000","last_became_peered":"2023-10-14T10:36:59.066488+0000","last_unstale":"2023-10-14T10:37:01.787645+0000","last_undegraded":"2023-10-14T10:37:01.787645+0000","last_fullsized":"2023-10-14T10:37:01.787645+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:39:01.994329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.787569+0000","last_change":"2023-10-14T10:37:00.077997+0000","last_active":"2023-10-14T10:37:01.787569+0000","last_peered":"2023-10-14T10:37:01.787569+0000","last_clean":"2023-10-14T10:37:01.787569+0000","last_became_active":"2023-10-14T10:37:00.077734+0000","last_became_peered":"2023-10-14T10:37:00.077734+0000","last_unstale":"2023-10-14T10:37:01.787569+0000","last_undegraded":"2023-10-14T10:37:01.787569+0000","last_fullsized":"2023-10-14T10:37:01.787569+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:28:45.821377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.787856+0000","last_change":"2023-10-14T10:36:59.070768+0000","last_active":"2023-10-14T10:37:01.787856+0000","last_peered":"2023-10-14T10:37:01.787856+0000","last_clean":"2023-10-14T10:37:01.787856+0000","last_became_active":"2023-10-14T10:36:59.070463+0000","last_became_peered":"2023-10-14T10:36:59.070463+0000","last_unstale":"2023-10-14T10:37:01.787856+0000","last_undegraded":"2023-10-14T10:37:01.787856+0000","last_fullsized":"2023-10-14T10:37:01.787856+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:08:05.335325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.489357+0000","last_change":"2023-10-14T10:37:00.080768+0000","last_active":"2023-10-14T10:37:01.489357+0000","last_peered":"2023-10-14T10:37:01.489357+0000","last_clean":"2023-10-14T10:37:01.489357+0000","last_became_active":"2023-10-14T10:37:00.080160+0000","last_became_peered":"2023-10-14T10:37:00.080160+0000","last_unstale":"2023-10-14T10:37:01.489357+0000","last_undegraded":"2023-10-14T10:37:01.489357+0000","last_fullsized":"2023-10-14T10:37:01.489357+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T20:27:28.798883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788006+0000","last_change":"2023-10-14T10:36:59.067900+0000","last_active":"2023-10-14T10:37:01.788006+0000","last_peered":"2023-10-14T10:37:01.788006+0000","last_clean":"2023-10-14T10:37:01.788006+0000","last_became_active":"2023-10-14T10:36:59.067616+0000","last_became_peered":"2023-10-14T10:36:59.067616+0000","last_unstale":"2023-10-14T10:37:01.788006+0000","last_undegraded":"2023-10-14T10:37:01.788006+0000","last_fullsized":"2023-10-14T10:37:01.788006+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:28:44.815010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.060586+0000","last_change":"2023-10-14T10:37:00.079200+0000","last_active":"2023-10-14T10:37:01.060586+0000","last_peered":"2023-10-14T10:37:01.060586+0000","last_clean":"2023-10-14T10:37:01.060586+0000","last_became_active":"2023-10-14T10:37:00.078829+0000","last_became_peered":"2023-10-14T10:37:00.078829+0000","last_unstale":"2023-10-14T10:37:01.060586+0000","last_undegraded":"2023-10-14T10:37:01.060586+0000","last_fullsized":"2023-10-14T10:37:01.060586+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:31:16.127752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788507+0000","last_change":"2023-10-14T10:36:59.065882+0000","last_active":"2023-10-14T10:37:01.788507+0000","last_peered":"2023-10-14T10:37:01.788507+0000","last_clean":"2023-10-14T10:37:01.788507+0000","last_became_active":"2023-10-14T10:36:59.065608+0000","last_became_peered":"2023-10-14T10:36:59.065608+0000","last_unstale":"2023-10-14T10:37:01.788507+0000","last_undegraded":"2023-10-14T10:37:01.788507+0000","last_fullsized":"2023-10-14T10:37:01.788507+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T21:18:46.547776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.062326+0000","last_change":"2023-10-14T10:37:00.077115+0000","last_active":"2023-10-14T10:37:01.062326+0000","last_peered":"2023-10-14T10:37:01.062326+0000","last_clean":"2023-10-14T10:37:01.062326+0000","last_became_active":"2023-10-14T10:37:00.076840+0000","last_became_peered":"2023-10-14T10:37:00.076840+0000","last_unstale":"2023-10-14T10:37:01.062326+0000","last_undegraded":"2023-10-14T10:37:01.062326+0000","last_fullsized":"2023-10-14T10:37:01.062326+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T12:47:24.725885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.489315+0000","last_change":"2023-10-14T10:36:59.069533+0000","last_active":"2023-10-14T10:37:01.489315+0000","last_peered":"2023-10-14T10:37:01.489315+0000","last_clean":"2023-10-14T10:37:01.489315+0000","last_became_active":"2023-10-14T10:36:59.069192+0000","last_became_peered":"2023-10-14T10:36:59.069192+0000","last_unstale":"2023-10-14T10:37:01.489315+0000","last_undegraded":"2023-10-14T10:37:01.489315+0000","last_fullsized":"2023-10-14T10:37:01.489315+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T21:39:34.581362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.061634+0000","last_change":"2023-10-14T10:37:00.068786+0000","last_active":"2023-10-14T10:37:01.061634+0000","last_peered":"2023-10-14T10:37:01.061634+0000","last_clean":"2023-10-14T10:37:01.061634+0000","last_became_active":"2023-10-14T10:37:00.068517+0000","last_became_peered":"2023-10-14T10:37:00.068517+0000","last_unstale":"2023-10-14T10:37:01.061634+0000","last_undegraded":"2023-10-14T10:37:01.061634+0000","last_fullsized":"2023-10-14T10:37:01.061634+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:22:32.154663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.489303+0000","last_change":"2023-10-14T10:36:59.069445+0000","last_active":"2023-10-14T10:37:01.489303+0000","last_peered":"2023-10-14T10:37:01.489303+0000","last_clean":"2023-10-14T10:37:01.489303+0000","last_became_active":"2023-10-14T10:36:59.069151+0000","last_became_peered":"2023-10-14T10:36:59.069151+0000","last_unstale":"2023-10-14T10:37:01.489303+0000","last_undegraded":"2023-10-14T10:37:01.489303+0000","last_fullsized":"2023-10-14T10:37:01.489303+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:54:15.011628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.488953+0000","last_change":"2023-10-14T10:37:00.076984+0000","last_active":"2023-10-14T10:37:01.488953+0000","last_peered":"2023-10-14T10:37:01.488953+0000","last_clean":"2023-10-14T10:37:01.488953+0000","last_became_active":"2023-10-14T10:37:00.076706+0000","last_became_peered":"2023-10-14T10:37:00.076706+0000","last_unstale":"2023-10-14T10:37:01.488953+0000","last_undegraded":"2023-10-14T10:37:01.488953+0000","last_fullsized":"2023-10-14T10:37:01.488953+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:30:57.784792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.787709+0000","last_change":"2023-10-14T10:36:59.062324+0000","last_active":"2023-10-14T10:37:01.787709+0000","last_peered":"2023-10-14T10:37:01.787709+0000","last_clean":"2023-10-14T10:37:01.787709+0000","last_became_active":"2023-10-14T10:36:59.062075+0000","last_became_peered":"2023-10-14T10:36:59.062075+0000","last_unstale":"2023-10-14T10:37:01.787709+0000","last_undegraded":"2023-10-14T10:37:01.787709+0000","last_fullsized":"2023-10-14T10:37:01.787709+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:45:50.919718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.059848+0000","last_change":"2023-10-14T10:37:00.067486+0000","last_active":"2023-10-14T10:37:01.059848+0000","last_peered":"2023-10-14T10:37:01.059848+0000","last_clean":"2023-10-14T10:37:01.059848+0000","last_became_active":"2023-10-14T10:37:00.067187+0000","last_became_peered":"2023-10-14T10:37:00.067187+0000","last_unstale":"2023-10-14T10:37:01.059848+0000","last_undegraded":"2023-10-14T10:37:01.059848+0000","last_fullsized":"2023-10-14T10:37:01.059848+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:10:46.473503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.488998+0000","last_change":"2023-10-14T10:36:59.066330+0000","last_active":"2023-10-14T10:37:01.488998+0000","last_peered":"2023-10-14T10:37:01.488998+0000","last_clean":"2023-10-14T10:37:01.488998+0000","last_became_active":"2023-10-14T10:36:59.064990+0000","last_became_peered":"2023-10-14T10:36:59.064990+0000","last_unstale":"2023-10-14T10:37:01.488998+0000","last_undegraded":"2023-10-14T10:37:01.488998+0000","last_fullsized":"2023-10-14T10:37:01.488998+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T12:15:16.823069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.061974+0000","last_change":"2023-10-14T10:37:00.077465+0000","last_active":"2023-10-14T10:37:01.061974+0000","last_peered":"2023-10-14T10:37:01.061974+0000","last_clean":"2023-10-14T10:37:01.061974+0000","last_became_active":"2023-10-14T10:37:00.077152+0000","last_became_peered":"2023-10-14T10:37:00.077152+0000","last_unstale":"2023-10-14T10:37:01.061974+0000","last_undegraded":"2023-10-14T10:37:01.061974+0000","last_fullsized":"2023-10-14T10:37:01.061974+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:51:00.167498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788575+0000","last_change":"2023-10-14T10:36:59.061903+0000","last_active":"2023-10-14T10:37:01.788575+0000","last_peered":"2023-10-14T10:37:01.788575+0000","last_clean":"2023-10-14T10:37:01.788575+0000","last_became_active":"2023-10-14T10:36:59.061597+0000","last_became_peered":"2023-10-14T10:36:59.061597+0000","last_unstale":"2023-10-14T10:37:01.788575+0000","last_undegraded":"2023-10-14T10:37:01.788575+0000","last_fullsized":"2023-10-14T10:37:01.788575+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:22:31.148297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788727+0000","last_change":"2023-10-14T10:37:00.076213+0000","last_active":"2023-10-14T10:37:01.788727+0000","last_peered":"2023-10-14T10:37:01.788727+0000","last_clean":"2023-10-14T10:37:01.788727+0000","last_became_active":"2023-10-14T10:37:00.075889+0000","last_became_peered":"2023-10-14T10:37:00.075889+0000","last_unstale":"2023-10-14T10:37:01.788727+0000","last_undegraded":"2023-10-14T10:37:01.788727+0000","last_fullsized":"2023-10-14T10:37:01.788727+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T14:19:22.310866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.787968+0000","last_change":"2023-10-14T10:36:59.070183+0000","last_active":"2023-10-14T10:37:01.787968+0000","last_peered":"2023-10-14T10:37:01.787968+0000","last_clean":"2023-10-14T10:37:01.787968+0000","last_became_active":"2023-10-14T10:36:59.069731+0000","last_became_peered":"2023-10-14T10:36:59.069731+0000","last_unstale":"2023-10-14T10:37:01.787968+0000","last_undegraded":"2023-10-14T10:37:01.787968+0000","last_fullsized":"2023-10-14T10:37:01.787968+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:58:55.552343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788210+0000","last_change":"2023-10-14T10:37:00.072592+0000","last_active":"2023-10-14T10:37:01.788210+0000","last_peered":"2023-10-14T10:37:01.788210+0000","last_clean":"2023-10-14T10:37:01.788210+0000","last_became_active":"2023-10-14T10:37:00.072288+0000","last_became_peered":"2023-10-14T10:37:00.072288+0000","last_unstale":"2023-10-14T10:37:01.788210+0000","last_undegraded":"2023-10-14T10:37:01.788210+0000","last_fullsized":"2023-10-14T10:37:01.788210+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T17:27:50.406084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.062406+0000","last_change":"2023-10-14T10:36:59.062120+0000","last_active":"2023-10-14T10:37:01.062406+0000","last_peered":"2023-10-14T10:37:01.062406+0000","last_clean":"2023-10-14T10:37:01.062406+0000","last_became_active":"2023-10-14T10:36:59.061700+0000","last_became_peered":"2023-10-14T10:36:59.061700+0000","last_unstale":"2023-10-14T10:37:01.062406+0000","last_undegraded":"2023-10-14T10:37:01.062406+0000","last_fullsized":"2023-10-14T10:37:01.062406+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:45:24.296819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.062364+0000","last_change":"2023-10-14T10:37:00.074977+0000","last_active":"2023-10-14T10:37:01.062364+0000","last_peered":"2023-10-14T10:37:01.062364+0000","last_clean":"2023-10-14T10:37:01.062364+0000","last_became_active":"2023-10-14T10:37:00.074700+0000","last_became_peered":"2023-10-14T10:37:00.074700+0000","last_unstale":"2023-10-14T10:37:01.062364+0000","last_undegraded":"2023-10-14T10:37:01.062364+0000","last_fullsized":"2023-10-14T10:37:01.062364+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T18:57:50.136605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788323+0000","last_change":"2023-10-14T10:36:59.069400+0000","last_active":"2023-10-14T10:37:01.788323+0000","last_peered":"2023-10-14T10:37:01.788323+0000","last_clean":"2023-10-14T10:37:01.788323+0000","last_became_active":"2023-10-14T10:36:59.069043+0000","last_became_peered":"2023-10-14T10:36:59.069043+0000","last_unstale":"2023-10-14T10:37:01.788323+0000","last_undegraded":"2023-10-14T10:37:01.788323+0000","last_fullsized":"2023-10-14T10:37:01.788323+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:10:45.467136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.060040+0000","last_change":"2023-10-14T10:37:00.068613+0000","last_active":"2023-10-14T10:37:01.060040+0000","last_peered":"2023-10-14T10:37:01.060040+0000","last_clean":"2023-10-14T10:37:01.060040+0000","last_became_active":"2023-10-14T10:37:00.065345+0000","last_became_peered":"2023-10-14T10:37:00.065345+0000","last_unstale":"2023-10-14T10:37:01.060040+0000","last_undegraded":"2023-10-14T10:37:01.060040+0000","last_fullsized":"2023-10-14T10:37:01.060040+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:45:25.303185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.489387+0000","last_change":"2023-10-14T10:36:59.065846+0000","last_active":"2023-10-14T10:37:01.489387+0000","last_peered":"2023-10-14T10:37:01.489387+0000","last_clean":"2023-10-14T10:37:01.489387+0000","last_became_active":"2023-10-14T10:36:59.065569+0000","last_became_peered":"2023-10-14T10:36:59.065569+0000","last_unstale":"2023-10-14T10:37:01.489387+0000","last_undegraded":"2023-10-14T10:37:01.489387+0000","last_fullsized":"2023-10-14T10:37:01.489387+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:28:44.815010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788692+0000","last_change":"2023-10-14T10:37:00.079137+0000","last_active":"2023-10-14T10:37:01.788692+0000","last_peered":"2023-10-14T10:37:01.788692+0000","last_clean":"2023-10-14T10:37:01.788692+0000","last_became_active":"2023-10-14T10:37:00.078844+0000","last_became_peered":"2023-10-14T10:37:00.078844+0000","last_unstale":"2023-10-14T10:37:01.788692+0000","last_undegraded":"2023-10-14T10:37:01.788692+0000","last_fullsized":"2023-10-14T10:37:01.788692+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:30:51.914014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.059920+0000","last_change":"2023-10-14T10:36:59.067245+0000","last_active":"2023-10-14T10:37:01.059920+0000","last_peered":"2023-10-14T10:37:01.059920+0000","last_clean":"2023-10-14T10:37:01.059920+0000","last_became_active":"2023-10-14T10:36:59.066952+0000","last_became_peered":"2023-10-14T10:36:59.066952+0000","last_unstale":"2023-10-14T10:37:01.059920+0000","last_undegraded":"2023-10-14T10:37:01.059920+0000","last_fullsized":"2023-10-14T10:37:01.059920+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T12:08:55.620308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.062454+0000","last_change":"2023-10-14T10:37:00.078646+0000","last_active":"2023-10-14T10:37:01.062454+0000","last_peered":"2023-10-14T10:37:01.062454+0000","last_clean":"2023-10-14T10:37:01.062454+0000","last_became_active":"2023-10-14T10:37:00.078353+0000","last_became_peered":"2023-10-14T10:37:00.078353+0000","last_unstale":"2023-10-14T10:37:01.062454+0000","last_undegraded":"2023-10-14T10:37:01.062454+0000","last_fullsized":"2023-10-14T10:37:01.062454+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:30:51.914014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788938+0000","last_change":"2023-10-14T10:36:59.063118+0000","last_active":"2023-10-14T10:37:01.788938+0000","last_peered":"2023-10-14T10:37:01.788938+0000","last_clean":"2023-10-14T10:37:01.788938+0000","last_became_active":"2023-10-14T10:36:59.062380+0000","last_became_peered":"2023-10-14T10:36:59.062380+0000","last_unstale":"2023-10-14T10:37:01.788938+0000","last_undegraded":"2023-10-14T10:37:01.788938+0000","last_fullsized":"2023-10-14T10:37:01.788938+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T21:39:34.581362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.787507+0000","last_change":"2023-10-14T10:37:00.078077+0000","last_active":"2023-10-14T10:37:01.787507+0000","last_peered":"2023-10-14T10:37:01.787507+0000","last_clean":"2023-10-14T10:37:01.787507+0000","last_became_active":"2023-10-14T10:37:00.077806+0000","last_became_peered":"2023-10-14T10:37:00.077806+0000","last_unstale":"2023-10-14T10:37:01.787507+0000","last_undegraded":"2023-10-14T10:37:01.787507+0000","last_fullsized":"2023-10-14T10:37:01.787507+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:45:51.926085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788260+0000","last_change":"2023-10-14T10:36:59.068012+0000","last_active":"2023-10-14T10:37:01.788260+0000","last_peered":"2023-10-14T10:37:01.788260+0000","last_clean":"2023-10-14T10:37:01.788260+0000","last_became_active":"2023-10-14T10:36:59.067727+0000","last_became_peered":"2023-10-14T10:36:59.067727+0000","last_unstale":"2023-10-14T10:37:01.788260+0000","last_undegraded":"2023-10-14T10:37:01.788260+0000","last_fullsized":"2023-10-14T10:37:01.788260+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:44:49.671921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.489522+0000","last_change":"2023-10-14T10:37:00.076926+0000","last_active":"2023-10-14T10:37:01.489522+0000","last_peered":"2023-10-14T10:37:01.489522+0000","last_clean":"2023-10-14T10:37:01.489522+0000","last_became_active":"2023-10-14T10:37:00.076658+0000","last_became_peered":"2023-10-14T10:37:00.076658+0000","last_unstale":"2023-10-14T10:37:01.489522+0000","last_undegraded":"2023-10-14T10:37:01.489522+0000","last_fullsized":"2023-10-14T10:37:01.489522+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:52:37.894720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.060490+0000","last_change":"2023-10-14T10:36:59.071714+0000","last_active":"2023-10-14T10:37:01.060490+0000","last_peered":"2023-10-14T10:37:01.060490+0000","last_clean":"2023-10-14T10:37:01.060490+0000","last_became_active":"2023-10-14T10:36:59.071406+0000","last_became_peered":"2023-10-14T10:36:59.071406+0000","last_unstale":"2023-10-14T10:37:01.060490+0000","last_undegraded":"2023-10-14T10:37:01.060490+0000","last_fullsized":"2023-10-14T10:37:01.060490+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:59:06.202215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.062345+0000","last_change":"2023-10-14T10:37:00.080060+0000","last_active":"2023-10-14T10:37:01.062345+0000","last_peered":"2023-10-14T10:37:01.062345+0000","last_clean":"2023-10-14T10:37:01.062345+0000","last_became_active":"2023-10-14T10:37:00.079795+0000","last_became_peered":"2023-10-14T10:37:00.079795+0000","last_unstale":"2023-10-14T10:37:01.062345+0000","last_undegraded":"2023-10-14T10:37:01.062345+0000","last_fullsized":"2023-10-14T10:37:01.062345+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T17:27:50.406084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.489554+0000","last_change":"2023-10-14T10:36:59.065271+0000","last_active":"2023-10-14T10:37:01.489554+0000","last_peered":"2023-10-14T10:37:01.489554+0000","last_clean":"2023-10-14T10:37:01.489554+0000","last_became_active":"2023-10-14T10:36:59.064959+0000","last_became_peered":"2023-10-14T10:36:59.064959+0000","last_unstale":"2023-10-14T10:37:01.489554+0000","last_undegraded":"2023-10-14T10:37:01.489554+0000","last_fullsized":"2023-10-14T10:37:01.489554+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T21:56:55.137838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.060192+0000","last_change":"2023-10-14T10:37:00.077751+0000","last_active":"2023-10-14T10:37:01.060192+0000","last_peered":"2023-10-14T10:37:01.060192+0000","last_clean":"2023-10-14T10:37:01.060192+0000","last_became_active":"2023-10-14T10:37:00.077459+0000","last_became_peered":"2023-10-14T10:37:00.077459+0000","last_unstale":"2023-10-14T10:37:01.060192+0000","last_undegraded":"2023-10-14T10:37:01.060192+0000","last_fullsized":"2023-10-14T10:37:01.060192+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T20:27:28.798883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.489473+0000","last_change":"2023-10-14T10:36:59.058893+0000","last_active":"2023-10-14T10:37:01.489473+0000","last_peered":"2023-10-14T10:37:01.489473+0000","last_clean":"2023-10-14T10:37:01.489473+0000","last_became_active":"2023-10-14T10:36:59.058558+0000","last_became_peered":"2023-10-14T10:36:59.058558+0000","last_unstale":"2023-10-14T10:37:01.489473+0000","last_undegraded":"2023-10-14T10:37:01.489473+0000","last_fullsized":"2023-10-14T10:37:01.489473+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:45:50.919718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788693+0000","last_change":"2023-10-14T10:37:00.071398+0000","last_active":"2023-10-14T10:37:01.788693+0000","last_peered":"2023-10-14T10:37:01.788693+0000","last_clean":"2023-10-14T10:37:01.788693+0000","last_became_active":"2023-10-14T10:37:00.071167+0000","last_became_peered":"2023-10-14T10:37:00.071167+0000","last_unstale":"2023-10-14T10:37:01.788693+0000","last_undegraded":"2023-10-14T10:37:01.788693+0000","last_fullsized":"2023-10-14T10:37:01.788693+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:31:12.123956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.489473+0000","last_change":"2023-10-14T10:36:59.069420+0000","last_active":"2023-10-14T10:37:01.489473+0000","last_peered":"2023-10-14T10:37:01.489473+0000","last_clean":"2023-10-14T10:37:01.489473+0000","last_became_active":"2023-10-14T10:36:59.069110+0000","last_became_peered":"2023-10-14T10:36:59.069110+0000","last_unstale":"2023-10-14T10:37:01.489473+0000","last_undegraded":"2023-10-14T10:37:01.489473+0000","last_fullsized":"2023-10-14T10:37:01.489473+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:09:59.812757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788586+0000","last_change":"2023-10-14T10:37:00.077550+0000","last_active":"2023-10-14T10:37:01.788586+0000","last_peered":"2023-10-14T10:37:01.788586+0000","last_clean":"2023-10-14T10:37:01.788586+0000","last_became_active":"2023-10-14T10:37:00.077175+0000","last_became_peered":"2023-10-14T10:37:00.077175+0000","last_unstale":"2023-10-14T10:37:01.788586+0000","last_undegraded":"2023-10-14T10:37:01.788586+0000","last_fullsized":"2023-10-14T10:37:01.788586+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T14:51:19.968923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.787861+0000","last_change":"2023-10-14T10:36:59.067621+0000","last_active":"2023-10-14T10:37:01.787861+0000","last_peered":"2023-10-14T10:37:01.787861+0000","last_clean":"2023-10-14T10:37:01.787861+0000","last_became_active":"2023-10-14T10:36:59.067345+0000","last_became_peered":"2023-10-14T10:36:59.067345+0000","last_unstale":"2023-10-14T10:37:01.787861+0000","last_undegraded":"2023-10-14T10:37:01.787861+0000","last_fullsized":"2023-10-14T10:37:01.787861+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:54:15.011628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788501+0000","last_change":"2023-10-14T10:37:00.077744+0000","last_active":"2023-10-14T10:37:01.788501+0000","last_peered":"2023-10-14T10:37:01.788501+0000","last_clean":"2023-10-14T10:37:01.788501+0000","last_became_active":"2023-10-14T10:37:00.077465+0000","last_became_peered":"2023-10-14T10:37:00.077465+0000","last_unstale":"2023-10-14T10:37:01.788501+0000","last_undegraded":"2023-10-14T10:37:01.788501+0000","last_fullsized":"2023-10-14T10:37:01.788501+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T20:36:23.971821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.787432+0000","last_change":"2023-10-14T10:36:59.069707+0000","last_active":"2023-10-14T10:37:01.787432+0000","last_peered":"2023-10-14T10:37:01.787432+0000","last_clean":"2023-10-14T10:37:01.787432+0000","last_became_active":"2023-10-14T10:36:59.069269+0000","last_became_peered":"2023-10-14T10:36:59.069269+0000","last_unstale":"2023-10-14T10:37:01.787432+0000","last_undegraded":"2023-10-14T10:37:01.787432+0000","last_fullsized":"2023-10-14T10:37:01.787432+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T12:15:16.823069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788221+0000","last_change":"2023-10-14T10:37:00.068887+0000","last_active":"2023-10-14T10:37:01.788221+0000","last_peered":"2023-10-14T10:37:01.788221+0000","last_clean":"2023-10-14T10:37:01.788221+0000","last_became_active":"2023-10-14T10:37:00.068620+0000","last_became_peered":"2023-10-14T10:37:00.068620+0000","last_unstale":"2023-10-14T10:37:01.788221+0000","last_undegraded":"2023-10-14T10:37:01.788221+0000","last_fullsized":"2023-10-14T10:37:01.788221+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T18:04:59.338050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.489058+0000","last_change":"2023-10-14T10:36:59.061947+0000","last_active":"2023-10-14T10:37:01.489058+0000","last_peered":"2023-10-14T10:37:01.489058+0000","last_clean":"2023-10-14T10:37:01.489058+0000","last_became_active":"2023-10-14T10:36:59.061678+0000","last_became_peered":"2023-10-14T10:36:59.061678+0000","last_unstale":"2023-10-14T10:37:01.489058+0000","last_undegraded":"2023-10-14T10:37:01.489058+0000","last_fullsized":"2023-10-14T10:37:01.489058+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:39:01.994329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.061998+0000","last_change":"2023-10-14T10:37:00.073204+0000","last_active":"2023-10-14T10:37:01.061998+0000","last_peered":"2023-10-14T10:37:01.061998+0000","last_clean":"2023-10-14T10:37:01.061998+0000","last_became_active":"2023-10-14T10:37:00.071692+0000","last_became_peered":"2023-10-14T10:37:00.071692+0000","last_unstale":"2023-10-14T10:37:01.061998+0000","last_undegraded":"2023-10-14T10:37:01.061998+0000","last_fullsized":"2023-10-14T10:37:01.061998+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:17:54.730152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.061467+0000","last_change":"2023-10-14T10:36:59.063545+0000","last_active":"2023-10-14T10:37:01.061467+0000","last_peered":"2023-10-14T10:37:01.061467+0000","last_clean":"2023-10-14T10:37:01.061467+0000","last_became_active":"2023-10-14T10:36:59.063332+0000","last_became_peered":"2023-10-14T10:36:59.063332+0000","last_unstale":"2023-10-14T10:37:01.061467+0000","last_undegraded":"2023-10-14T10:37:01.061467+0000","last_fullsized":"2023-10-14T10:37:01.061467+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:10:45.467136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788558+0000","last_change":"2023-10-14T10:37:00.075249+0000","last_active":"2023-10-14T10:37:01.788558+0000","last_peered":"2023-10-14T10:37:01.788558+0000","last_clean":"2023-10-14T10:37:01.788558+0000","last_became_active":"2023-10-14T10:37:00.074859+0000","last_became_peered":"2023-10-14T10:37:00.074859+0000","last_unstale":"2023-10-14T10:37:01.788558+0000","last_undegraded":"2023-10-14T10:37:01.788558+0000","last_fullsized":"2023-10-14T10:37:01.788558+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:04:51.088268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.060369+0000","last_change":"2023-10-14T10:36:59.069418+0000","last_active":"2023-10-14T10:37:01.060369+0000","last_peered":"2023-10-14T10:37:01.060369+0000","last_clean":"2023-10-14T10:37:01.060369+0000","last_became_active":"2023-10-14T10:36:59.069037+0000","last_became_peered":"2023-10-14T10:36:59.069037+0000","last_unstale":"2023-10-14T10:37:01.060369+0000","last_undegraded":"2023-10-14T10:37:01.060369+0000","last_fullsized":"2023-10-14T10:37:01.060369+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:45:50.919718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.062253+0000","last_change":"2023-10-14T10:37:00.075731+0000","last_active":"2023-10-14T10:37:01.062253+0000","last_peered":"2023-10-14T10:37:01.062253+0000","last_clean":"2023-10-14T10:37:01.062253+0000","last_became_active":"2023-10-14T10:37:00.075491+0000","last_became_peered":"2023-10-14T10:37:00.075491+0000","last_unstale":"2023-10-14T10:37:01.062253+0000","last_undegraded":"2023-10-14T10:37:01.062253+0000","last_fullsized":"2023-10-14T10:37:01.062253+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T14:52:18.060480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"33'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.821131+0000","last_change":"2023-10-14T10:36:59.066461+0000","last_active":"2023-10-14T10:37:01.821131+0000","last_peered":"2023-10-14T10:37:01.821131+0000","last_clean":"2023-10-14T10:37:01.821131+0000","last_became_active":"2023-10-14T10:36:59.066146+0000","last_became_peered":"2023-10-14T10:36:59.066146+0000","last_unstale":"2023-10-14T10:37:01.821131+0000","last_undegraded":"2023-10-14T10:37:01.821131+0000","last_fullsized":"2023-10-14T10:37:01.821131+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:45:50.919718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.787608+0000","last_change":"2023-10-14T10:37:00.082851+0000","last_active":"2023-10-14T10:37:01.787608+0000","last_peered":"2023-10-14T10:37:01.787608+0000","last_clean":"2023-10-14T10:37:01.787608+0000","last_became_active":"2023-10-14T10:37:00.082570+0000","last_became_peered":"2023-10-14T10:37:00.082570+0000","last_unstale":"2023-10-14T10:37:01.787608+0000","last_undegraded":"2023-10-14T10:37:01.787608+0000","last_fullsized":"2023-10-14T10:37:01.787608+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:28:45.821377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.059981+0000","last_change":"2023-10-14T10:36:59.061936+0000","last_active":"2023-10-14T10:37:01.059981+0000","last_peered":"2023-10-14T10:37:01.059981+0000","last_clean":"2023-10-14T10:37:01.059981+0000","last_became_active":"2023-10-14T10:36:59.061642+0000","last_became_peered":"2023-10-14T10:36:59.061642+0000","last_unstale":"2023-10-14T10:37:01.059981+0000","last_undegraded":"2023-10-14T10:37:01.059981+0000","last_fullsized":"2023-10-14T10:37:01.059981+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:54:15.011628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.787449+0000","last_change":"2023-10-14T10:37:00.071551+0000","last_active":"2023-10-14T10:37:01.787449+0000","last_peered":"2023-10-14T10:37:01.787449+0000","last_clean":"2023-10-14T10:37:01.787449+0000","last_became_active":"2023-10-14T10:37:00.071244+0000","last_became_peered":"2023-10-14T10:37:00.071244+0000","last_unstale":"2023-10-14T10:37:01.787449+0000","last_undegraded":"2023-10-14T10:37:01.787449+0000","last_fullsized":"2023-10-14T10:37:01.787449+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:08:06.341692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.787778+0000","last_change":"2023-10-14T10:36:59.066839+0000","last_active":"2023-10-14T10:37:01.787778+0000","last_peered":"2023-10-14T10:37:01.787778+0000","last_clean":"2023-10-14T10:37:01.787778+0000","last_became_active":"2023-10-14T10:36:59.066592+0000","last_became_peered":"2023-10-14T10:36:59.066592+0000","last_unstale":"2023-10-14T10:37:01.787778+0000","last_undegraded":"2023-10-14T10:37:01.787778+0000","last_fullsized":"2023-10-14T10:37:01.787778+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T12:15:16.823069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788013+0000","last_change":"2023-10-14T10:37:00.076076+0000","last_active":"2023-10-14T10:37:01.788013+0000","last_peered":"2023-10-14T10:37:01.788013+0000","last_clean":"2023-10-14T10:37:01.788013+0000","last_became_active":"2023-10-14T10:37:00.075794+0000","last_became_peered":"2023-10-14T10:37:00.075794+0000","last_unstale":"2023-10-14T10:37:01.788013+0000","last_undegraded":"2023-10-14T10:37:01.788013+0000","last_fullsized":"2023-10-14T10:37:01.788013+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:44:50.678288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.060021+0000","last_change":"2023-10-14T10:36:59.065626+0000","last_active":"2023-10-14T10:37:01.060021+0000","last_peered":"2023-10-14T10:37:01.060021+0000","last_clean":"2023-10-14T10:37:01.060021+0000","last_became_active":"2023-10-14T10:36:59.065163+0000","last_became_peered":"2023-10-14T10:36:59.065163+0000","last_unstale":"2023-10-14T10:37:01.060021+0000","last_undegraded":"2023-10-14T10:37:01.060021+0000","last_fullsized":"2023-10-14T10:37:01.060021+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:08:05.335325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788427+0000","last_change":"2023-10-14T10:37:00.078024+0000","last_active":"2023-10-14T10:37:01.788427+0000","last_peered":"2023-10-14T10:37:01.788427+0000","last_clean":"2023-10-14T10:37:01.788427+0000","last_became_active":"2023-10-14T10:37:00.077621+0000","last_became_peered":"2023-10-14T10:37:00.077621+0000","last_unstale":"2023-10-14T10:37:01.788427+0000","last_undegraded":"2023-10-14T10:37:01.788427+0000","last_fullsized":"2023-10-14T10:37:01.788427+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:06:28.391046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.489094+0000","last_change":"2023-10-14T10:36:59.064632+0000","last_active":"2023-10-14T10:37:01.489094+0000","last_peered":"2023-10-14T10:37:01.489094+0000","last_clean":"2023-10-14T10:37:01.489094+0000","last_became_active":"2023-10-14T10:36:59.064336+0000","last_became_peered":"2023-10-14T10:36:59.064336+0000","last_unstale":"2023-10-14T10:37:01.489094+0000","last_undegraded":"2023-10-14T10:37:01.489094+0000","last_fullsized":"2023-10-14T10:37:01.489094+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:08:05.335325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.488971+0000","last_change":"2023-10-14T10:37:00.081042+0000","last_active":"2023-10-14T10:37:01.488971+0000","last_peered":"2023-10-14T10:37:01.488971+0000","last_clean":"2023-10-14T10:37:01.488971+0000","last_became_active":"2023-10-14T10:37:00.080340+0000","last_became_peered":"2023-10-14T10:37:00.080340+0000","last_unstale":"2023-10-14T10:37:01.488971+0000","last_undegraded":"2023-10-14T10:37:01.488971+0000","last_fullsized":"2023-10-14T10:37:01.488971+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:44:50.678288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788029+0000","last_change":"2023-10-14T10:36:59.063568+0000","last_active":"2023-10-14T10:37:01.788029+0000","last_peered":"2023-10-14T10:37:01.788029+0000","last_clean":"2023-10-14T10:37:01.788029+0000","last_became_active":"2023-10-14T10:36:59.063130+0000","last_became_peered":"2023-10-14T10:36:59.063130+0000","last_unstale":"2023-10-14T10:37:01.788029+0000","last_undegraded":"2023-10-14T10:37:01.788029+0000","last_fullsized":"2023-10-14T10:37:01.788029+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:59:06.202215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.061998+0000","last_change":"2023-10-14T10:37:00.068870+0000","last_active":"2023-10-14T10:37:01.061998+0000","last_peered":"2023-10-14T10:37:01.061998+0000","last_clean":"2023-10-14T10:37:01.061998+0000","last_became_active":"2023-10-14T10:37:00.068556+0000","last_became_peered":"2023-10-14T10:37:00.068556+0000","last_unstale":"2023-10-14T10:37:01.061998+0000","last_undegraded":"2023-10-14T10:37:01.061998+0000","last_fullsized":"2023-10-14T10:37:01.061998+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T14:52:18.060480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.787484+0000","last_change":"2023-10-14T10:36:59.066703+0000","last_active":"2023-10-14T10:37:01.787484+0000","last_peered":"2023-10-14T10:37:01.787484+0000","last_clean":"2023-10-14T10:37:01.787484+0000","last_became_active":"2023-10-14T10:36:59.066424+0000","last_became_peered":"2023-10-14T10:36:59.066424+0000","last_unstale":"2023-10-14T10:37:01.787484+0000","last_undegraded":"2023-10-14T10:37:01.787484+0000","last_fullsized":"2023-10-14T10:37:01.787484+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:54:15.011628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":176,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-14T10:37:00.060617+0000","last_change":"2023-10-14T10:35:59.770163+0000","last_active":"2023-10-14T10:37:00.060617+0000","last_peered":"2023-10-14T10:37:00.060617+0000","last_clean":"2023-10-14T10:37:00.060617+0000","last_became_active":"2023-10-14T10:35:59.769199+0000","last_became_peered":"2023-10-14T10:35:59.769199+0000","last_unstale":"2023-10-14T10:37:00.060617+0000","last_undegraded":"2023-10-14T10:37:00.060617+0000","last_fullsized":"2023-10-14T10:37:00.060617+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:35:58.675572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:35:58.675572+0000","last_clean_scrub_stamp":"2023-10-14T10:35:58.675572+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:03:25.281187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542162,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27972,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343868,"statfs":{"total":96636764160,"available":96608120832,"internally_reserved":0,"allocated":966656,"data_stored":732594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 10:36:59 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":2.4649999999999999,"5min":2.4649999999999999,"15min":2.4649999999999999},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.45500000000000002}]},{"osd":1,"last update":"Sat Oct 14 10:36:59 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":2.4289999999999998,"5min":2.4289999999999998,"15min":2.4289999999999998},"last":0.60199999999999998}]},{"osd":2,"last update":"Sat Oct 14 10:36:59 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.59099999999999997}]},{"osd":3,"last update":"Sat Oct 14 10:36:59 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.405,"5min":1.405,"15min":1.405},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.47699999999999998}]},{"osd":4,"last update":"Sat Oct 14 10:36:59 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.57799999999999996}]},{"osd":5,"last update":"Sat Oct 14 10:36:59 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.51900000000000002}]},{"osd":6,"last update":"Sat Oct 14 10:36:59 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.42499999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542163,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27388,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344452,"statfs":{"total":96636764160,"available":96608718848,"internally_reserved":0,"allocated":376832,"data_stored":142752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 10:36:57 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.58499999999999996}]},{"osd":1,"last update":"Sat Oct 14 10:36:57 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.389,"5min":1.389,"15min":1.389},"last":0.68899999999999995}]},{"osd":2,"last update":"Sat Oct 14 10:36:57 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.74299999999999999}]},{"osd":3,"last update":"Sat Oct 14 10:36:57 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.65100000000000002}]},{"osd":4,"last update":"Sat Oct 14 10:36:57 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.499}]},{"osd":5,"last update":"Sat Oct 14 10:36:57 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.52700000000000002}]},{"osd":7,"last update":"Sat Oct 14 10:36:57 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.435},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.41299999999999998}]}]},{"osd":1,"up_from":14,"seq":60129542162,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":376,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":385024,"data_stored":143770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.254,"15min":1.254},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":16.855,"5min":16.855,"15min":16.855},"last":0.28399999999999997},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":19.353000000000002,"5min":19.353000000000002,"15min":19.353000000000002},"last":0.36399999999999999}]},{"osd":2,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":3.0670000000000002,"5min":3.0670000000000002,"15min":3.0670000000000002},"last":0.26100000000000001},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.32500000000000001}]},{"osd":3,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":10.005000000000001,"5min":10.005000000000001,"15min":10.005000000000001},"last":0.30299999999999999}]},{"osd":4,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":2.9500000000000002,"5min":2.9500000000000002,"15min":2.9500000000000002},"last":0.495},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":5.7270000000000003,"5min":5.7270000000000003,"15min":5.7270000000000003},"last":0.65400000000000003}]},{"osd":5,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.129,"15min":1.129},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":9.8800000000000008,"5min":9.8800000000000008,"15min":9.8800000000000008},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":3.0350000000000001,"5min":3.0350000000000001,"15min":3.0350000000000001},"last":0.66700000000000004}]},{"osd":6,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":5.681,"5min":5.681,"15min":5.681},"last":0.54600000000000004},{"interface":"front","average":{"1min":1.173,"5min":1.173,"15min":1.173},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":9.9380000000000006,"5min":9.9380000000000006,"15min":9.9380000000000006},"last":0.68700000000000006}]},{"osd":7,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":9.9730000000000008,"5min":9.9730000000000008,"15min":9.9730000000000008},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":3.0009999999999999,"5min":3.0009999999999999,"15min":3.0009999999999999},"last":0.58899999999999997}]}]},{"osd":0,"up_from":14,"seq":60129542161,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27960,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343880,"statfs":{"total":96636764160,"available":96608133120,"internally_reserved":0,"allocated":962560,"data_stored":732504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.62,"5min":0.62,"15min":0.62},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.495,"5min":0.495,"15min":0.495},"last":0.48199999999999998}]},{"osd":2,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"last":0.378},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"last":0.40400000000000003}]},{"osd":3,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"last":0.42699999999999999}]},{"osd":4,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.66500000000000004}]},{"osd":5,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.55800000000000005}]},{"osd":6,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.57299999999999995}]},{"osd":7,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.69699999999999995}]}]},{"osd":2,"up_from":14,"seq":60129542162,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":372736,"data_stored":142206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":1.429,"5min":1.429,"15min":1.429},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.36199999999999999}]},{"osd":1,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":1.5269999999999999,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.40500000000000003}]},{"osd":3,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":4.0629999999999997,"5min":4.0629999999999997,"15min":4.0629999999999997},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.381,"5min":1.381,"15min":1.381},"last":0.42899999999999999}]},{"osd":4,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.58399999999999996}]},{"osd":5,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":3.8599999999999999,"5min":3.8599999999999999,"15min":3.8599999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.464,"5min":1.464,"15min":1.464},"last":0.68899999999999995}]},{"osd":6,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.753},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":3.9820000000000002,"5min":3.9820000000000002,"15min":3.9820000000000002},"last":0.63600000000000001}]},{"osd":7,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":3.9049999999999998,"5min":3.9049999999999998,"15min":3.9049999999999998},"last":0.61699999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542162,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":376,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":385024,"data_stored":143770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"last":0.34300000000000003}]},{"osd":1,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"last":0.373}]},{"osd":2,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"last":0.26600000000000001},{"interface":"front","average":{"1min":0.437,"5min":0.437,"15min":0.437},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.40300000000000002}]},{"osd":4,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":6.0350000000000001,"5min":6.0350000000000001,"15min":6.0350000000000001},"last":0.71499999999999997}]},{"osd":5,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":6.0540000000000003,"5min":6.0540000000000003,"15min":6.0540000000000003},"last":0.65700000000000003}]},{"osd":6,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.621}]},{"osd":7,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":6.0819999999999999,"5min":6.0819999999999999,"15min":6.0819999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.61199999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542161,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27388,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344452,"statfs":{"total":96636764160,"available":96608718848,"internally_reserved":0,"allocated":376832,"data_stored":142240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 10:36:56 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.47699999999999998}]},{"osd":1,"last update":"Sat Oct 14 10:36:56 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.39000000000000001}]},{"osd":2,"last update":"Sat Oct 14 10:36:56 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.42699999999999999}]},{"osd":3,"last update":"Sat Oct 14 10:36:56 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.48999999999999999}]},{"osd":5,"last update":"Sat Oct 14 10:36:56 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.60299999999999998}]},{"osd":6,"last update":"Sat Oct 14 10:36:56 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.61899999999999999}]},{"osd":7,"last update":"Sat Oct 14 10:36:56 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.66300000000000003}]}]},{"osd":5,"up_from":14,"seq":60129542162,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27388,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344452,"statfs":{"total":96636764160,"available":96608718848,"internally_reserved":0,"allocated":376832,"data_stored":142808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.64600000000000002}]},{"osd":1,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.72099999999999997}]},{"osd":2,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.70799999999999996}]},{"osd":3,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.66200000000000003}]},{"osd":4,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.372},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.41099999999999998}]},{"osd":6,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.439}]},{"osd":7,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.51900000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-14T10:37:09.277 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-14T10:37:09.436 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.434+0000 7fc19e157700 1 Processor -- start 2023-10-14T10:37:09.437 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.434+0000 7fc19e157700 1 -- start start 2023-10-14T10:37:09.440 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.435+0000 7fc19e157700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc19806e750 0x7fc1980ceec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:09.440 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.435+0000 7fc19e157700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fc19806eb30 0x7fc1980cf400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:09.440 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.435+0000 7fc19e157700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fc1980b54f0 0x7fc19815e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:09.441 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.435+0000 7fc19e157700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1980b5c70 con 0x7fc19806e750 2023-10-14T10:37:09.441 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.435+0000 7fc19e157700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1980b5990 con 0x7fc19806eb30 2023-10-14T10:37:09.441 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.435+0000 7fc19e157700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc19815b2b0 con 0x7fc1980b54f0 2023-10-14T10:37:09.442 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.435+0000 7fc1977fe700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc19806e750 0x7fc1980ceec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:09.442 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.435+0000 7fc1977fe700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc19806e750 0x7fc1980ceec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45992/0 (socket says 172.21.15.67:45992) 2023-10-14T10:37:09.442 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.435+0000 7fc1977fe700 1 -- 172.21.15.67:0/1541555993 learned_addr learned my addr 172.21.15.67:0/1541555993 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:37:09.443 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.436+0000 7fc196ffd700 1 --2- 172.21.15.67:0/1541555993 >> v2:172.21.15.72:3300/0 conn(0x7fc19806eb30 0x7fc1980cf400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:09.443 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.436+0000 7fc197fff700 1 --2- 172.21.15.67:0/1541555993 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fc1980b54f0 0x7fc19815e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:09.443 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.436+0000 7fc1977fe700 1 -- 172.21.15.67:0/1541555993 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fc1980b54f0 msgr2=0x7fc19815e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:37:09.443 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.436+0000 7fc1977fe700 1 --2- 172.21.15.67:0/1541555993 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fc1980b54f0 0x7fc19815e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:09.444 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.436+0000 7fc1977fe700 1 -- 172.21.15.67:0/1541555993 >> v2:172.21.15.72:3300/0 conn(0x7fc19806eb30 msgr2=0x7fc1980cf400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:37:09.444 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.436+0000 7fc1977fe700 1 --2- 172.21.15.67:0/1541555993 >> v2:172.21.15.72:3300/0 conn(0x7fc19806eb30 0x7fc1980cf400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:09.444 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.436+0000 7fc1977fe700 1 -- 172.21.15.67:0/1541555993 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc19815b590 con 0x7fc19806e750 2023-10-14T10:37:09.444 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.436+0000 7fc197fff700 1 --2- 172.21.15.67:0/1541555993 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fc1980b54f0 0x7fc19815e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-14T10:37:09.445 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.436+0000 7fc196ffd700 1 --2- 172.21.15.67:0/1541555993 >> v2:172.21.15.72:3300/0 conn(0x7fc19806eb30 0x7fc1980cf400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-14T10:37:09.445 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.436+0000 7fc1977fe700 1 --2- 172.21.15.67:0/1541555993 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc19806e750 0x7fc1980ceec0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fc18800bb90 tx=0x7fc18800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d4a8225f828c2f server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:09.445 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.436+0000 7fc1967fc700 1 -- 172.21.15.67:0/1541555993 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc18800dd10 con 0x7fc19806e750 2023-10-14T10:37:09.446 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.436+0000 7fc1967fc700 1 -- 172.21.15.67:0/1541555993 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc18800de70 con 0x7fc19806e750 2023-10-14T10:37:09.446 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.436+0000 7fc1967fc700 1 -- 172.21.15.67:0/1541555993 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc188010670 con 0x7fc19806e750 2023-10-14T10:37:09.446 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.437+0000 7fc19e157700 1 -- 172.21.15.67:0/1541555993 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc19806e750 msgr2=0x7fc1980ceec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:09.446 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.437+0000 7fc19e157700 1 --2- 172.21.15.67:0/1541555993 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc19806e750 0x7fc1980ceec0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fc18800bb90 tx=0x7fc18800bea0 comp rx=0 tx=0).stop 2023-10-14T10:37:09.447 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.437+0000 7fc19e157700 1 -- 172.21.15.67:0/1541555993 shutdown_connections 2023-10-14T10:37:09.447 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.437+0000 7fc19e157700 1 --2- 172.21.15.67:0/1541555993 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fc1980b54f0 0x7fc19815e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:09.447 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.437+0000 7fc19e157700 1 --2- 172.21.15.67:0/1541555993 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc19806e750 0x7fc1980ceec0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:09.448 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.437+0000 7fc19e157700 1 --2- 172.21.15.67:0/1541555993 >> v2:172.21.15.72:3300/0 conn(0x7fc19806eb30 0x7fc1980cf400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:09.448 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.437+0000 7fc19e157700 1 -- 172.21.15.67:0/1541555993 >> 172.21.15.67:0/1541555993 conn(0x7fc1980cc6f0 msgr2=0x7fc1980ccb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:37:09.448 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.437+0000 7fc19e157700 1 -- 172.21.15.67:0/1541555993 shutdown_connections 2023-10-14T10:37:09.448 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.437+0000 7fc19e157700 1 -- 172.21.15.67:0/1541555993 wait complete. 2023-10-14T10:37:09.449 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.438+0000 7fc19e157700 1 Processor -- start 2023-10-14T10:37:09.449 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.438+0000 7fc19e157700 1 -- start start 2023-10-14T10:37:09.450 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-14T10:37:09.452 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.439+0000 7fc19e157700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fc19806e750 0x7fc1980b0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:09.453 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.439+0000 7fc19e157700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc19806eb30 0x7fc1980c6d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:09.453 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.439+0000 7fc19e157700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1980b54f0 0x7fc1980c5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:09.453 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.439+0000 7fc19e157700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1980c9350 con 0x7fc1980b54f0 2023-10-14T10:37:09.453 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.439+0000 7fc19e157700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1980c7280 con 0x7fc19806eb30 2023-10-14T10:37:09.454 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.439+0000 7fc19e157700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1980c6500 con 0x7fc19806e750 2023-10-14T10:37:09.454 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.439+0000 7fc197fff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1980b54f0 0x7fc1980c5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:09.454 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.439+0000 7fc197fff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1980b54f0 0x7fc1980c5700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46002/0 (socket says 172.21.15.67:46002) 2023-10-14T10:37:09.455 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.439+0000 7fc197fff700 1 -- 172.21.15.67:0/2564041534 learned_addr learned my addr 172.21.15.67:0/2564041534 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:37:09.455 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.439+0000 7fc196ffd700 1 --2- 172.21.15.67:0/2564041534 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc19806eb30 0x7fc1980c6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:09.455 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.439+0000 7fc1977fe700 1 --2- 172.21.15.67:0/2564041534 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fc19806e750 0x7fc1980b0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:09.455 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.439+0000 7fc197fff700 1 -- 172.21.15.67:0/2564041534 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fc19806e750 msgr2=0x7fc1980b0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:09.456 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.439+0000 7fc197fff700 1 --2- 172.21.15.67:0/2564041534 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fc19806e750 0x7fc1980b0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:09.456 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.439+0000 7fc197fff700 1 -- 172.21.15.67:0/2564041534 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc19806eb30 msgr2=0x7fc1980c6d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:09.456 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.439+0000 7fc197fff700 1 --2- 172.21.15.67:0/2564041534 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc19806eb30 0x7fc1980c6d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:09.457 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.439+0000 7fc197fff700 1 -- 172.21.15.67:0/2564041534 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc18800b7f0 con 0x7fc1980b54f0 2023-10-14T10:37:09.457 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.440+0000 7fc197fff700 1 --2- 172.21.15.67:0/2564041534 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1980b54f0 0x7fc1980c5700 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fc19000cd50 tx=0x7fc19000a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:09.457 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.440+0000 7fc18ffff700 1 -- 172.21.15.67:0/2564041534 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc19000cea0 con 0x7fc1980b54f0 2023-10-14T10:37:09.457 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.440+0000 7fc18ffff700 1 -- 172.21.15.67:0/2564041534 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc19000f480 con 0x7fc1980b54f0 2023-10-14T10:37:09.458 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.440+0000 7fc19e157700 1 -- 172.21.15.67:0/2564041534 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1980c67e0 con 0x7fc1980b54f0 2023-10-14T10:37:09.458 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.440+0000 7fc18ffff700 1 -- 172.21.15.67:0/2564041534 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc19000f830 con 0x7fc1980b54f0 2023-10-14T10:37:09.458 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.440+0000 7fc19e157700 1 -- 172.21.15.67:0/2564041534 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1980b4cf0 con 0x7fc1980b54f0 2023-10-14T10:37:09.459 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.441+0000 7fc19e157700 1 -- 172.21.15.67:0/2564041534 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1980c97e0 con 0x7fc1980b54f0 2023-10-14T10:37:09.459 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.441+0000 7fc18ffff700 1 -- 172.21.15.67:0/2564041534 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fc19000b040 con 0x7fc1980b54f0 2023-10-14T10:37:09.459 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.441+0000 7fc18ffff700 1 --2- 172.21.15.67:0/2564041534 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fc17806af30 0x7fc17806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:09.459 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.442+0000 7fc18ffff700 1 -- 172.21.15.67:0/2564041534 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5729+0+0 (secure 0 0 0) 0x7fc190089bb0 con 0x7fc1980b54f0 2023-10-14T10:37:09.460 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.442+0000 7fc1977fe700 1 --2- 172.21.15.67:0/2564041534 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fc17806af30 0x7fc17806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:09.460 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.442+0000 7fc1977fe700 1 --2- 172.21.15.67:0/2564041534 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fc17806af30 0x7fc17806d3f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc188000c00 tx=0x7fc188013000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:09.460 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.446+0000 7fc18ffff700 1 -- 172.21.15.67:0/2564041534 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc1980c97e0 con 0x7fc1980b54f0 2023-10-14T10:37:09.462 DEBUG:teuthology.orchestra.run.smithi072:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-14T10:37:09.651 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.649+0000 7fc19e157700 1 -- 172.21.15.67:0/2564041534 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc1980c97e0 con 0x7fc1980b54f0 2023-10-14T10:37:09.652 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.650+0000 7fc18ffff700 1 -- 172.21.15.67:0/2564041534 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16541 (secure 0 0 0) 0x7fc1980c97e0 con 0x7fc1980b54f0 2023-10-14T10:37:09.653 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-14T10:37:09.653 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":33,"fsid":"37f9ab46-3082-4b71-8b24-08d39897b008","created":"2023-10-14T10:35:46.067554+0000","modified":"2023-10-14T10:37:01.056657+0000","last_up_change":"2023-10-14T10:35:56.668142+0000","last_in_change":"2023-10-14T10:35:48.551490+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-14T10:35:57.809566+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"testcephfs1_metadata","create_time":"2023-10-14T10:36:57.591719+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"testcephfs1"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"testcephfs1_data","create_time":"2023-10-14T10:36:58.436708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"testcephfs1"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2b59a07e-11ef-440c-b766-9f64ac885838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6801","nonce":1287255499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6803","nonce":1287255499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6807","nonce":1287255499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1287255499},{"type":"v1","addr":"172.21.15.67:6805","nonce":1287255499}]},"public_addr":"172.21.15.67:6801/1287255499","cluster_addr":"172.21.15.67:6803/1287255499","heartbeat_back_addr":"172.21.15.67:6807/1287255499","heartbeat_front_addr":"172.21.15.67:6805/1287255499","state":["exists","up"]},{"osd":1,"uuid":"6f813bd6-fa0b-487d-a204-431eb1276a60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6818","nonce":2435538787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6822","nonce":2435538787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6830","nonce":2435538787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":2435538787},{"type":"v1","addr":"172.21.15.67:6826","nonce":2435538787}]},"public_addr":"172.21.15.67:6818/2435538787","cluster_addr":"172.21.15.67:6822/2435538787","heartbeat_back_addr":"172.21.15.67:6830/2435538787","heartbeat_front_addr":"172.21.15.67:6826/2435538787","state":["exists","up"]},{"osd":2,"uuid":"6af42c6a-1b59-4669-9c04-b6876788446d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6809","nonce":2827491791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6811","nonce":2827491791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6815","nonce":2827491791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":2827491791},{"type":"v1","addr":"172.21.15.67:6813","nonce":2827491791}]},"public_addr":"172.21.15.67:6809/2827491791","cluster_addr":"172.21.15.67:6811/2827491791","heartbeat_back_addr":"172.21.15.67:6815/2827491791","heartbeat_front_addr":"172.21.15.67:6813/2827491791","state":["exists","up"]},{"osd":3,"uuid":"1d889d76-a646-499d-bfff-ed5bae5434ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6819","nonce":1848001176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6821","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6823","nonce":1848001176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6831","nonce":1848001176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6825","nonce":1848001176},{"type":"v1","addr":"172.21.15.67:6827","nonce":1848001176}]},"public_addr":"172.21.15.67:6819/1848001176","cluster_addr":"172.21.15.67:6823/1848001176","heartbeat_back_addr":"172.21.15.67:6831/1848001176","heartbeat_front_addr":"172.21.15.67:6827/1848001176","state":["exists","up"]},{"osd":4,"uuid":"c3271dff-b437-4fdd-b13b-6e49a35473af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6804","nonce":1891230785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6808","nonce":1891230785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6815","nonce":1891230785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6811","nonce":1891230785},{"type":"v1","addr":"172.21.15.72:6813","nonce":1891230785}]},"public_addr":"172.21.15.72:6804/1891230785","cluster_addr":"172.21.15.72:6808/1891230785","heartbeat_back_addr":"172.21.15.72:6815/1891230785","heartbeat_front_addr":"172.21.15.72:6813/1891230785","state":["exists","up"]},{"osd":5,"uuid":"0337bb18-4b68-4eff-b528-a667275e6926","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6801","nonce":569686991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6803","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6805","nonce":569686991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6812","nonce":569686991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6807","nonce":569686991},{"type":"v1","addr":"172.21.15.72:6809","nonce":569686991}]},"public_addr":"172.21.15.72:6801/569686991","cluster_addr":"172.21.15.72:6805/569686991","heartbeat_back_addr":"172.21.15.72:6812/569686991","heartbeat_front_addr":"172.21.15.72:6809/569686991","state":["exists","up"]},{"osd":6,"uuid":"8e92d9cf-0c8c-46eb-89a7-db2023ef1509","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6817","nonce":3767685934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6819","nonce":3767685934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6823","nonce":3767685934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":3767685934},{"type":"v1","addr":"172.21.15.72:6821","nonce":3767685934}]},"public_addr":"172.21.15.72:6817/3767685934","cluster_addr":"172.21.15.72:6819/3767685934","heartbeat_back_addr":"172.21.15.72:6823/3767685934","heartbeat_front_addr":"172.21.15.72:6821/3767685934","state":["exists","up"]},{"osd":7,"uuid":"3fd6aa7d-50a9-45c9-9561-5902a044a119","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6825","nonce":3567465926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6827","nonce":3567465926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6831","nonce":3567465926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":3567465926},{"type":"v1","addr":"172.21.15.72:6829","nonce":3567465926}]},"public_addr":"172.21.15.72:6825/3567465926","cluster_addr":"172.21.15.72:6827/3567465926","heartbeat_back_addr":"172.21.15.72:6831/3567465926","heartbeat_front_addr":"172.21.15.72:6829/3567465926","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.511466+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.795107+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.858674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.728895+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.794842+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.750064+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.732122+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-14T10:35:53.765185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-14T10:37:09.655 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.652+0000 7fc19e157700 1 -- 172.21.15.67:0/2564041534 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fc17806af30 msgr2=0x7fc17806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:09.655 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.652+0000 7fc19e157700 1 --2- 172.21.15.67:0/2564041534 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fc17806af30 0x7fc17806d3f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc188000c00 tx=0x7fc188013000 comp rx=0 tx=0).stop 2023-10-14T10:37:09.655 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.652+0000 7fc19e157700 1 -- 172.21.15.67:0/2564041534 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1980b54f0 msgr2=0x7fc1980c5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:09.656 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.652+0000 7fc19e157700 1 --2- 172.21.15.67:0/2564041534 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1980b54f0 0x7fc1980c5700 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fc19000cd50 tx=0x7fc19000a560 comp rx=0 tx=0).stop 2023-10-14T10:37:09.656 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.652+0000 7fc19e157700 1 -- 172.21.15.67:0/2564041534 shutdown_connections 2023-10-14T10:37:09.656 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.652+0000 7fc19e157700 1 --2- 172.21.15.67:0/2564041534 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fc17806af30 0x7fc17806d3f0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:09.657 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.652+0000 7fc19e157700 1 --2- 172.21.15.67:0/2564041534 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1980b54f0 0x7fc1980c5700 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:09.657 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.652+0000 7fc19e157700 1 --2- 172.21.15.67:0/2564041534 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fc19806e750 0x7fc1980b0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:09.657 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.652+0000 7fc19e157700 1 --2- 172.21.15.67:0/2564041534 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc19806eb30 0x7fc1980c6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:09.657 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.652+0000 7fc19e157700 1 -- 172.21.15.67:0/2564041534 >> 172.21.15.67:0/2564041534 conn(0x7fc1980cc6f0 msgr2=0x7fc1980719e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:37:09.658 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.652+0000 7fc19e157700 1 -- 172.21.15.67:0/2564041534 shutdown_connections 2023-10-14T10:37:09.658 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:09.652+0000 7fc19e157700 1 -- 172.21.15.67:0/2564041534 wait complete. 2023-10-14T10:37:10.667 INFO:tasks.ceph:Scrubbing osd.0 2023-10-14T10:37:10.668 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-10-14T10:37:10.811 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.809+0000 7f80847eb700 1 Processor -- start 2023-10-14T10:37:10.813 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.809+0000 7f80847eb700 1 -- start start 2023-10-14T10:37:10.817 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.810+0000 7f80847eb700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f807c06da30 0x7f807c06dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:10.817 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.810+0000 7f80847eb700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f807c0b2e40 con 0x7f807c151680 2023-10-14T10:37:10.817 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.810+0000 7f80847eb700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f807c15e7c0 con 0x7f807c0727e0 2023-10-14T10:37:10.818 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.810+0000 7f80847eb700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f807c06e420 con 0x7f807c06da30 2023-10-14T10:37:10.818 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.810+0000 7f8082587700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f807c151680 0x7f807c15ec50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:53774/0 (socket says 172.21.15.67:53774) 2023-10-14T10:37:10.818 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.810+0000 7f8082587700 1 -- 172.21.15.67:0/2060194972 learned_addr learned my addr 172.21.15.67:0/2060194972 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:37:10.819 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.811+0000 7f8082d88700 1 --2- 172.21.15.67:0/2060194972 >> v2:172.21.15.72:3300/0 conn(0x7f807c06da30 0x7f807c06dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:10.819 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.811+0000 7f8081585700 1 -- 172.21.15.67:0/2060194972 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2613879120 0 0) 0x7f806c001110 con 0x7f807c151680 2023-10-14T10:37:10.819 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.811+0000 7f8081585700 1 -- 172.21.15.67:0/2060194972 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80680035b0 con 0x7f807c151680 2023-10-14T10:37:10.820 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.811+0000 7f8081585700 1 -- 172.21.15.67:0/2060194972 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 128212795 0 0) 0x7f80680035b0 con 0x7f807c151680 2023-10-14T10:37:10.820 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.811+0000 7f8081585700 1 -- 172.21.15.67:0/2060194972 >> v2:172.21.15.72:3300/0 conn(0x7f807c06da30 msgr2=0x7f807c06dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:37:10.820 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.811+0000 7f8081585700 1 --2- 172.21.15.67:0/2060194972 >> v2:172.21.15.72:3300/0 conn(0x7f807c06da30 0x7f807c06dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:10.820 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.811+0000 7f8081585700 1 -- 172.21.15.67:0/2060194972 >> v1:172.21.15.72:6789/0 conn(0x7f807c0727e0 legacy=0x7f807c0b1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:10.821 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.811+0000 7f8081585700 1 -- 172.21.15.67:0/2060194972 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f807c0b2620 con 0x7f807c151680 2023-10-14T10:37:10.821 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.811+0000 7f8081585700 1 -- 172.21.15.67:0/2060194972 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3180043755 0 0) 0x7f806c001800 con 0x7f807c151680 2023-10-14T10:37:10.821 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.811+0000 7f8081585700 1 -- 172.21.15.67:0/2060194972 >> v1:172.21.15.67:6789/0 conn(0x7f807c151680 legacy=0x7f807c15ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:10.822 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.811+0000 7f8081585700 1 --2- 172.21.15.67:0/2060194972 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f8068003a00 0x7f8068003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:10.822 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.811+0000 7f8081585700 1 --2- 172.21.15.67:0/2060194972 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8068004850 0x7f8068004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:10.822 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.811+0000 7f8081585700 1 --2- 172.21.15.67:0/2060194972 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8068005210 0x7f806800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:10.823 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.811+0000 7f8081585700 1 -- 172.21.15.67:0/2060194972 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f807c06e420 con 0x7f8068004850 2023-10-14T10:37:10.823 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.811+0000 7f8081585700 1 -- 172.21.15.67:0/2060194972 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f806800c0b0 con 0x7f8068005210 2023-10-14T10:37:10.823 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.811+0000 7f8081585700 1 -- 172.21.15.67:0/2060194972 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f806800c270 con 0x7f8068003a00 2023-10-14T10:37:10.823 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.811+0000 7f8081d86700 1 --2- 172.21.15.67:0/2060194972 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8068004850 0x7f8068004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:10.824 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.811+0000 7f8082587700 1 --2- 172.21.15.67:0/2060194972 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f8068003a00 0x7f8068003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:10.824 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.811+0000 7f8082d88700 1 --2- 172.21.15.67:0/2060194972 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8068005210 0x7f806800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:10.824 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.812+0000 7f8081d86700 1 -- 172.21.15.67:0/2060194972 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f8068003a00 msgr2=0x7f8068003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:37:10.825 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.812+0000 7f8081d86700 1 --2- 172.21.15.67:0/2060194972 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f8068003a00 0x7f8068003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:10.825 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.812+0000 7f8081d86700 1 -- 172.21.15.67:0/2060194972 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8068005210 msgr2=0x7f806800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:37:10.825 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.812+0000 7f8081d86700 1 --2- 172.21.15.67:0/2060194972 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8068005210 0x7f806800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:10.825 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.812+0000 7f8082d88700 1 -- 172.21.15.67:0/2060194972 reap_dead start 2023-10-14T10:37:10.826 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.812+0000 7f8081d86700 1 -- 172.21.15.67:0/2060194972 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80680035b0 con 0x7f8068004850 2023-10-14T10:37:10.826 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.812+0000 7f8081d86700 1 --2- 172.21.15.67:0/2060194972 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8068004850 0x7f8068004cd0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f80780017c0 tx=0x7f8068005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c7043ea2b370d4c server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:10.826 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.812+0000 7f8081585700 1 -- 172.21.15.67:0/2060194972 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80780052d0 con 0x7f8068004850 2023-10-14T10:37:10.827 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.812+0000 7f8081585700 1 -- 172.21.15.67:0/2060194972 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8078005450 con 0x7f8068004850 2023-10-14T10:37:10.827 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.812+0000 7f8081585700 1 -- 172.21.15.67:0/2060194972 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80780038c0 con 0x7f8068004850 2023-10-14T10:37:10.827 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.813+0000 7f80847eb700 1 -- 172.21.15.67:0/2060194972 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8068004850 msgr2=0x7f8068004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:10.827 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.813+0000 7f80847eb700 1 --2- 172.21.15.67:0/2060194972 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8068004850 0x7f8068004cd0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f80780017c0 tx=0x7f8068005880 comp rx=0 tx=0).stop 2023-10-14T10:37:10.828 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.813+0000 7f80847eb700 1 -- 172.21.15.67:0/2060194972 shutdown_connections 2023-10-14T10:37:10.828 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.813+0000 7f80847eb700 1 --2- 172.21.15.67:0/2060194972 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8068004850 0x7f8068004cd0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:10.828 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.813+0000 7f80847eb700 1 -- 172.21.15.67:0/2060194972 >> 172.21.15.67:0/2060194972 conn(0x7f807c06bd70 msgr2=0x7f807c06c180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:37:10.828 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.813+0000 7f80847eb700 1 -- 172.21.15.67:0/2060194972 shutdown_connections 2023-10-14T10:37:10.829 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.813+0000 7f80847eb700 1 -- 172.21.15.67:0/2060194972 wait complete. 2023-10-14T10:37:10.829 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.815+0000 7f80847eb700 1 Processor -- start 2023-10-14T10:37:10.829 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.815+0000 7f80847eb700 1 -- start start 2023-10-14T10:37:10.832 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.816+0000 7f80847eb700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8068004850 0x7f807c150a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:10.833 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.816+0000 7f80847eb700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f807c0b5a80 0x7f807c1f68d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:10.833 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.816+0000 7f80847eb700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f807c0ad5c0 0x7f807c0ada40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:10.833 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.816+0000 7f80847eb700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f807c150f70 con 0x7f8068004850 2023-10-14T10:37:10.833 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.816+0000 7f80847eb700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f807c0ad130 con 0x7f807c0b5a80 2023-10-14T10:37:10.834 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.816+0000 7f80847eb700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f807c1f71d0 con 0x7f807c0ad5c0 2023-10-14T10:37:10.834 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.816+0000 7f8082587700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8068004850 0x7f807c150a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:10.834 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.816+0000 7f8082587700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8068004850 0x7f807c150a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46018/0 (socket says 172.21.15.67:46018) 2023-10-14T10:37:10.835 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.816+0000 7f8082587700 1 -- 172.21.15.67:0/673043291 learned_addr learned my addr 172.21.15.67:0/673043291 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:37:10.835 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.816+0000 7f8081d86700 1 --2- 172.21.15.67:0/673043291 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f807c0b5a80 0x7f807c1f68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:10.835 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.816+0000 7f8082d88700 1 --2- 172.21.15.67:0/673043291 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f807c0ad5c0 0x7f807c0ada40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:10.835 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.816+0000 7f8082587700 1 -- 172.21.15.67:0/673043291 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f807c0ad5c0 msgr2=0x7f807c0ada40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:10.836 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.816+0000 7f8082587700 1 --2- 172.21.15.67:0/673043291 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f807c0ad5c0 0x7f807c0ada40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:10.836 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.816+0000 7f8082587700 1 -- 172.21.15.67:0/673043291 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f807c0b5a80 msgr2=0x7f807c1f68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:10.836 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.816+0000 7f8082587700 1 --2- 172.21.15.67:0/673043291 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f807c0b5a80 0x7f807c1f68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:10.837 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.816+0000 7f8082587700 1 -- 172.21.15.67:0/673043291 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f807c1f74b0 con 0x7f8068004850 2023-10-14T10:37:10.837 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.816+0000 7f8082587700 1 --2- 172.21.15.67:0/673043291 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8068004850 0x7f807c150a30 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f806c001930 tx=0x7f806c007570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:10.837 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.816+0000 7f8072ffd700 1 -- 172.21.15.67:0/673043291 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f806c011580 con 0x7f8068004850 2023-10-14T10:37:10.837 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.816+0000 7f8072ffd700 1 -- 172.21.15.67:0/673043291 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f806c011720 con 0x7f8068004850 2023-10-14T10:37:10.838 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.816+0000 7f80847eb700 1 -- 172.21.15.67:0/673043291 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f807c1f7790 con 0x7f8068004850 2023-10-14T10:37:10.838 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.817+0000 7f8072ffd700 1 -- 172.21.15.67:0/673043291 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f806c011ad0 con 0x7f8068004850 2023-10-14T10:37:10.838 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.817+0000 7f80847eb700 1 -- 172.21.15.67:0/673043291 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f807c1f6e10 con 0x7f8068004850 2023-10-14T10:37:10.839 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.817+0000 7f80847eb700 1 -- 172.21.15.67:0/673043291 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f807c0acc50 con 0x7f8068004850 2023-10-14T10:37:10.839 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.818+0000 7f8072ffd700 1 -- 172.21.15.67:0/673043291 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f806c011c30 con 0x7f8068004850 2023-10-14T10:37:10.839 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.818+0000 7f8072ffd700 1 --2- 172.21.15.67:0/673043291 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f806006af00 0x7f806006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:10.839 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.818+0000 7f8072ffd700 1 -- 172.21.15.67:0/673043291 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5729+0+0 (secure 0 0 0) 0x7f806c0892f0 con 0x7f8068004850 2023-10-14T10:37:10.840 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.819+0000 7f8072ffd700 1 --2- 172.21.15.67:0/673043291 >> [v2:172.21.15.67:6800/1287255499,v1:172.21.15.67:6801/1287255499] conn(0x7f8060073100 0x7f8060075540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:10.840 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.819+0000 7f8072ffd700 1 -- 172.21.15.67:0/673043291 --> [v2:172.21.15.67:6800/1287255499,v1:172.21.15.67:6801/1287255499] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8060075c10 con 0x7f8060073100 2023-10-14T10:37:10.840 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.819+0000 7f8072ffd700 1 -- 172.21.15.67:0/673043291 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f806c08b9b0 con 0x7f8068004850 2023-10-14T10:37:10.840 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.819+0000 7f8081d86700 1 --2- 172.21.15.67:0/673043291 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f806006af00 0x7f806006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:10.841 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.819+0000 7f8082d88700 1 --2- 172.21.15.67:0/673043291 >> [v2:172.21.15.67:6800/1287255499,v1:172.21.15.67:6801/1287255499] conn(0x7f8060073100 0x7f8060075540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:10.841 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.819+0000 7f8082d88700 1 --2- 172.21.15.67:0/673043291 >> [v2:172.21.15.67:6800/1287255499,v1:172.21.15.67:6801/1287255499] conn(0x7f8060073100 0x7f8060075540 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:10.841 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.819+0000 7f8081d86700 1 --2- 172.21.15.67:0/673043291 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f806006af00 0x7f806006d3c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8078000c00 tx=0x7f8078009030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:10.841 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.820+0000 7f8072ffd700 1 -- 172.21.15.67:0/673043291 <== osd.0 v2:172.21.15.67:6800/1287255499 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7f8060075c10 con 0x7f8060073100 2023-10-14T10:37:10.842 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.835+0000 7f80847eb700 1 -- 172.21.15.67:0/673043291 --> [v2:172.21.15.67:6800/1287255499,v1:172.21.15.67:6801/1287255499] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f807c0b2b60 con 0x7f8060073100 2023-10-14T10:37:10.842 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.835+0000 7f8072ffd700 1 -- 172.21.15.67:0/673043291 <== osd.0 v2:172.21.15.67:6800/1287255499 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f807400f040 con 0x7f8060073100 2023-10-14T10:37:10.843 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2023-10-14T10:37:10.843 INFO:teuthology.orchestra.run.smithi067.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-14T10:37:10.843 INFO:teuthology.orchestra.run.smithi067.stdout:} 2023-10-14T10:37:10.845 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.835+0000 7f80847eb700 1 -- 172.21.15.67:0/673043291 >> [v2:172.21.15.67:6800/1287255499,v1:172.21.15.67:6801/1287255499] conn(0x7f8060073100 msgr2=0x7f8060075540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:10.845 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.835+0000 7f80847eb700 1 --2- 172.21.15.67:0/673043291 >> [v2:172.21.15.67:6800/1287255499,v1:172.21.15.67:6801/1287255499] conn(0x7f8060073100 0x7f8060075540 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:10.845 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.835+0000 7f80847eb700 1 -- 172.21.15.67:0/673043291 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f806006af00 msgr2=0x7f806006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:10.845 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.835+0000 7f80847eb700 1 --2- 172.21.15.67:0/673043291 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f806006af00 0x7f806006d3c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8078000c00 tx=0x7f8078009030 comp rx=0 tx=0).stop 2023-10-14T10:37:10.846 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.835+0000 7f80847eb700 1 -- 172.21.15.67:0/673043291 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8068004850 msgr2=0x7f807c150a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:10.846 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.835+0000 7f80847eb700 1 --2- 172.21.15.67:0/673043291 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8068004850 0x7f807c150a30 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f806c001930 tx=0x7f806c007570 comp rx=0 tx=0).stop 2023-10-14T10:37:10.846 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.835+0000 7f8082d88700 1 -- 172.21.15.67:0/673043291 reap_dead start 2023-10-14T10:37:10.847 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.835+0000 7f80847eb700 1 -- 172.21.15.67:0/673043291 shutdown_connections 2023-10-14T10:37:10.847 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.835+0000 7f80847eb700 1 -- 172.21.15.67:0/673043291 >> 172.21.15.67:0/673043291 conn(0x7f807c06bd70 msgr2=0x7f807c0cba30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:37:10.847 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.835+0000 7f80847eb700 1 -- 172.21.15.67:0/673043291 shutdown_connections 2023-10-14T10:37:10.847 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.835+0000 7f80847eb700 1 -- 172.21.15.67:0/673043291 wait complete. 2023-10-14T10:37:10.849 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-10-14T10:37:10.976 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.973+0000 7fd8bbf6c700 1 Processor -- start 2023-10-14T10:37:10.976 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.974+0000 7fd8bbf6c700 1 -- start start 2023-10-14T10:37:10.979 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.974+0000 7fd8bbf6c700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fd8b40cb560 0x7fd8b40cdd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:10.979 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.974+0000 7fd8bbf6c700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fd8b40ce290 0x7fd8b406a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:10.979 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.974+0000 7fd8bbf6c700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8b40ca570 con 0x7fd8b406d390 2023-10-14T10:37:10.980 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.974+0000 7fd8bbf6c700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8b40cb9c0 con 0x7fd8b40ce290 2023-10-14T10:37:10.980 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.974+0000 7fd8bbf6c700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b40cb0d0 con 0x7fd8b40cb560 2023-10-14T10:37:10.980 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.975+0000 7fd8ba509700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fd8b40ce290 0x7fd8b406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:10.980 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.975+0000 7fd8b9507700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fd8b40cb560 0x7fd8b40cdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:10.981 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.975+0000 7fd8ba509700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fd8b40ce290 0x7fd8b406a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.67:40822/0 (socket says 172.21.15.67:40822) 2023-10-14T10:37:10.981 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.975+0000 7fd8b9d08700 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7fd8b406d390 0x7fd8b406d770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.67:33804/0 (socket says 172.21.15.67:33804) 2023-10-14T10:37:10.981 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.975+0000 7fd8ba509700 1 -- 172.21.15.67:0/969265630 learned_addr learned my addr 172.21.15.67:0/969265630 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:37:10.982 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.975+0000 7fd8b9507700 1 -- 172.21.15.67:0/969265630 >> v2:172.21.15.72:3300/0 conn(0x7fd8b40ce290 msgr2=0x7fd8b406a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:37:10.982 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.975+0000 7fd8b9507700 1 --2- 172.21.15.67:0/969265630 >> v2:172.21.15.72:3300/0 conn(0x7fd8b40ce290 0x7fd8b406a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:10.982 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.975+0000 7fd8b9507700 1 -- 172.21.15.67:0/969265630 >> v1:172.21.15.72:6789/0 conn(0x7fd8b406d390 legacy=0x7fd8b406d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:10.982 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.975+0000 7fd8b9507700 1 -- 172.21.15.67:0/969265630 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8b415f1d0 con 0x7fd8b40cb560 2023-10-14T10:37:10.983 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.975+0000 7fd8b8d06700 1 -- 172.21.15.67:0/969265630 <== mon.1 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1347480454 0 0) 0x7fd8b40ca570 con 0x7fd8b406d390 2023-10-14T10:37:10.983 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.975+0000 7fd8ba509700 1 --2- 172.21.15.67:0/969265630 >> v2:172.21.15.72:3300/0 conn(0x7fd8b40ce290 0x7fd8b406a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-14T10:37:10.983 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.975+0000 7fd8b9507700 1 --2- 172.21.15.67:0/969265630 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fd8b40cb560 0x7fd8b40cdd50 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd8a8009840 tx=0x7fd8a8009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6403e878e953865 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:10.983 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.976+0000 7fd8b8d06700 1 -- 172.21.15.67:0/969265630 <== mon.2 v2:172.21.15.72:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a8011830 con 0x7fd8b40cb560 2023-10-14T10:37:10.984 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.976+0000 7fd8b8d06700 1 -- 172.21.15.67:0/969265630 <== mon.2 v2:172.21.15.72:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8a8011990 con 0x7fd8b40cb560 2023-10-14T10:37:10.984 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.976+0000 7fd8b8d06700 1 -- 172.21.15.67:0/969265630 <== mon.2 v2:172.21.15.72:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a8011d20 con 0x7fd8b40cb560 2023-10-14T10:37:10.984 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.976+0000 7fd8bbf6c700 1 -- 172.21.15.67:0/969265630 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fd8b40cb560 msgr2=0x7fd8b40cdd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:10.985 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.976+0000 7fd8bbf6c700 1 --2- 172.21.15.67:0/969265630 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fd8b40cb560 0x7fd8b40cdd50 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd8a8009840 tx=0x7fd8a8009b50 comp rx=0 tx=0).stop 2023-10-14T10:37:10.985 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.976+0000 7fd8bbf6c700 1 -- 172.21.15.67:0/969265630 shutdown_connections 2023-10-14T10:37:10.985 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.976+0000 7fd8bbf6c700 1 --2- 172.21.15.67:0/969265630 >> v2:172.21.15.72:3300/0 conn(0x7fd8b40ce290 0x7fd8b406a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:10.985 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.976+0000 7fd8bbf6c700 1 --2- 172.21.15.67:0/969265630 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fd8b40cb560 0x7fd8b40cdd50 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:10.986 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.976+0000 7fd8bbf6c700 1 -- 172.21.15.67:0/969265630 >> 172.21.15.67:0/969265630 conn(0x7fd8b40cfc40 msgr2=0x7fd8b40d0020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:37:10.986 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.976+0000 7fd8bbf6c700 1 -- 172.21.15.67:0/969265630 shutdown_connections 2023-10-14T10:37:10.986 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.976+0000 7fd8bbf6c700 1 -- 172.21.15.67:0/969265630 wait complete. 2023-10-14T10:37:10.989 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.977+0000 7fd8bbf6c700 1 Processor -- start 2023-10-14T10:37:10.989 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.977+0000 7fd8bbf6c700 1 -- start start 2023-10-14T10:37:10.990 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.978+0000 7fd8bbf6c700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fd8b406d390 0x7fd8b40c9520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:10.990 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.978+0000 7fd8bbf6c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd8b40cb560 0x7fd8b41526c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:10.990 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.978+0000 7fd8bbf6c700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd8b40ce290 0x7fd8b41f6980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:10.991 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.978+0000 7fd8bbf6c700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b40af7f0 con 0x7fd8b40cb560 2023-10-14T10:37:10.991 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.978+0000 7fd8bbf6c700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b415bb90 con 0x7fd8b40ce290 2023-10-14T10:37:10.991 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.978+0000 7fd8bbf6c700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b415be70 con 0x7fd8b406d390 2023-10-14T10:37:10.991 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.978+0000 7fd8b9507700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd8b40cb560 0x7fd8b41526c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:10.992 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.978+0000 7fd8b9507700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd8b40cb560 0x7fd8b41526c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46036/0 (socket says 172.21.15.67:46036) 2023-10-14T10:37:10.992 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.978+0000 7fd8b9507700 1 -- 172.21.15.67:0/247153746 learned_addr learned my addr 172.21.15.67:0/247153746 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:37:10.992 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.978+0000 7fd8b9d08700 1 --2- 172.21.15.67:0/247153746 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fd8b406d390 0x7fd8b40c9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:10.993 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.978+0000 7fd8ba509700 1 --2- 172.21.15.67:0/247153746 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd8b40ce290 0x7fd8b41f6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:10.993 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.979+0000 7fd8b9507700 1 -- 172.21.15.67:0/247153746 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fd8b406d390 msgr2=0x7fd8b40c9520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:10.993 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.979+0000 7fd8b9507700 1 --2- 172.21.15.67:0/247153746 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fd8b406d390 0x7fd8b40c9520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:10.993 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.979+0000 7fd8b9507700 1 -- 172.21.15.67:0/247153746 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd8b40ce290 msgr2=0x7fd8b41f6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:10.994 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.979+0000 7fd8b9507700 1 --2- 172.21.15.67:0/247153746 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd8b40ce290 0x7fd8b41f6980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:10.994 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.979+0000 7fd8b9507700 1 -- 172.21.15.67:0/247153746 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8ac009b30 con 0x7fd8b40cb560 2023-10-14T10:37:10.994 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.979+0000 7fd8ba509700 1 --2- 172.21.15.67:0/247153746 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd8b40ce290 0x7fd8b41f6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-14T10:37:10.995 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.979+0000 7fd8b9507700 1 --2- 172.21.15.67:0/247153746 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd8b40cb560 0x7fd8b41526c0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fd8a8009b90 tx=0x7fd8a8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:10.995 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.979+0000 7fd8a67fc700 1 -- 172.21.15.67:0/247153746 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a80038a0 con 0x7fd8b40cb560 2023-10-14T10:37:10.995 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.979+0000 7fd8bbf6c700 1 -- 172.21.15.67:0/247153746 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8a80094a0 con 0x7fd8b40cb560 2023-10-14T10:37:10.995 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.979+0000 7fd8a67fc700 1 -- 172.21.15.67:0/247153746 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8a801e070 con 0x7fd8b40cb560 2023-10-14T10:37:10.996 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.979+0000 7fd8a67fc700 1 -- 172.21.15.67:0/247153746 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a8003c40 con 0x7fd8b40cb560 2023-10-14T10:37:10.996 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.979+0000 7fd8bbf6c700 1 -- 172.21.15.67:0/247153746 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8b40c9a60 con 0x7fd8b40cb560 2023-10-14T10:37:10.996 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.980+0000 7fd8bbf6c700 1 -- 172.21.15.67:0/247153746 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8b40c35e0 con 0x7fd8b40cb560 2023-10-14T10:37:10.997 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.980+0000 7fd8a67fc700 1 -- 172.21.15.67:0/247153746 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd8a802d450 con 0x7fd8b40cb560 2023-10-14T10:37:10.997 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.981+0000 7fd8a67fc700 1 --2- 172.21.15.67:0/247153746 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fd89806af00 0x7fd89806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:10.997 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.981+0000 7fd8a67fc700 1 -- 172.21.15.67:0/247153746 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5729+0+0 (secure 0 0 0) 0x7fd8a808a650 con 0x7fd8b40cb560 2023-10-14T10:37:10.997 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.981+0000 7fd8b9d08700 1 --2- 172.21.15.67:0/247153746 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fd89806af00 0x7fd89806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:10.998 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.981+0000 7fd8b9d08700 1 --2- 172.21.15.67:0/247153746 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fd89806af00 0x7fd89806d3c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd8b0000f60 tx=0x7fd8b0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:10.998 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:10.985+0000 7fd8a67fc700 1 -- 172.21.15.67:0/247153746 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd8a80556f0 con 0x7fd8b40cb560 2023-10-14T10:37:11.186 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.183+0000 7fd8bbf6c700 1 -- 172.21.15.67:0/247153746 --> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7fd8b40caa80 con 0x7fd89806af00 2023-10-14T10:37:11.186 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.184+0000 7fd8a67fc700 1 -- 172.21.15.67:0/247153746 <== mgr.4100 v2:172.21.15.72:6832/703488382 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fd8b00070d0 con 0x7fd89806af00 2023-10-14T10:37:11.189 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.186+0000 7fd8bbf6c700 1 -- 172.21.15.67:0/247153746 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fd89806af00 msgr2=0x7fd89806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:11.189 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.186+0000 7fd8bbf6c700 1 --2- 172.21.15.67:0/247153746 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fd89806af00 0x7fd89806d3c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd8b0000f60 tx=0x7fd8b0009040 comp rx=0 tx=0).stop 2023-10-14T10:37:11.190 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.186+0000 7fd8bbf6c700 1 -- 172.21.15.67:0/247153746 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd8b40cb560 msgr2=0x7fd8b41526c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:11.190 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.186+0000 7fd8bbf6c700 1 --2- 172.21.15.67:0/247153746 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd8b40cb560 0x7fd8b41526c0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fd8a8009b90 tx=0x7fd8a8004750 comp rx=0 tx=0).stop 2023-10-14T10:37:11.190 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.186+0000 7fd8bbf6c700 1 -- 172.21.15.67:0/247153746 shutdown_connections 2023-10-14T10:37:11.190 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.186+0000 7fd8bbf6c700 1 --2- 172.21.15.67:0/247153746 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fd89806af00 0x7fd89806d3c0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:11.191 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.186+0000 7fd8bbf6c700 1 --2- 172.21.15.67:0/247153746 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd8b40ce290 0x7fd8b41f6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:11.191 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.186+0000 7fd8bbf6c700 1 --2- 172.21.15.67:0/247153746 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fd8b406d390 0x7fd8b40c9520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:11.191 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.186+0000 7fd8bbf6c700 1 --2- 172.21.15.67:0/247153746 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd8b40cb560 0x7fd8b41526c0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:11.192 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.186+0000 7fd8bbf6c700 1 -- 172.21.15.67:0/247153746 >> 172.21.15.67:0/247153746 conn(0x7fd8b40cfc40 msgr2=0x7fd8b40ccda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:37:11.192 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.187+0000 7fd8bbf6c700 1 -- 172.21.15.67:0/247153746 shutdown_connections 2023-10-14T10:37:11.192 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.187+0000 7fd8bbf6c700 1 -- 172.21.15.67:0/247153746 wait complete. 2023-10-14T10:37:11.192 INFO:teuthology.orchestra.run.smithi067.stderr:instructed osd(s) 0 to deep-scrub 2023-10-14T10:37:11.201 INFO:tasks.ceph:Scrubbing osd.1 2023-10-14T10:37:11.201 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-10-14T10:37:11.330 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.328+0000 7f6fb8f85700 1 Processor -- start 2023-10-14T10:37:11.330 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.328+0000 7f6fb8f85700 1 -- start start 2023-10-14T10:37:11.333 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.329+0000 7f6fb8f85700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f6fb40cf430 0x7f6fb40d2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:11.333 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.329+0000 7f6fb8f85700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f6fb40b0d30 0x7f6fb40b11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:11.333 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.329+0000 7f6fb8f85700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fb40d05e0 con 0x7f6fb40cdf40 2023-10-14T10:37:11.334 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.329+0000 7f6fb8f85700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fb40b08d0 con 0x7f6fb40cf430 2023-10-14T10:37:11.334 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.329+0000 7f6fb8f85700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb40cfcb0 con 0x7f6fb40b0d30 2023-10-14T10:37:11.334 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.329+0000 7f6fb1d9b700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f6fb40cf430 0x7f6fb40d2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:11.335 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.329+0000 7f6fb1d9b700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f6fb40cf430 0x7f6fb40d2050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46044/0 (socket says 172.21.15.67:46044) 2023-10-14T10:37:11.335 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.329+0000 7f6fb1d9b700 1 -- 172.21.15.67:0/853609514 learned_addr learned my addr 172.21.15.67:0/853609514 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:37:11.335 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.329+0000 7f6fb2d9d700 1 --2- 172.21.15.67:0/853609514 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f6fb40b0d30 0x7f6fb40b11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:11.336 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.329+0000 7f6fb1d9b700 1 -- 172.21.15.67:0/853609514 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f6fb40b0d30 msgr2=0x7f6fb40b11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:37:11.336 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.329+0000 7f6fb1d9b700 1 --2- 172.21.15.67:0/853609514 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f6fb40b0d30 0x7f6fb40b11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:11.336 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.329+0000 7f6fb1d9b700 1 -- 172.21.15.67:0/853609514 >> v1:172.21.15.72:6789/0 conn(0x7f6fb40cdf40 legacy=0x7f6fb40ce320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:11.337 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.329+0000 7f6fb1d9b700 1 -- 172.21.15.67:0/853609514 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fb40cff90 con 0x7f6fb40cf430 2023-10-14T10:37:11.337 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.329+0000 7f6fb159a700 1 -- 172.21.15.67:0/853609514 <== mon.1 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3305204565 0 0) 0x7f6fb40d05e0 con 0x7f6fb40cdf40 2023-10-14T10:37:11.338 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.329+0000 7f6fb1d9b700 1 --2- 172.21.15.67:0/853609514 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6fb40cf430 0x7f6fb40d2050 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f6fa80098f0 tx=0x7f6fa8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffae08b4f6c6fbb8 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:11.338 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.329+0000 7f6fb159a700 1 -- 172.21.15.67:0/853609514 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa8011830 con 0x7f6fb40cf430 2023-10-14T10:37:11.339 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.330+0000 7f6fb159a700 1 -- 172.21.15.67:0/853609514 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fa8011990 con 0x7f6fb40cf430 2023-10-14T10:37:11.339 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.330+0000 7f6fb159a700 1 -- 172.21.15.67:0/853609514 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa8011d20 con 0x7f6fb40cf430 2023-10-14T10:37:11.340 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.330+0000 7f6fb8f85700 1 -- 172.21.15.67:0/853609514 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6fb40cf430 msgr2=0x7f6fb40d2050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:11.340 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.330+0000 7f6fb8f85700 1 --2- 172.21.15.67:0/853609514 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6fb40cf430 0x7f6fb40d2050 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f6fa80098f0 tx=0x7f6fa8009c00 comp rx=0 tx=0).stop 2023-10-14T10:37:11.340 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.330+0000 7f6fb8f85700 1 -- 172.21.15.67:0/853609514 shutdown_connections 2023-10-14T10:37:11.341 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.330+0000 7f6fb8f85700 1 --2- 172.21.15.67:0/853609514 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f6fb40b0d30 0x7f6fb40b11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:11.341 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.330+0000 7f6fb8f85700 1 --2- 172.21.15.67:0/853609514 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6fb40cf430 0x7f6fb40d2050 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:11.342 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.330+0000 7f6fb8f85700 1 -- 172.21.15.67:0/853609514 >> 172.21.15.67:0/853609514 conn(0x7f6fb40d1530 msgr2=0x7f6fb40d1940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:37:11.342 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.330+0000 7f6fb8f85700 1 -- 172.21.15.67:0/853609514 shutdown_connections 2023-10-14T10:37:11.343 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.330+0000 7f6fb8f85700 1 -- 172.21.15.67:0/853609514 wait complete. 2023-10-14T10:37:11.346 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.331+0000 7f6fb8f85700 1 Processor -- start 2023-10-14T10:37:11.347 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.332+0000 7f6fb8f85700 1 -- start start 2023-10-14T10:37:11.347 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.332+0000 7f6fb8f85700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6fb40b0d30 0x7f6fb41598d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:11.347 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.332+0000 7f6fb8f85700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f6fb40cdf40 0x7f6fb4159090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:11.348 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.332+0000 7f6fb8f85700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6fb40cf430 0x7f6fb40c8600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:11.348 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.332+0000 7f6fb8f85700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb40b67a0 con 0x7f6fb40b0d30 2023-10-14T10:37:11.348 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.332+0000 7f6fb8f85700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb415a100 con 0x7f6fb40cf430 2023-10-14T10:37:11.348 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.332+0000 7f6fb8f85700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb41e5cc0 con 0x7f6fb40cdf40 2023-10-14T10:37:11.349 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.332+0000 7f6fb259c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6fb40b0d30 0x7f6fb41598d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:11.349 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.332+0000 7f6fb259c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6fb40b0d30 0x7f6fb41598d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46048/0 (socket says 172.21.15.67:46048) 2023-10-14T10:37:11.349 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.332+0000 7f6fb259c700 1 -- 172.21.15.67:0/641372857 learned_addr learned my addr 172.21.15.67:0/641372857 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:37:11.350 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.333+0000 7f6fb2d9d700 1 --2- 172.21.15.67:0/641372857 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6fb40cf430 0x7f6fb40c8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:11.350 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.333+0000 7f6fb1d9b700 1 --2- 172.21.15.67:0/641372857 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f6fb40cdf40 0x7f6fb4159090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:11.350 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.333+0000 7f6fb259c700 1 -- 172.21.15.67:0/641372857 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f6fb40cdf40 msgr2=0x7f6fb4159090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:11.350 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.333+0000 7f6fb259c700 1 --2- 172.21.15.67:0/641372857 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f6fb40cdf40 0x7f6fb4159090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:11.351 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.333+0000 7f6fb259c700 1 -- 172.21.15.67:0/641372857 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6fb40cf430 msgr2=0x7f6fb40c8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:11.351 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.333+0000 7f6fb259c700 1 --2- 172.21.15.67:0/641372857 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6fb40cf430 0x7f6fb40c8600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:11.351 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.333+0000 7f6fb259c700 1 -- 172.21.15.67:0/641372857 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fa8009550 con 0x7f6fb40b0d30 2023-10-14T10:37:11.352 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.333+0000 7f6fb1d9b700 1 --2- 172.21.15.67:0/641372857 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f6fb40cdf40 0x7f6fb4159090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-14T10:37:11.352 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.333+0000 7f6fb2d9d700 1 --2- 172.21.15.67:0/641372857 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6fb40cf430 0x7f6fb40c8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-14T10:37:11.352 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.333+0000 7f6fb259c700 1 --2- 172.21.15.67:0/641372857 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6fb40b0d30 0x7f6fb41598d0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f6fa400c550 tx=0x7f6fa400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:11.352 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.333+0000 7f6fa2ffd700 1 -- 172.21.15.67:0/641372857 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa40054e0 con 0x7f6fb40b0d30 2023-10-14T10:37:11.353 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.333+0000 7f6fa2ffd700 1 -- 172.21.15.67:0/641372857 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fa40036d0 con 0x7f6fb40b0d30 2023-10-14T10:37:11.353 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.333+0000 7f6fb8f85700 1 -- 172.21.15.67:0/641372857 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fb41e5390 con 0x7f6fb40b0d30 2023-10-14T10:37:11.354 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.333+0000 7f6fa2ffd700 1 -- 172.21.15.67:0/641372857 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa4003a80 con 0x7f6fb40b0d30 2023-10-14T10:37:11.354 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.333+0000 7f6fb8f85700 1 -- 172.21.15.67:0/641372857 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fb41e5950 con 0x7f6fb40b0d30 2023-10-14T10:37:11.354 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.334+0000 7f6fb8f85700 1 -- 172.21.15.67:0/641372857 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6fb40d0270 con 0x7f6fb40b0d30 2023-10-14T10:37:11.354 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.334+0000 7f6fa2ffd700 1 -- 172.21.15.67:0/641372857 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f6fa4003be0 con 0x7f6fb40b0d30 2023-10-14T10:37:11.355 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.335+0000 7f6fa2ffd700 1 --2- 172.21.15.67:0/641372857 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f6f9406af00 0x7f6f9406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:11.355 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.335+0000 7f6fa2ffd700 1 -- 172.21.15.67:0/641372857 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5729+0+0 (secure 0 0 0) 0x7f6fa4088ab0 con 0x7f6fb40b0d30 2023-10-14T10:37:11.355 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.335+0000 7f6fa2ffd700 1 --2- 172.21.15.67:0/641372857 >> [v2:172.21.15.67:6816/2435538787,v1:172.21.15.67:6818/2435538787] conn(0x7f6f94073100 0x7f6f94075540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:11.356 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.335+0000 7f6fa2ffd700 1 -- 172.21.15.67:0/641372857 --> [v2:172.21.15.67:6816/2435538787,v1:172.21.15.67:6818/2435538787] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6f94075c10 con 0x7f6f94073100 2023-10-14T10:37:11.356 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.335+0000 7f6fa2ffd700 1 -- 172.21.15.67:0/641372857 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6fa408d2c0 con 0x7f6fb40b0d30 2023-10-14T10:37:11.356 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.335+0000 7f6fb1d9b700 1 --2- 172.21.15.67:0/641372857 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f6f9406af00 0x7f6f9406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:11.356 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.335+0000 7f6fb2d9d700 1 --2- 172.21.15.67:0/641372857 >> [v2:172.21.15.67:6816/2435538787,v1:172.21.15.67:6818/2435538787] conn(0x7f6f94073100 0x7f6f94075540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:11.357 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.336+0000 7f6fb2d9d700 1 --2- 172.21.15.67:0/641372857 >> [v2:172.21.15.67:6816/2435538787,v1:172.21.15.67:6818/2435538787] conn(0x7f6f94073100 0x7f6f94075540 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:11.357 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.336+0000 7f6fb1d9b700 1 --2- 172.21.15.67:0/641372857 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f6f9406af00 0x7f6f9406d3c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6fb415ac70 tx=0x7f6fa8008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:11.357 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.336+0000 7f6fa2ffd700 1 -- 172.21.15.67:0/641372857 <== osd.1 v2:172.21.15.67:6816/2435538787 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7f6f94075c10 con 0x7f6f94073100 2023-10-14T10:37:11.358 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.351+0000 7f6fb8f85700 1 -- 172.21.15.67:0/641372857 --> [v2:172.21.15.67:6816/2435538787,v1:172.21.15.67:6818/2435538787] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f6fb4067d20 con 0x7f6f94073100 2023-10-14T10:37:11.358 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.351+0000 7f6fa2ffd700 1 -- 172.21.15.67:0/641372857 <== osd.1 v2:172.21.15.67:6816/2435538787 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f6fb4067d20 con 0x7f6f94073100 2023-10-14T10:37:11.359 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2023-10-14T10:37:11.359 INFO:teuthology.orchestra.run.smithi067.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-14T10:37:11.360 INFO:teuthology.orchestra.run.smithi067.stdout:} 2023-10-14T10:37:11.361 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.351+0000 7f6fb8f85700 1 -- 172.21.15.67:0/641372857 >> [v2:172.21.15.67:6816/2435538787,v1:172.21.15.67:6818/2435538787] conn(0x7f6f94073100 msgr2=0x7f6f94075540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:11.361 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.352+0000 7f6fb8f85700 1 --2- 172.21.15.67:0/641372857 >> [v2:172.21.15.67:6816/2435538787,v1:172.21.15.67:6818/2435538787] conn(0x7f6f94073100 0x7f6f94075540 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:11.361 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.352+0000 7f6fb8f85700 1 -- 172.21.15.67:0/641372857 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f6f9406af00 msgr2=0x7f6f9406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:11.362 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.352+0000 7f6fb8f85700 1 --2- 172.21.15.67:0/641372857 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f6f9406af00 0x7f6f9406d3c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6fb415ac70 tx=0x7f6fa8008030 comp rx=0 tx=0).stop 2023-10-14T10:37:11.362 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.352+0000 7f6fb8f85700 1 -- 172.21.15.67:0/641372857 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6fb40b0d30 msgr2=0x7f6fb41598d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:11.362 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.352+0000 7f6fb8f85700 1 --2- 172.21.15.67:0/641372857 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6fb40b0d30 0x7f6fb41598d0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f6fa400c550 tx=0x7f6fa400c910 comp rx=0 tx=0).stop 2023-10-14T10:37:11.363 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.352+0000 7f6fb2d9d700 1 -- 172.21.15.67:0/641372857 reap_dead start 2023-10-14T10:37:11.363 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.352+0000 7f6fb8f85700 1 -- 172.21.15.67:0/641372857 shutdown_connections 2023-10-14T10:37:11.363 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.352+0000 7f6fb8f85700 1 -- 172.21.15.67:0/641372857 >> 172.21.15.67:0/641372857 conn(0x7f6fb40d1530 msgr2=0x7f6fb40b2840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:37:11.363 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.352+0000 7f6fb8f85700 1 -- 172.21.15.67:0/641372857 shutdown_connections 2023-10-14T10:37:11.364 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.352+0000 7f6fb8f85700 1 -- 172.21.15.67:0/641372857 wait complete. 2023-10-14T10:37:11.366 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-10-14T10:37:11.491 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.489+0000 7f8c0e622700 1 Processor -- start 2023-10-14T10:37:11.492 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.489+0000 7f8c0e622700 1 -- start start 2023-10-14T10:37:11.494 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.490+0000 7f8c0e622700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f8c0815ddc0 0x7f8c08161800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:11.495 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.490+0000 7f8c0e622700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c080ce350 con 0x7f8c08161d40 2023-10-14T10:37:11.495 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.490+0000 7f8c0e622700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c08162630 con 0x7f8c0815d9e0 2023-10-14T10:37:11.495 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.490+0000 7f8c0e622700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c081621c0 con 0x7f8c0815ddc0 2023-10-14T10:37:11.496 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.490+0000 7f8c0cbbf700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f8c08161d40 0x7f8c08167b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:53830/0 (socket says 172.21.15.67:53830) 2023-10-14T10:37:11.496 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.490+0000 7f8c0cbbf700 1 -- 172.21.15.67:0/2462332173 learned_addr learned my addr 172.21.15.67:0/2462332173 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:37:11.496 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.490+0000 7f8c077fe700 1 --2- 172.21.15.67:0/2462332173 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f8c0815ddc0 0x7f8c08161800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:11.496 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.490+0000 7f8c06ffd700 1 -- 172.21.15.67:0/2462332173 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3330467440 0 0) 0x7f8c080ce350 con 0x7f8c08161d40 2023-10-14T10:37:11.497 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.491+0000 7f8c06ffd700 1 -- 172.21.15.67:0/2462332173 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8bf00035b0 con 0x7f8c08161d40 2023-10-14T10:37:11.497 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.491+0000 7f8c06ffd700 1 -- 172.21.15.67:0/2462332173 <== mon.1 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 808789647 0 0) 0x7f8c08162630 con 0x7f8c0815d9e0 2023-10-14T10:37:11.497 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.491+0000 7f8c06ffd700 1 -- 172.21.15.67:0/2462332173 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c080ce350 con 0x7f8c0815d9e0 2023-10-14T10:37:11.498 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.491+0000 7f8c06ffd700 1 -- 172.21.15.67:0/2462332173 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1358711164 0 0) 0x7f8bf00035b0 con 0x7f8c08161d40 2023-10-14T10:37:11.498 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.491+0000 7f8c077fe700 1 -- 172.21.15.67:0/2462332173 >> v1:172.21.15.72:6789/0 conn(0x7f8c0815d9e0 legacy=0x7f8c080d1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:11.498 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.491+0000 7f8c077fe700 1 -- 172.21.15.67:0/2462332173 >> v1:172.21.15.67:6789/0 conn(0x7f8c08161d40 legacy=0x7f8c08167b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:11.498 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.491+0000 7f8c077fe700 1 -- 172.21.15.67:0/2462332173 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c080ce5f0 con 0x7f8c0815ddc0 2023-10-14T10:37:11.499 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.491+0000 7f8c077fe700 1 --2- 172.21.15.67:0/2462332173 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f8c0815ddc0 0x7f8c08161800 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc009870 tx=0x7f8bfc009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f26aded2c2ee2cca server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:11.499 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.491+0000 7f8c06ffd700 1 -- 172.21.15.67:0/2462332173 <== mon.2 v2:172.21.15.72:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c000019f0 con 0x7f8c0815ddc0 2023-10-14T10:37:11.499 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.491+0000 7f8c06ffd700 1 -- 172.21.15.67:0/2462332173 <== mon.2 v2:172.21.15.72:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bfc004030 con 0x7f8c0815ddc0 2023-10-14T10:37:11.500 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.491+0000 7f8c06ffd700 1 -- 172.21.15.67:0/2462332173 <== mon.2 v2:172.21.15.72:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bfc0043c0 con 0x7f8c0815ddc0 2023-10-14T10:37:11.500 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.492+0000 7f8c0e622700 1 -- 172.21.15.67:0/2462332173 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f8c0815ddc0 msgr2=0x7f8c08161800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:11.500 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.492+0000 7f8c0e622700 1 --2- 172.21.15.67:0/2462332173 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f8c0815ddc0 0x7f8c08161800 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc009870 tx=0x7f8bfc009b80 comp rx=0 tx=0).stop 2023-10-14T10:37:11.500 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.492+0000 7f8c0e622700 1 -- 172.21.15.67:0/2462332173 shutdown_connections 2023-10-14T10:37:11.501 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.492+0000 7f8c0e622700 1 --2- 172.21.15.67:0/2462332173 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f8c0815ddc0 0x7f8c08161800 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:11.501 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.492+0000 7f8c0e622700 1 -- 172.21.15.67:0/2462332173 >> 172.21.15.67:0/2462332173 conn(0x7f8c0809cb40 msgr2=0x7f8c080d37d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:37:11.501 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.492+0000 7f8c0e622700 1 -- 172.21.15.67:0/2462332173 shutdown_connections 2023-10-14T10:37:11.502 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.492+0000 7f8c0e622700 1 -- 172.21.15.67:0/2462332173 wait complete. 2023-10-14T10:37:11.504 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.493+0000 7f8c0e622700 1 Processor -- start 2023-10-14T10:37:11.504 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.493+0000 7f8c0e622700 1 -- start start 2023-10-14T10:37:11.505 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.494+0000 7f8c0e622700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f8c0815d9e0 0x7f8c08157b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:11.505 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.494+0000 7f8c0e622700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c0815ddc0 0x7f8c0815a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:11.505 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.494+0000 7f8c0e622700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8c08161d40 0x7f8c080c9110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:11.506 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.494+0000 7f8c0e622700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c08159e70 con 0x7f8c0815ddc0 2023-10-14T10:37:11.506 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.494+0000 7f8c0e622700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c08159260 con 0x7f8c08161d40 2023-10-14T10:37:11.506 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.494+0000 7f8c0e622700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c08159540 con 0x7f8c0815d9e0 2023-10-14T10:37:11.506 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.494+0000 7f8c077fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c0815ddc0 0x7f8c0815a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:11.507 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.494+0000 7f8c077fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c0815ddc0 0x7f8c0815a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46066/0 (socket says 172.21.15.67:46066) 2023-10-14T10:37:11.507 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.494+0000 7f8c077fe700 1 -- 172.21.15.67:0/2904228520 learned_addr learned my addr 172.21.15.67:0/2904228520 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:37:11.507 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.494+0000 7f8c07fff700 1 --2- 172.21.15.67:0/2904228520 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f8c0815d9e0 0x7f8c08157b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:11.508 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.494+0000 7f8c0cbbf700 1 --2- 172.21.15.67:0/2904228520 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8c08161d40 0x7f8c080c9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:11.508 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.494+0000 7f8c077fe700 1 -- 172.21.15.67:0/2904228520 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f8c0815d9e0 msgr2=0x7f8c08157b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:11.508 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.494+0000 7f8c077fe700 1 --2- 172.21.15.67:0/2904228520 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f8c0815d9e0 0x7f8c08157b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:11.509 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.494+0000 7f8c077fe700 1 -- 172.21.15.67:0/2904228520 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8c08161d40 msgr2=0x7f8c080c9110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:11.509 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.494+0000 7f8c077fe700 1 --2- 172.21.15.67:0/2904228520 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8c08161d40 0x7f8c080c9110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:11.509 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.494+0000 7f8c077fe700 1 -- 172.21.15.67:0/2904228520 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bfc0094a0 con 0x7f8c0815ddc0 2023-10-14T10:37:11.509 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.494+0000 7f8c077fe700 1 --2- 172.21.15.67:0/2904228520 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c0815ddc0 0x7f8c0815a480 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc0090b0 tx=0x7f8bfc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:11.510 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.495+0000 7f8c04ff9700 1 -- 172.21.15.67:0/2904228520 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bfc003e60 con 0x7f8c0815ddc0 2023-10-14T10:37:11.510 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.495+0000 7f8c04ff9700 1 -- 172.21.15.67:0/2904228520 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bfc01f070 con 0x7f8c0815ddc0 2023-10-14T10:37:11.510 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.495+0000 7f8c0e622700 1 -- 172.21.15.67:0/2904228520 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c08159820 con 0x7f8c0815ddc0 2023-10-14T10:37:11.511 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.495+0000 7f8c04ff9700 1 -- 172.21.15.67:0/2904228520 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bfc011850 con 0x7f8c0815ddc0 2023-10-14T10:37:11.511 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.495+0000 7f8c0e622700 1 -- 172.21.15.67:0/2904228520 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c0815a9c0 con 0x7f8c0815ddc0 2023-10-14T10:37:11.511 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.496+0000 7f8c0e622700 1 -- 172.21.15.67:0/2904228520 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c08067d20 con 0x7f8c0815ddc0 2023-10-14T10:37:11.511 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.496+0000 7f8c04ff9700 1 -- 172.21.15.67:0/2904228520 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f8bfc0119b0 con 0x7f8c0815ddc0 2023-10-14T10:37:11.512 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.497+0000 7f8c04ff9700 1 --2- 172.21.15.67:0/2904228520 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f8bf406af00 0x7f8bf406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:11.512 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.497+0000 7f8c04ff9700 1 -- 172.21.15.67:0/2904228520 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5729+0+0 (secure 0 0 0) 0x7f8bfc021070 con 0x7f8c0815ddc0 2023-10-14T10:37:11.512 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.497+0000 7f8c07fff700 1 --2- 172.21.15.67:0/2904228520 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f8bf406af00 0x7f8bf406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:11.513 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.497+0000 7f8c07fff700 1 --2- 172.21.15.67:0/2904228520 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f8bf406af00 0x7f8bf406d3c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8bf8001220 tx=0x7f8bf8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:11.513 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.501+0000 7f8c04ff9700 1 -- 172.21.15.67:0/2904228520 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8bfc055f00 con 0x7f8c0815ddc0 2023-10-14T10:37:11.704 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.702+0000 7f8c0e622700 1 -- 172.21.15.67:0/2904228520 --> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f8c080ce8d0 con 0x7f8bf406af00 2023-10-14T10:37:11.718 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.703+0000 7f8c04ff9700 1 -- 172.21.15.67:0/2904228520 <== mgr.4100 v2:172.21.15.72:6832/703488382 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f8bf8001550 con 0x7f8bf406af00 2023-10-14T10:37:11.718 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.705+0000 7f8c0e622700 1 -- 172.21.15.67:0/2904228520 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f8bf406af00 msgr2=0x7f8bf406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:11.718 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.705+0000 7f8c0e622700 1 --2- 172.21.15.67:0/2904228520 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f8bf406af00 0x7f8bf406d3c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8bf8001220 tx=0x7f8bf8009040 comp rx=0 tx=0).stop 2023-10-14T10:37:11.719 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.705+0000 7f8c0e622700 1 -- 172.21.15.67:0/2904228520 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c0815ddc0 msgr2=0x7f8c0815a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:11.719 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.705+0000 7f8c0e622700 1 --2- 172.21.15.67:0/2904228520 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c0815ddc0 0x7f8c0815a480 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc0090b0 tx=0x7f8bfc004750 comp rx=0 tx=0).stop 2023-10-14T10:37:11.719 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.705+0000 7f8c0e622700 1 -- 172.21.15.67:0/2904228520 shutdown_connections 2023-10-14T10:37:11.719 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.705+0000 7f8c0e622700 1 --2- 172.21.15.67:0/2904228520 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f8bf406af00 0x7f8bf406d3c0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:11.720 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.705+0000 7f8c0e622700 1 --2- 172.21.15.67:0/2904228520 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8c08161d40 0x7f8c080c9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:11.720 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.705+0000 7f8c0e622700 1 --2- 172.21.15.67:0/2904228520 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f8c0815d9e0 0x7f8c08157b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:11.720 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.705+0000 7f8c0e622700 1 --2- 172.21.15.67:0/2904228520 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c0815ddc0 0x7f8c0815a480 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:11.721 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.705+0000 7f8c0e622700 1 -- 172.21.15.67:0/2904228520 >> 172.21.15.67:0/2904228520 conn(0x7f8c0809cb40 msgr2=0x7f8c080b52c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:37:11.721 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.705+0000 7f8c0e622700 1 -- 172.21.15.67:0/2904228520 shutdown_connections 2023-10-14T10:37:11.721 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.705+0000 7f8c0e622700 1 -- 172.21.15.67:0/2904228520 wait complete. 2023-10-14T10:37:11.721 INFO:teuthology.orchestra.run.smithi067.stderr:instructed osd(s) 1 to deep-scrub 2023-10-14T10:37:11.723 INFO:tasks.ceph:Scrubbing osd.2 2023-10-14T10:37:11.723 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-10-14T10:37:11.852 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.850+0000 7f3cb3139700 1 Processor -- start 2023-10-14T10:37:11.852 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.850+0000 7f3cb3139700 1 -- start start 2023-10-14T10:37:11.854 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.850+0000 7f3cb3139700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f3cac162070 0x7f3cac0cd330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:11.854 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.850+0000 7f3cb3139700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cac0cee20 con 0x7f3cac15d840 2023-10-14T10:37:11.855 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.850+0000 7f3cb3139700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cac0b2ce0 con 0x7f3cac0cd870 2023-10-14T10:37:11.855 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.850+0000 7f3cb3139700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cac0cdce0 con 0x7f3cac162070 2023-10-14T10:37:11.855 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.850+0000 7f3cabfff700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f3cac162070 0x7f3cac0cd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:11.855 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.851+0000 7f3cabfff700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f3cac162070 0x7f3cac0cd330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46074/0 (socket says 172.21.15.67:46074) 2023-10-14T10:37:11.856 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.851+0000 7f3cabfff700 1 -- 172.21.15.67:0/3395281468 learned_addr learned my addr 172.21.15.67:0/3395281468 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:37:11.856 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.851+0000 7f3cb0ed5700 1 --1- 172.21.15.67:0/3395281468 >> v1:172.21.15.67:6789/0 conn(0x7f3cac15d840 0x7f3cac0cdfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:53840/0 (socket says 172.21.15.67:53840) 2023-10-14T10:37:11.856 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.851+0000 7f3cabfff700 1 -- 172.21.15.67:0/3395281468 >> v1:172.21.15.72:6789/0 conn(0x7f3cac0cd870 legacy=0x7f3cac0caa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:37:11.856 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.851+0000 7f3cabfff700 1 -- 172.21.15.67:0/3395281468 >> v1:172.21.15.67:6789/0 conn(0x7f3cac15d840 legacy=0x7f3cac0cdfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:11.857 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.851+0000 7f3cabfff700 1 -- 172.21.15.67:0/3395281468 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cac0ceb40 con 0x7f3cac162070 2023-10-14T10:37:11.857 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.851+0000 7f3cab7fe700 1 -- 172.21.15.67:0/3395281468 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 252978514 0 0) 0x7f3cac0cee20 con 0x7f3cac15d840 2023-10-14T10:37:11.857 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.851+0000 7f3cabfff700 1 --2- 172.21.15.67:0/3395281468 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3cac162070 0x7f3cac0cd330 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f3c980098f0 tx=0x7f3c98009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0c0689f4d2b4339 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:11.858 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.851+0000 7f3cab7fe700 1 -- 172.21.15.67:0/3395281468 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c9800b8a0 con 0x7f3cac162070 2023-10-14T10:37:11.858 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.851+0000 7f3cab7fe700 1 -- 172.21.15.67:0/3395281468 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c9800ba00 con 0x7f3cac162070 2023-10-14T10:37:11.858 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.851+0000 7f3cab7fe700 1 -- 172.21.15.67:0/3395281468 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c9800bd90 con 0x7f3cac162070 2023-10-14T10:37:11.858 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.852+0000 7f3cb3139700 1 -- 172.21.15.67:0/3395281468 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3cac162070 msgr2=0x7f3cac0cd330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:11.859 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.852+0000 7f3cb3139700 1 --2- 172.21.15.67:0/3395281468 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3cac162070 0x7f3cac0cd330 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f3c980098f0 tx=0x7f3c98009c00 comp rx=0 tx=0).stop 2023-10-14T10:37:11.859 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.852+0000 7f3cb3139700 1 -- 172.21.15.67:0/3395281468 shutdown_connections 2023-10-14T10:37:11.859 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.852+0000 7f3cb3139700 1 --2- 172.21.15.67:0/3395281468 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3cac162070 0x7f3cac0cd330 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:11.860 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.852+0000 7f3cb3139700 1 -- 172.21.15.67:0/3395281468 >> 172.21.15.67:0/3395281468 conn(0x7f3cac06bd70 msgr2=0x7f3cac06c180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:37:11.860 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.852+0000 7f3cb3139700 1 -- 172.21.15.67:0/3395281468 shutdown_connections 2023-10-14T10:37:11.860 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.852+0000 7f3cb3139700 1 -- 172.21.15.67:0/3395281468 wait complete. 2023-10-14T10:37:11.863 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.853+0000 7f3cb3139700 1 Processor -- start 2023-10-14T10:37:11.864 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.853+0000 7f3cb3139700 1 -- start start 2023-10-14T10:37:11.864 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.854+0000 7f3cb3139700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3cac0cd870 0x7f3cac0c9be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:11.864 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.854+0000 7f3cb3139700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3cac15d840 0x7f3cac0d0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:11.864 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.854+0000 7f3cb3139700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f3cac162070 0x7f3cac0c74d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:11.865 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.854+0000 7f3cb3139700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cac0b4ce0 con 0x7f3cac0cd870 2023-10-14T10:37:11.865 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.854+0000 7f3cb3139700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cac0b4fc0 con 0x7f3cac15d840 2023-10-14T10:37:11.865 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.854+0000 7f3cb3139700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cac0b52a0 con 0x7f3cac162070 2023-10-14T10:37:11.866 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.854+0000 7f3cb0ed5700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3cac0cd870 0x7f3cac0c9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:11.866 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.854+0000 7f3cb0ed5700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3cac0cd870 0x7f3cac0c9be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46078/0 (socket says 172.21.15.67:46078) 2023-10-14T10:37:11.866 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.854+0000 7f3cb0ed5700 1 -- 172.21.15.67:0/605006594 learned_addr learned my addr 172.21.15.67:0/605006594 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:37:11.866 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.854+0000 7f3cabfff700 1 --2- 172.21.15.67:0/605006594 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3cac15d840 0x7f3cac0d0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:11.867 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.854+0000 7f3cb16d6700 1 --2- 172.21.15.67:0/605006594 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f3cac162070 0x7f3cac0c74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:11.867 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.854+0000 7f3cb0ed5700 1 -- 172.21.15.67:0/605006594 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f3cac162070 msgr2=0x7f3cac0c74d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:11.867 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.854+0000 7f3cb0ed5700 1 --2- 172.21.15.67:0/605006594 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f3cac162070 0x7f3cac0c74d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:11.867 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.854+0000 7f3cb0ed5700 1 -- 172.21.15.67:0/605006594 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3cac15d840 msgr2=0x7f3cac0d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:11.868 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.854+0000 7f3cb0ed5700 1 --2- 172.21.15.67:0/605006594 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3cac15d840 0x7f3cac0d0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:11.868 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.854+0000 7f3cb0ed5700 1 -- 172.21.15.67:0/605006594 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c98009550 con 0x7f3cac0cd870 2023-10-14T10:37:11.868 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.854+0000 7f3cabfff700 1 --2- 172.21.15.67:0/605006594 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3cac15d840 0x7f3cac0d0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-14T10:37:11.869 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.855+0000 7f3cb0ed5700 1 --2- 172.21.15.67:0/605006594 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3cac0cd870 0x7f3cac0c9be0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f3ca000c520 tx=0x7f3ca000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:11.869 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.855+0000 7f3ca97fa700 1 -- 172.21.15.67:0/605006594 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca0004c80 con 0x7f3cac0cd870 2023-10-14T10:37:11.869 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.855+0000 7f3ca97fa700 1 -- 172.21.15.67:0/605006594 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ca0004de0 con 0x7f3cac0cd870 2023-10-14T10:37:11.869 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.855+0000 7f3cb3139700 1 -- 172.21.15.67:0/605006594 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cac0b5580 con 0x7f3cac0cd870 2023-10-14T10:37:11.870 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.855+0000 7f3ca97fa700 1 -- 172.21.15.67:0/605006594 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca0007310 con 0x7f3cac0cd870 2023-10-14T10:37:11.870 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.855+0000 7f3cb3139700 1 -- 172.21.15.67:0/605006594 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cac0ca120 con 0x7f3cac0cd870 2023-10-14T10:37:11.870 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.856+0000 7f3cb3139700 1 -- 172.21.15.67:0/605006594 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3cac0b5130 con 0x7f3cac0cd870 2023-10-14T10:37:11.870 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.856+0000 7f3ca97fa700 1 -- 172.21.15.67:0/605006594 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f3ca0007470 con 0x7f3cac0cd870 2023-10-14T10:37:11.871 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.857+0000 7f3ca97fa700 1 --2- 172.21.15.67:0/605006594 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f3c9006af00 0x7f3c9006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:11.871 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.857+0000 7f3ca97fa700 1 -- 172.21.15.67:0/605006594 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5729+0+0 (secure 0 0 0) 0x7f3ca0089320 con 0x7f3cac0cd870 2023-10-14T10:37:11.871 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.857+0000 7f3ca97fa700 1 --2- 172.21.15.67:0/605006594 >> [v2:172.21.15.67:6808/2827491791,v1:172.21.15.67:6809/2827491791] conn(0x7f3c90073100 0x7f3c90075540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:11.872 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.857+0000 7f3ca97fa700 1 -- 172.21.15.67:0/605006594 --> [v2:172.21.15.67:6808/2827491791,v1:172.21.15.67:6809/2827491791] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3c90075c10 con 0x7f3c90073100 2023-10-14T10:37:11.872 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.857+0000 7f3ca97fa700 1 -- 172.21.15.67:0/605006594 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3ca008b9e0 con 0x7f3cac0cd870 2023-10-14T10:37:11.872 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.857+0000 7f3cabfff700 1 --2- 172.21.15.67:0/605006594 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f3c9006af00 0x7f3c9006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:11.872 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.857+0000 7f3cb16d6700 1 --2- 172.21.15.67:0/605006594 >> [v2:172.21.15.67:6808/2827491791,v1:172.21.15.67:6809/2827491791] conn(0x7f3c90073100 0x7f3c90075540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:11.873 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.857+0000 7f3cb16d6700 1 --2- 172.21.15.67:0/605006594 >> [v2:172.21.15.67:6808/2827491791,v1:172.21.15.67:6809/2827491791] conn(0x7f3c90073100 0x7f3c90075540 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:11.873 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.857+0000 7f3cabfff700 1 --2- 172.21.15.67:0/605006594 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f3c9006af00 0x7f3c9006d3c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3c980098f0 tx=0x7f3c98008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:11.873 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.858+0000 7f3ca97fa700 1 -- 172.21.15.67:0/605006594 <== osd.2 v2:172.21.15.67:6808/2827491791 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7f3c90075c10 con 0x7f3c90073100 2023-10-14T10:37:11.875 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.873+0000 7f3cb3139700 1 -- 172.21.15.67:0/605006594 --> [v2:172.21.15.67:6808/2827491791,v1:172.21.15.67:6809/2827491791] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f3cac161870 con 0x7f3c90073100 2023-10-14T10:37:11.875 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.873+0000 7f3ca97fa700 1 -- 172.21.15.67:0/605006594 <== osd.2 v2:172.21.15.67:6808/2827491791 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f3ca4003a00 con 0x7f3c90073100 2023-10-14T10:37:11.876 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2023-10-14T10:37:11.876 INFO:teuthology.orchestra.run.smithi067.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-14T10:37:11.876 INFO:teuthology.orchestra.run.smithi067.stdout:} 2023-10-14T10:37:11.877 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.873+0000 7f3cb3139700 1 -- 172.21.15.67:0/605006594 >> [v2:172.21.15.67:6808/2827491791,v1:172.21.15.67:6809/2827491791] conn(0x7f3c90073100 msgr2=0x7f3c90075540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:11.878 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.873+0000 7f3cb3139700 1 --2- 172.21.15.67:0/605006594 >> [v2:172.21.15.67:6808/2827491791,v1:172.21.15.67:6809/2827491791] conn(0x7f3c90073100 0x7f3c90075540 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:11.878 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.873+0000 7f3cb3139700 1 -- 172.21.15.67:0/605006594 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f3c9006af00 msgr2=0x7f3c9006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:11.878 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.873+0000 7f3cb3139700 1 --2- 172.21.15.67:0/605006594 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f3c9006af00 0x7f3c9006d3c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3c980098f0 tx=0x7f3c98008030 comp rx=0 tx=0).stop 2023-10-14T10:37:11.879 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.873+0000 7f3cb3139700 1 -- 172.21.15.67:0/605006594 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3cac0cd870 msgr2=0x7f3cac0c9be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:11.879 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.873+0000 7f3cb3139700 1 --2- 172.21.15.67:0/605006594 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3cac0cd870 0x7f3cac0c9be0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f3ca000c520 tx=0x7f3ca000c8e0 comp rx=0 tx=0).stop 2023-10-14T10:37:11.879 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.873+0000 7f3cb16d6700 1 -- 172.21.15.67:0/605006594 reap_dead start 2023-10-14T10:37:11.880 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.873+0000 7f3cb3139700 1 -- 172.21.15.67:0/605006594 shutdown_connections 2023-10-14T10:37:11.880 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.873+0000 7f3cb3139700 1 -- 172.21.15.67:0/605006594 >> 172.21.15.67:0/605006594 conn(0x7f3cac06bd70 msgr2=0x7f3cac15b1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:37:11.880 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.873+0000 7f3cb3139700 1 -- 172.21.15.67:0/605006594 shutdown_connections 2023-10-14T10:37:11.881 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:11.873+0000 7f3cb3139700 1 -- 172.21.15.67:0/605006594 wait complete. 2023-10-14T10:37:11.887 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-10-14T10:37:12.014 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.012+0000 7f92e4942700 1 Processor -- start 2023-10-14T10:37:12.014 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.012+0000 7f92e4942700 1 -- start start 2023-10-14T10:37:12.017 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.013+0000 7f92e4942700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f92e00cc3f0 0x7f92e00cc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:12.017 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.013+0000 7f92e4942700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92e0071ab0 con 0x7f92e0161670 2023-10-14T10:37:12.018 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.013+0000 7f92e4942700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92e0161bc0 con 0x7f92e006af50 2023-10-14T10:37:12.018 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.013+0000 7f92e4942700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92e0161ea0 con 0x7f92e00cc3f0 2023-10-14T10:37:12.018 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.013+0000 7f92ddd9b700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f92e0161670 0x7f92e006a840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:53856/0 (socket says 172.21.15.67:53856) 2023-10-14T10:37:12.019 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.013+0000 7f92ddd9b700 1 -- 172.21.15.67:0/2975266444 learned_addr learned my addr 172.21.15.67:0/2975266444 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:37:12.019 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.013+0000 7f92de59c700 1 --2- 172.21.15.67:0/2975266444 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f92e00cc3f0 0x7f92e00cc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:12.019 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.013+0000 7f92dd59a700 1 -- 172.21.15.67:0/2975266444 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2796036913 0 0) 0x7f92e0071ab0 con 0x7f92e0161670 2023-10-14T10:37:12.020 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.013+0000 7f92dd59a700 1 -- 172.21.15.67:0/2975266444 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92c80035b0 con 0x7f92e0161670 2023-10-14T10:37:12.020 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.013+0000 7f92dd59a700 1 -- 172.21.15.67:0/2975266444 <== mon.1 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1468856269 0 0) 0x7f92e0161bc0 con 0x7f92e006af50 2023-10-14T10:37:12.020 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.013+0000 7f92dd59a700 1 -- 172.21.15.67:0/2975266444 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92e0071ab0 con 0x7f92e006af50 2023-10-14T10:37:12.021 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.013+0000 7f92de59c700 1 -- 172.21.15.67:0/2975266444 >> v1:172.21.15.72:6789/0 conn(0x7f92e006af50 legacy=0x7f92e0071f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:12.021 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.013+0000 7f92de59c700 1 -- 172.21.15.67:0/2975266444 >> v1:172.21.15.67:6789/0 conn(0x7f92e0161670 legacy=0x7f92e006a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:12.022 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.013+0000 7f92de59c700 1 -- 172.21.15.67:0/2975266444 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92e0162180 con 0x7f92e00cc3f0 2023-10-14T10:37:12.022 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.014+0000 7f92de59c700 1 --2- 172.21.15.67:0/2975266444 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f92e00cc3f0 0x7f92e00cc7d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f92d0009870 tx=0x7f92d0009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a9a57b48b811164 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:12.022 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.014+0000 7f92dd59a700 1 -- 172.21.15.67:0/2975266444 <== mon.2 v2:172.21.15.72:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d0011830 con 0x7f92e00cc3f0 2023-10-14T10:37:12.022 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.014+0000 7f92dd59a700 1 -- 172.21.15.67:0/2975266444 <== mon.2 v2:172.21.15.72:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92d0011990 con 0x7f92e00cc3f0 2023-10-14T10:37:12.023 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.014+0000 7f92dd59a700 1 -- 172.21.15.67:0/2975266444 <== mon.2 v2:172.21.15.72:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d0011d20 con 0x7f92e00cc3f0 2023-10-14T10:37:12.023 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.014+0000 7f92e4942700 1 -- 172.21.15.67:0/2975266444 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f92e00cc3f0 msgr2=0x7f92e00cc7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:12.023 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.014+0000 7f92e4942700 1 --2- 172.21.15.67:0/2975266444 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f92e00cc3f0 0x7f92e00cc7d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f92d0009870 tx=0x7f92d0009b80 comp rx=0 tx=0).stop 2023-10-14T10:37:12.024 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.014+0000 7f92e4942700 1 -- 172.21.15.67:0/2975266444 shutdown_connections 2023-10-14T10:37:12.024 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.014+0000 7f92e4942700 1 --2- 172.21.15.67:0/2975266444 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f92e00cc3f0 0x7f92e00cc7d0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:12.024 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.014+0000 7f92e4942700 1 -- 172.21.15.67:0/2975266444 >> 172.21.15.67:0/2975266444 conn(0x7f92e006dac0 msgr2=0x7f92e006dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:37:12.024 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.014+0000 7f92e4942700 1 -- 172.21.15.67:0/2975266444 shutdown_connections 2023-10-14T10:37:12.025 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.014+0000 7f92e4942700 1 -- 172.21.15.67:0/2975266444 wait complete. 2023-10-14T10:37:12.027 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.016+0000 7f92e4942700 1 Processor -- start 2023-10-14T10:37:12.028 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.016+0000 7f92e4942700 1 -- start start 2023-10-14T10:37:12.028 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.016+0000 7f92e4942700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f92e006af50 0x7f92e01f2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:12.028 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.016+0000 7f92e4942700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f92e00cc3f0 0x7f92e00ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:12.029 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.016+0000 7f92e4942700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f92e0161670 0x7f92e00c4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:12.029 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.016+0000 7f92e4942700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e0155f10 con 0x7f92e006af50 2023-10-14T10:37:12.029 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.016+0000 7f92e4942700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e0152440 con 0x7f92e0161670 2023-10-14T10:37:12.030 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.016+0000 7f92e4942700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92e00b6210 con 0x7f92e00cc3f0 2023-10-14T10:37:12.030 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.017+0000 7f92de59c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f92e006af50 0x7f92e01f2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:12.030 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.017+0000 7f92de59c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f92e006af50 0x7f92e01f2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46094/0 (socket says 172.21.15.67:46094) 2023-10-14T10:37:12.030 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.017+0000 7f92de59c700 1 -- 172.21.15.67:0/3700183570 learned_addr learned my addr 172.21.15.67:0/3700183570 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:37:12.031 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.017+0000 7f92ddd9b700 1 --2- 172.21.15.67:0/3700183570 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f92e00cc3f0 0x7f92e00ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:12.031 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.017+0000 7f92ded9d700 1 --2- 172.21.15.67:0/3700183570 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f92e0161670 0x7f92e00c4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:12.031 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.017+0000 7f92de59c700 1 -- 172.21.15.67:0/3700183570 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f92e00cc3f0 msgr2=0x7f92e00ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:12.032 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.017+0000 7f92de59c700 1 --2- 172.21.15.67:0/3700183570 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f92e00cc3f0 0x7f92e00ac8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:12.032 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.017+0000 7f92de59c700 1 -- 172.21.15.67:0/3700183570 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f92e0161670 msgr2=0x7f92e00c4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:12.032 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.017+0000 7f92de59c700 1 --2- 172.21.15.67:0/3700183570 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f92e0161670 0x7f92e00c4ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:12.032 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.017+0000 7f92de59c700 1 -- 172.21.15.67:0/3700183570 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92d00094a0 con 0x7f92e006af50 2023-10-14T10:37:12.033 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.017+0000 7f92de59c700 1 --2- 172.21.15.67:0/3700183570 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f92e006af50 0x7f92e01f2cb0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f92d0014040 tx=0x7f92d0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:12.033 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.017+0000 7f92ceffd700 1 -- 172.21.15.67:0/3700183570 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d001a860 con 0x7f92e006af50 2023-10-14T10:37:12.033 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.017+0000 7f92ceffd700 1 -- 172.21.15.67:0/3700183570 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92d001a9c0 con 0x7f92e006af50 2023-10-14T10:37:12.034 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.017+0000 7f92e4942700 1 -- 172.21.15.67:0/3700183570 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92e00c4130 con 0x7f92e006af50 2023-10-14T10:37:12.034 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.017+0000 7f92ceffd700 1 -- 172.21.15.67:0/3700183570 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d001ad30 con 0x7f92e006af50 2023-10-14T10:37:12.034 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.017+0000 7f92e4942700 1 -- 172.21.15.67:0/3700183570 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92e00c46f0 con 0x7f92e006af50 2023-10-14T10:37:12.034 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.018+0000 7f92e4942700 1 -- 172.21.15.67:0/3700183570 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92e0160390 con 0x7f92e006af50 2023-10-14T10:37:12.035 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.019+0000 7f92ceffd700 1 -- 172.21.15.67:0/3700183570 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f92d0022070 con 0x7f92e006af50 2023-10-14T10:37:12.035 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.019+0000 7f92ceffd700 1 --2- 172.21.15.67:0/3700183570 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f92c006af00 0x7f92c006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:12.035 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.019+0000 7f92ceffd700 1 -- 172.21.15.67:0/3700183570 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5729+0+0 (secure 0 0 0) 0x7f92d008a690 con 0x7f92e006af50 2023-10-14T10:37:12.036 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.019+0000 7f92ddd9b700 1 --2- 172.21.15.67:0/3700183570 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f92c006af00 0x7f92c006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:12.036 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.020+0000 7f92ddd9b700 1 --2- 172.21.15.67:0/3700183570 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f92c006af00 0x7f92c006d3c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f92e00cded0 tx=0x7f92d4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:12.036 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.024+0000 7f92ceffd700 1 -- 172.21.15.67:0/3700183570 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f92d00558d0 con 0x7f92e006af50 2023-10-14T10:37:12.225 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.223+0000 7f92e4942700 1 -- 172.21.15.67:0/3700183570 --> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f92e0162460 con 0x7f92c006af00 2023-10-14T10:37:12.226 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.223+0000 7f92ceffd700 1 -- 172.21.15.67:0/3700183570 <== mgr.4100 v2:172.21.15.72:6832/703488382 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f92d40014a0 con 0x7f92c006af00 2023-10-14T10:37:12.228 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.225+0000 7f92e4942700 1 -- 172.21.15.67:0/3700183570 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f92c006af00 msgr2=0x7f92c006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:12.229 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.225+0000 7f92e4942700 1 --2- 172.21.15.67:0/3700183570 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f92c006af00 0x7f92c006d3c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f92e00cded0 tx=0x7f92d4009040 comp rx=0 tx=0).stop 2023-10-14T10:37:12.229 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.225+0000 7f92e4942700 1 -- 172.21.15.67:0/3700183570 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f92e006af50 msgr2=0x7f92e01f2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:12.230 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.225+0000 7f92e4942700 1 --2- 172.21.15.67:0/3700183570 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f92e006af50 0x7f92e01f2cb0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f92d0014040 tx=0x7f92d0004750 comp rx=0 tx=0).stop 2023-10-14T10:37:12.230 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.225+0000 7f92e4942700 1 -- 172.21.15.67:0/3700183570 shutdown_connections 2023-10-14T10:37:12.230 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.225+0000 7f92e4942700 1 --2- 172.21.15.67:0/3700183570 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f92c006af00 0x7f92c006d3c0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:12.231 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.225+0000 7f92e4942700 1 --2- 172.21.15.67:0/3700183570 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f92e0161670 0x7f92e00c4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:12.231 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.225+0000 7f92e4942700 1 --2- 172.21.15.67:0/3700183570 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f92e006af50 0x7f92e01f2cb0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:12.231 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.225+0000 7f92e4942700 1 --2- 172.21.15.67:0/3700183570 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f92e00cc3f0 0x7f92e00ac8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:12.232 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.225+0000 7f92e4942700 1 -- 172.21.15.67:0/3700183570 >> 172.21.15.67:0/3700183570 conn(0x7f92e006dac0 msgr2=0x7f92e015fc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:37:12.232 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.225+0000 7f92e4942700 1 -- 172.21.15.67:0/3700183570 shutdown_connections 2023-10-14T10:37:12.232 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.225+0000 7f92e4942700 1 -- 172.21.15.67:0/3700183570 wait complete. 2023-10-14T10:37:12.233 INFO:teuthology.orchestra.run.smithi067.stderr:instructed osd(s) 2 to deep-scrub 2023-10-14T10:37:12.240 INFO:tasks.ceph:Scrubbing osd.3 2023-10-14T10:37:12.240 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-10-14T10:37:12.370 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.368+0000 7f0940b08700 1 Processor -- start 2023-10-14T10:37:12.370 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.368+0000 7f0940b08700 1 -- start start 2023-10-14T10:37:12.373 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.369+0000 7f0940b08700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f093c15a6f0 0x7f093c071210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:12.373 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.369+0000 7f0940b08700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f093c0cf0d0 0x7f093c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:12.373 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.369+0000 7f0940b08700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f093c15aad0 con 0x7f093c071750 2023-10-14T10:37:12.373 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.369+0000 7f0940b08700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f093c0cf680 con 0x7f093c15a6f0 2023-10-14T10:37:12.374 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.369+0000 7f0940b08700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f093c0cb200 con 0x7f093c0cf0d0 2023-10-14T10:37:12.374 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.369+0000 7f093a59c700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f093c15a6f0 0x7f093c071210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:12.374 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.369+0000 7f093a59c700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f093c15a6f0 0x7f093c071210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46102/0 (socket says 172.21.15.67:46102) 2023-10-14T10:37:12.375 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.369+0000 7f093a59c700 1 -- 172.21.15.67:0/1159854014 learned_addr learned my addr 172.21.15.67:0/1159854014 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:37:12.375 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.369+0000 7f093ad9d700 1 --2- 172.21.15.67:0/1159854014 >> v2:172.21.15.72:3300/0 conn(0x7f093c0cf0d0 0x7f093c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:12.375 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.369+0000 7f093a59c700 1 -- 172.21.15.67:0/1159854014 >> v2:172.21.15.72:3300/0 conn(0x7f093c0cf0d0 msgr2=0x7f093c06a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:37:12.375 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.369+0000 7f093a59c700 1 --2- 172.21.15.67:0/1159854014 >> v2:172.21.15.72:3300/0 conn(0x7f093c0cf0d0 0x7f093c06a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:12.376 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.369+0000 7f093a59c700 1 -- 172.21.15.67:0/1159854014 >> v1:172.21.15.72:6789/0 conn(0x7f093c071750 legacy=0x7f093c0ce9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:37:12.376 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.369+0000 7f093a59c700 1 -- 172.21.15.67:0/1159854014 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f093c071bb0 con 0x7f093c15a6f0 2023-10-14T10:37:12.376 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.369+0000 7f093a59c700 1 --2- 172.21.15.67:0/1159854014 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f093c15a6f0 0x7f093c071210 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f093c0b16f0 tx=0x7f092c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7120a3d46778951 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:12.377 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.370+0000 7f093959a700 1 -- 172.21.15.67:0/1159854014 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f092c004230 con 0x7f093c15a6f0 2023-10-14T10:37:12.377 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.370+0000 7f093959a700 1 -- 172.21.15.67:0/1159854014 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f092c004390 con 0x7f093c15a6f0 2023-10-14T10:37:12.377 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.370+0000 7f093959a700 1 -- 172.21.15.67:0/1159854014 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f092c0036a0 con 0x7f093c15a6f0 2023-10-14T10:37:12.377 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.370+0000 7f0940b08700 1 -- 172.21.15.67:0/1159854014 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f093c15a6f0 msgr2=0x7f093c071210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:12.378 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.370+0000 7f0940b08700 1 --2- 172.21.15.67:0/1159854014 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f093c15a6f0 0x7f093c071210 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f093c0b16f0 tx=0x7f092c009bd0 comp rx=0 tx=0).stop 2023-10-14T10:37:12.378 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.370+0000 7f0940b08700 1 -- 172.21.15.67:0/1159854014 shutdown_connections 2023-10-14T10:37:12.378 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.370+0000 7f0940b08700 1 --2- 172.21.15.67:0/1159854014 >> v2:172.21.15.72:3300/0 conn(0x7f093c0cf0d0 0x7f093c06a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:12.379 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.370+0000 7f0940b08700 1 --2- 172.21.15.67:0/1159854014 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f093c15a6f0 0x7f093c071210 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:12.379 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.370+0000 7f0940b08700 1 -- 172.21.15.67:0/1159854014 >> 172.21.15.67:0/1159854014 conn(0x7f093c0cfc10 msgr2=0x7f093c0d0020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:37:12.379 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.370+0000 7f0940b08700 1 -- 172.21.15.67:0/1159854014 shutdown_connections 2023-10-14T10:37:12.379 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.370+0000 7f0940b08700 1 -- 172.21.15.67:0/1159854014 wait complete. 2023-10-14T10:37:12.383 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.372+0000 7f0940b08700 1 Processor -- start 2023-10-14T10:37:12.383 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.372+0000 7f0940b08700 1 -- start start 2023-10-14T10:37:12.384 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.372+0000 7f0940b08700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f093c071750 0x7f093c153b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:12.384 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.372+0000 7f0940b08700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f093c0cf0d0 0x7f093c06b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:12.385 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.372+0000 7f0940b08700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f093c15a6f0 0x7f093c06b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:12.385 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.372+0000 7f0940b08700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f093c0c9b50 con 0x7f093c15a6f0 2023-10-14T10:37:12.385 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.372+0000 7f0940b08700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f093c0c8c20 con 0x7f093c0cf0d0 2023-10-14T10:37:12.385 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.372+0000 7f0940b08700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f093c0c7e60 con 0x7f093c071750 2023-10-14T10:37:12.386 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.372+0000 7f093ad9d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f093c15a6f0 0x7f093c06b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:12.386 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.372+0000 7f093ad9d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f093c15a6f0 0x7f093c06b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46112/0 (socket says 172.21.15.67:46112) 2023-10-14T10:37:12.386 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.372+0000 7f093ad9d700 1 -- 172.21.15.67:0/1845487297 learned_addr learned my addr 172.21.15.67:0/1845487297 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:37:12.387 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.373+0000 7f0939d9b700 1 --2- 172.21.15.67:0/1845487297 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f093c0cf0d0 0x7f093c06b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:12.387 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.373+0000 7f093a59c700 1 --2- 172.21.15.67:0/1845487297 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f093c071750 0x7f093c153b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:12.387 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.373+0000 7f093ad9d700 1 -- 172.21.15.67:0/1845487297 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f093c071750 msgr2=0x7f093c153b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:12.388 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.373+0000 7f093ad9d700 1 --2- 172.21.15.67:0/1845487297 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f093c071750 0x7f093c153b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:12.388 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.373+0000 7f093ad9d700 1 -- 172.21.15.67:0/1845487297 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f093c0cf0d0 msgr2=0x7f093c06b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:12.389 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.373+0000 7f093ad9d700 1 --2- 172.21.15.67:0/1845487297 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f093c0cf0d0 0x7f093c06b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:12.389 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.373+0000 7f093ad9d700 1 -- 172.21.15.67:0/1845487297 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f092c009550 con 0x7f093c15a6f0 2023-10-14T10:37:12.389 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.373+0000 7f093ad9d700 1 --2- 172.21.15.67:0/1845487297 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f093c15a6f0 0x7f093c06b940 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f0934007e40 tx=0x7f093400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:12.389 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.373+0000 7f092affd700 1 -- 172.21.15.67:0/1845487297 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f093400ed30 con 0x7f093c15a6f0 2023-10-14T10:37:12.390 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.373+0000 7f092affd700 1 -- 172.21.15.67:0/1845487297 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0934014420 con 0x7f093c15a6f0 2023-10-14T10:37:12.390 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.373+0000 7f0940b08700 1 -- 172.21.15.67:0/1845487297 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f093c0c8140 con 0x7f093c15a6f0 2023-10-14T10:37:12.390 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.373+0000 7f092affd700 1 -- 172.21.15.67:0/1845487297 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09340147d0 con 0x7f093c15a6f0 2023-10-14T10:37:12.391 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.373+0000 7f0940b08700 1 -- 172.21.15.67:0/1845487297 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f093c0c8700 con 0x7f093c15a6f0 2023-10-14T10:37:12.391 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.374+0000 7f0940b08700 1 -- 172.21.15.67:0/1845487297 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f093c0ce740 con 0x7f093c15a6f0 2023-10-14T10:37:12.391 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.375+0000 7f092affd700 1 -- 172.21.15.67:0/1845487297 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f0934014930 con 0x7f093c15a6f0 2023-10-14T10:37:12.391 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.375+0000 7f092affd700 1 --2- 172.21.15.67:0/1845487297 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f091c06af00 0x7f091c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:12.392 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.375+0000 7f092affd700 1 -- 172.21.15.67:0/1845487297 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5729+0+0 (secure 0 0 0) 0x7f093408aac0 con 0x7f093c15a6f0 2023-10-14T10:37:12.392 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.375+0000 7f092affd700 1 --2- 172.21.15.67:0/1845487297 >> [v2:172.21.15.67:6817/1848001176,v1:172.21.15.67:6819/1848001176] conn(0x7f091c073100 0x7f091c075540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:12.392 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.375+0000 7f092affd700 1 -- 172.21.15.67:0/1845487297 --> [v2:172.21.15.67:6817/1848001176,v1:172.21.15.67:6819/1848001176] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f091c075c10 con 0x7f091c073100 2023-10-14T10:37:12.393 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.375+0000 7f092affd700 1 -- 172.21.15.67:0/1845487297 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f093408af10 con 0x7f093c15a6f0 2023-10-14T10:37:12.393 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.375+0000 7f093a59c700 1 --2- 172.21.15.67:0/1845487297 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f091c06af00 0x7f091c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:12.393 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.375+0000 7f0939d9b700 1 --2- 172.21.15.67:0/1845487297 >> [v2:172.21.15.67:6817/1848001176,v1:172.21.15.67:6819/1848001176] conn(0x7f091c073100 0x7f091c075540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:12.393 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.376+0000 7f0939d9b700 1 --2- 172.21.15.67:0/1845487297 >> [v2:172.21.15.67:6817/1848001176,v1:172.21.15.67:6819/1848001176] conn(0x7f091c073100 0x7f091c075540 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:12.394 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.376+0000 7f093a59c700 1 --2- 172.21.15.67:0/1845487297 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f091c06af00 0x7f091c06d3c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f092c009130 tx=0x7f092c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:12.394 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.376+0000 7f092affd700 1 -- 172.21.15.67:0/1845487297 <== osd.3 v2:172.21.15.67:6817/1848001176 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7f091c075c10 con 0x7f091c073100 2023-10-14T10:37:12.395 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.391+0000 7f0940b08700 1 -- 172.21.15.67:0/1845487297 --> [v2:172.21.15.67:6817/1848001176,v1:172.21.15.67:6819/1848001176] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f093c0ce2b0 con 0x7f091c073100 2023-10-14T10:37:12.395 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.391+0000 7f092affd700 1 -- 172.21.15.67:0/1845487297 <== osd.3 v2:172.21.15.67:6817/1848001176 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f09300099f0 con 0x7f091c073100 2023-10-14T10:37:12.396 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2023-10-14T10:37:12.396 INFO:teuthology.orchestra.run.smithi067.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-14T10:37:12.396 INFO:teuthology.orchestra.run.smithi067.stdout:} 2023-10-14T10:37:12.397 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.392+0000 7f0940b08700 1 -- 172.21.15.67:0/1845487297 >> [v2:172.21.15.67:6817/1848001176,v1:172.21.15.67:6819/1848001176] conn(0x7f091c073100 msgr2=0x7f091c075540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:12.398 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.392+0000 7f0940b08700 1 --2- 172.21.15.67:0/1845487297 >> [v2:172.21.15.67:6817/1848001176,v1:172.21.15.67:6819/1848001176] conn(0x7f091c073100 0x7f091c075540 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:12.398 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.392+0000 7f0940b08700 1 -- 172.21.15.67:0/1845487297 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f091c06af00 msgr2=0x7f091c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:12.398 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.392+0000 7f0940b08700 1 --2- 172.21.15.67:0/1845487297 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f091c06af00 0x7f091c06d3c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f092c009130 tx=0x7f092c008030 comp rx=0 tx=0).stop 2023-10-14T10:37:12.399 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.392+0000 7f0940b08700 1 -- 172.21.15.67:0/1845487297 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f093c15a6f0 msgr2=0x7f093c06b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:12.399 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.392+0000 7f0940b08700 1 --2- 172.21.15.67:0/1845487297 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f093c15a6f0 0x7f093c06b940 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f0934007e40 tx=0x7f093400e3c0 comp rx=0 tx=0).stop 2023-10-14T10:37:12.399 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.392+0000 7f093ad9d700 1 -- 172.21.15.67:0/1845487297 reap_dead start 2023-10-14T10:37:12.399 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.392+0000 7f0940b08700 1 -- 172.21.15.67:0/1845487297 shutdown_connections 2023-10-14T10:37:12.400 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.392+0000 7f0940b08700 1 -- 172.21.15.67:0/1845487297 >> 172.21.15.67:0/1845487297 conn(0x7f093c0cfc10 msgr2=0x7f093c0b4860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:37:12.400 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.392+0000 7f0940b08700 1 -- 172.21.15.67:0/1845487297 shutdown_connections 2023-10-14T10:37:12.400 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.392+0000 7f0940b08700 1 -- 172.21.15.67:0/1845487297 wait complete. 2023-10-14T10:37:12.406 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-10-14T10:37:12.534 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.529+0000 7fcc94d40700 1 Processor -- start 2023-10-14T10:37:12.535 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.529+0000 7fcc94d40700 1 -- start start 2023-10-14T10:37:12.535 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.530+0000 7fcc94d40700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fcc900cd4b0 0x7fcc900cbc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:12.535 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.530+0000 7fcc94d40700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fcc900b2940 0x7fcc900ce840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:12.536 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.530+0000 7fcc94d40700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc900ce3b0 con 0x7fcc9015a850 2023-10-14T10:37:12.536 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.530+0000 7fcc94d40700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc900b6290 con 0x7fcc900cd4b0 2023-10-14T10:37:12.536 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.530+0000 7fcc94d40700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc900cc220 con 0x7fcc900b2940 2023-10-14T10:37:12.537 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.530+0000 7fcc8e59c700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fcc9015a850 0x7fcc900ccda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:53886/0 (socket says 172.21.15.67:53886) 2023-10-14T10:37:12.537 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.531+0000 7fcc8e59c700 1 -- 172.21.15.67:0/2669874735 learned_addr learned my addr 172.21.15.67:0/2669874735 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:37:12.537 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.531+0000 7fcc8ed9d700 1 --2- 172.21.15.67:0/2669874735 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fcc900b2940 0x7fcc900ce840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:12.537 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.531+0000 7fcc8dd9b700 1 --2- 172.21.15.67:0/2669874735 >> v2:172.21.15.72:3300/0 conn(0x7fcc900cd4b0 0x7fcc900cbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:12.538 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.531+0000 7fcc8d59a700 1 -- 172.21.15.67:0/2669874735 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2355374680 0 0) 0x7fcc900ce3b0 con 0x7fcc9015a850 2023-10-14T10:37:12.538 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.531+0000 7fcc8d59a700 1 -- 172.21.15.67:0/2669874735 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc780035b0 con 0x7fcc9015a850 2023-10-14T10:37:12.538 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.531+0000 7fcc8ed9d700 1 -- 172.21.15.67:0/2669874735 >> v2:172.21.15.72:3300/0 conn(0x7fcc900cd4b0 msgr2=0x7fcc900cbc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:37:12.539 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.531+0000 7fcc8ed9d700 1 --2- 172.21.15.67:0/2669874735 >> v2:172.21.15.72:3300/0 conn(0x7fcc900cd4b0 0x7fcc900cbc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:12.539 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.531+0000 7fcc8ed9d700 1 -- 172.21.15.67:0/2669874735 >> v1:172.21.15.67:6789/0 conn(0x7fcc9015a850 legacy=0x7fcc900ccda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:12.539 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.531+0000 7fcc8ed9d700 1 -- 172.21.15.67:0/2669874735 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc900cc500 con 0x7fcc900b2940 2023-10-14T10:37:12.539 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.531+0000 7fcc8ed9d700 1 --2- 172.21.15.67:0/2669874735 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fcc900b2940 0x7fcc900ce840 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fcc88009d90 tx=0x7fcc8800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8be713232f98cc2a server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:12.540 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.531+0000 7fcc8d59a700 1 -- 172.21.15.67:0/2669874735 <== mon.2 v2:172.21.15.72:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc8800e070 con 0x7fcc900b2940 2023-10-14T10:37:12.540 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.531+0000 7fcc8d59a700 1 -- 172.21.15.67:0/2669874735 <== mon.2 v2:172.21.15.72:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc8800bb20 con 0x7fcc900b2940 2023-10-14T10:37:12.540 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.531+0000 7fcc8d59a700 1 -- 172.21.15.67:0/2669874735 <== mon.2 v2:172.21.15.72:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc88012430 con 0x7fcc900b2940 2023-10-14T10:37:12.540 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.532+0000 7fcc94d40700 1 -- 172.21.15.67:0/2669874735 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fcc900b2940 msgr2=0x7fcc900ce840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:12.541 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.532+0000 7fcc94d40700 1 --2- 172.21.15.67:0/2669874735 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fcc900b2940 0x7fcc900ce840 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fcc88009d90 tx=0x7fcc8800b330 comp rx=0 tx=0).stop 2023-10-14T10:37:12.541 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.532+0000 7fcc94d40700 1 -- 172.21.15.67:0/2669874735 shutdown_connections 2023-10-14T10:37:12.541 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.532+0000 7fcc94d40700 1 --2- 172.21.15.67:0/2669874735 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fcc900b2940 0x7fcc900ce840 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:12.542 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.532+0000 7fcc94d40700 1 --2- 172.21.15.67:0/2669874735 >> v2:172.21.15.72:3300/0 conn(0x7fcc900cd4b0 0x7fcc900cbc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:12.542 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.532+0000 7fcc94d40700 1 -- 172.21.15.67:0/2669874735 >> 172.21.15.67:0/2669874735 conn(0x7fcc9006dac0 msgr2=0x7fcc9006dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:37:12.542 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.532+0000 7fcc94d40700 1 -- 172.21.15.67:0/2669874735 shutdown_connections 2023-10-14T10:37:12.542 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.532+0000 7fcc94d40700 1 -- 172.21.15.67:0/2669874735 wait complete. 2023-10-14T10:37:12.545 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.533+0000 7fcc94d40700 1 Processor -- start 2023-10-14T10:37:12.546 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.533+0000 7fcc94d40700 1 -- start start 2023-10-14T10:37:12.546 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.534+0000 7fcc94d40700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc900b2940 0x7fcc900c9e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:12.546 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.534+0000 7fcc94d40700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc900cd4b0 0x7fcc901e3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:12.546 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.534+0000 7fcc94d40700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fcc9015a850 0x7fcc9006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:12.547 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.534+0000 7fcc94d40700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc90146800 con 0x7fcc900b2940 2023-10-14T10:37:12.547 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.534+0000 7fcc94d40700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc900ca3d0 con 0x7fcc900cd4b0 2023-10-14T10:37:12.547 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.534+0000 7fcc94d40700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc901e4450 con 0x7fcc9015a850 2023-10-14T10:37:12.548 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.534+0000 7fcc8e59c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc900b2940 0x7fcc900c9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:12.548 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.534+0000 7fcc8e59c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc900b2940 0x7fcc900c9e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46124/0 (socket says 172.21.15.67:46124) 2023-10-14T10:37:12.548 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.534+0000 7fcc8e59c700 1 -- 172.21.15.67:0/4170537315 learned_addr learned my addr 172.21.15.67:0/4170537315 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:37:12.548 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.535+0000 7fcc8ed9d700 1 --2- 172.21.15.67:0/4170537315 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fcc9015a850 0x7fcc9006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:12.549 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.535+0000 7fcc8dd9b700 1 --2- 172.21.15.67:0/4170537315 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc900cd4b0 0x7fcc901e3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:12.549 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.535+0000 7fcc8e59c700 1 -- 172.21.15.67:0/4170537315 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fcc9015a850 msgr2=0x7fcc9006a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:12.549 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.535+0000 7fcc8e59c700 1 --2- 172.21.15.67:0/4170537315 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fcc9015a850 0x7fcc9006a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:12.549 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.535+0000 7fcc8e59c700 1 -- 172.21.15.67:0/4170537315 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc900cd4b0 msgr2=0x7fcc901e3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:12.550 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.535+0000 7fcc8e59c700 1 --2- 172.21.15.67:0/4170537315 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc900cd4b0 0x7fcc901e3a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:12.550 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.535+0000 7fcc8e59c700 1 -- 172.21.15.67:0/4170537315 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc88009a70 con 0x7fcc900b2940 2023-10-14T10:37:12.550 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.535+0000 7fcc8e59c700 1 --2- 172.21.15.67:0/4170537315 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc900b2940 0x7fcc900c9e90 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fcc84001eb0 tx=0x7fcc8400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:12.551 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.535+0000 7fcc7effd700 1 -- 172.21.15.67:0/4170537315 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc84005d70 con 0x7fcc900b2940 2023-10-14T10:37:12.551 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.535+0000 7fcc7effd700 1 -- 172.21.15.67:0/4170537315 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc840070d0 con 0x7fcc900b2940 2023-10-14T10:37:12.551 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.535+0000 7fcc94d40700 1 -- 172.21.15.67:0/4170537315 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc901e4730 con 0x7fcc900b2940 2023-10-14T10:37:12.551 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.535+0000 7fcc7effd700 1 -- 172.21.15.67:0/4170537315 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc84007460 con 0x7fcc900b2940 2023-10-14T10:37:12.552 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.535+0000 7fcc94d40700 1 -- 172.21.15.67:0/4170537315 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc90146420 con 0x7fcc900b2940 2023-10-14T10:37:12.552 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.536+0000 7fcc94d40700 1 -- 172.21.15.67:0/4170537315 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc90146940 con 0x7fcc900b2940 2023-10-14T10:37:12.552 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.536+0000 7fcc7effd700 1 -- 172.21.15.67:0/4170537315 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fcc840075c0 con 0x7fcc900b2940 2023-10-14T10:37:12.553 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.537+0000 7fcc7effd700 1 --2- 172.21.15.67:0/4170537315 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fcc7006af00 0x7fcc7006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:12.553 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.537+0000 7fcc7effd700 1 -- 172.21.15.67:0/4170537315 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5729+0+0 (secure 0 0 0) 0x7fcc84088a80 con 0x7fcc900b2940 2023-10-14T10:37:12.553 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.537+0000 7fcc8dd9b700 1 --2- 172.21.15.67:0/4170537315 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fcc7006af00 0x7fcc7006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:12.553 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.538+0000 7fcc8dd9b700 1 --2- 172.21.15.67:0/4170537315 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fcc7006af00 0x7fcc7006d3c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fcc900cc010 tx=0x7fcc80008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:12.554 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.541+0000 7fcc7effd700 1 -- 172.21.15.67:0/4170537315 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcc90146940 con 0x7fcc900b2940 2023-10-14T10:37:12.744 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.742+0000 7fcc94d40700 1 -- 172.21.15.67:0/4170537315 --> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7fcc900ccac0 con 0x7fcc7006af00 2023-10-14T10:37:12.745 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.743+0000 7fcc7effd700 1 -- 172.21.15.67:0/4170537315 <== mgr.4100 v2:172.21.15.72:6832/703488382 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fcc80009660 con 0x7fcc7006af00 2023-10-14T10:37:12.747 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.745+0000 7fcc94d40700 1 -- 172.21.15.67:0/4170537315 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fcc7006af00 msgr2=0x7fcc7006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:12.748 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.745+0000 7fcc94d40700 1 --2- 172.21.15.67:0/4170537315 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fcc7006af00 0x7fcc7006d3c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fcc900cc010 tx=0x7fcc80008040 comp rx=0 tx=0).stop 2023-10-14T10:37:12.748 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.745+0000 7fcc94d40700 1 -- 172.21.15.67:0/4170537315 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc900b2940 msgr2=0x7fcc900c9e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:12.748 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.745+0000 7fcc94d40700 1 --2- 172.21.15.67:0/4170537315 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc900b2940 0x7fcc900c9e90 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fcc84001eb0 tx=0x7fcc8400c860 comp rx=0 tx=0).stop 2023-10-14T10:37:12.749 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.745+0000 7fcc94d40700 1 -- 172.21.15.67:0/4170537315 shutdown_connections 2023-10-14T10:37:12.749 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.745+0000 7fcc94d40700 1 --2- 172.21.15.67:0/4170537315 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fcc7006af00 0x7fcc7006d3c0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:12.749 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.745+0000 7fcc94d40700 1 --2- 172.21.15.67:0/4170537315 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fcc9015a850 0x7fcc9006a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:12.750 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.745+0000 7fcc94d40700 1 --2- 172.21.15.67:0/4170537315 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc900b2940 0x7fcc900c9e90 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:12.750 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.745+0000 7fcc94d40700 1 --2- 172.21.15.67:0/4170537315 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc900cd4b0 0x7fcc901e3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:12.750 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.745+0000 7fcc94d40700 1 -- 172.21.15.67:0/4170537315 >> 172.21.15.67:0/4170537315 conn(0x7fcc9006dac0 msgr2=0x7fcc900ccd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:37:12.750 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.745+0000 7fcc94d40700 1 -- 172.21.15.67:0/4170537315 shutdown_connections 2023-10-14T10:37:12.751 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.745+0000 7fcc94d40700 1 -- 172.21.15.67:0/4170537315 wait complete. 2023-10-14T10:37:12.751 INFO:teuthology.orchestra.run.smithi067.stderr:instructed osd(s) 3 to deep-scrub 2023-10-14T10:37:12.759 INFO:tasks.ceph:Scrubbing osd.4 2023-10-14T10:37:12.760 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-10-14T10:37:12.889 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.887+0000 7fb1c052d700 1 Processor -- start 2023-10-14T10:37:12.890 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.887+0000 7fb1c052d700 1 -- start start 2023-10-14T10:37:12.893 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.888+0000 7fb1c052d700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fb1b8071920 0x7fb1b80ce9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:12.893 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.888+0000 7fb1c052d700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1b815aad0 con 0x7fb1b815a6f0 2023-10-14T10:37:12.894 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.888+0000 7fb1c052d700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1b80ce2b0 con 0x7fb1b80cef00 2023-10-14T10:37:12.894 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.888+0000 7fb1c052d700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b80cf680 con 0x7fb1b8071920 2023-10-14T10:37:12.894 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.888+0000 7fb1be2c9700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fb1b815a6f0 0x7fb1b8071210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:53900/0 (socket says 172.21.15.67:53900) 2023-10-14T10:37:12.894 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.888+0000 7fb1be2c9700 1 -- 172.21.15.67:0/1279416870 learned_addr learned my addr 172.21.15.67:0/1279416870 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:37:12.895 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.888+0000 7fb1bdac8700 1 --2- 172.21.15.67:0/1279416870 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fb1b8071920 0x7fb1b80ce9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:12.895 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.888+0000 7fb1bd2c7700 1 -- 172.21.15.67:0/1279416870 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1976567459 0 0) 0x7fb1b815aad0 con 0x7fb1b815a6f0 2023-10-14T10:37:12.895 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.888+0000 7fb1bd2c7700 1 -- 172.21.15.67:0/1279416870 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1a40057c0 con 0x7fb1b815a6f0 2023-10-14T10:37:12.896 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.889+0000 7fb1bd2c7700 1 -- 172.21.15.67:0/1279416870 <== mon.1 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1200632516 0 0) 0x7fb1b80ce2b0 con 0x7fb1b80cef00 2023-10-14T10:37:12.896 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.889+0000 7fb1bd2c7700 1 -- 172.21.15.67:0/1279416870 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1b815aad0 con 0x7fb1b80cef00 2023-10-14T10:37:12.896 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.889+0000 7fb1bdac8700 1 -- 172.21.15.67:0/1279416870 >> v1:172.21.15.72:6789/0 conn(0x7fb1b80cef00 legacy=0x7fb1b806a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:12.896 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.889+0000 7fb1bdac8700 1 -- 172.21.15.67:0/1279416870 >> v1:172.21.15.67:6789/0 conn(0x7fb1b815a6f0 legacy=0x7fb1b8071210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:12.897 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.889+0000 7fb1bdac8700 1 -- 172.21.15.67:0/1279416870 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1b80ce740 con 0x7fb1b8071920 2023-10-14T10:37:12.897 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.889+0000 7fb1bdac8700 1 --2- 172.21.15.67:0/1279416870 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fb1b8071920 0x7fb1b80ce9c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb1b4009870 tx=0x7fb1b4009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f9377dff6faa591 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:12.897 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.889+0000 7fb1bd2c7700 1 -- 172.21.15.67:0/1279416870 <== mon.2 v2:172.21.15.72:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b4011830 con 0x7fb1b8071920 2023-10-14T10:37:12.898 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.889+0000 7fb1bd2c7700 1 -- 172.21.15.67:0/1279416870 <== mon.2 v2:172.21.15.72:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1b4011990 con 0x7fb1b8071920 2023-10-14T10:37:12.898 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.889+0000 7fb1bd2c7700 1 -- 172.21.15.67:0/1279416870 <== mon.2 v2:172.21.15.72:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b4011d20 con 0x7fb1b8071920 2023-10-14T10:37:12.898 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.889+0000 7fb1c052d700 1 -- 172.21.15.67:0/1279416870 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fb1b8071920 msgr2=0x7fb1b80ce9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:12.899 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.889+0000 7fb1c052d700 1 --2- 172.21.15.67:0/1279416870 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fb1b8071920 0x7fb1b80ce9c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb1b4009870 tx=0x7fb1b4009b80 comp rx=0 tx=0).stop 2023-10-14T10:37:12.899 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.890+0000 7fb1c052d700 1 -- 172.21.15.67:0/1279416870 shutdown_connections 2023-10-14T10:37:12.899 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.890+0000 7fb1c052d700 1 --2- 172.21.15.67:0/1279416870 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fb1b8071920 0x7fb1b80ce9c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:12.899 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.890+0000 7fb1c052d700 1 -- 172.21.15.67:0/1279416870 >> 172.21.15.67:0/1279416870 conn(0x7fb1b80cfc10 msgr2=0x7fb1b80d0020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:37:12.900 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.890+0000 7fb1c052d700 1 -- 172.21.15.67:0/1279416870 shutdown_connections 2023-10-14T10:37:12.900 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.890+0000 7fb1c052d700 1 -- 172.21.15.67:0/1279416870 wait complete. 2023-10-14T10:37:12.900 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.891+0000 7fb1c052d700 1 Processor -- start 2023-10-14T10:37:12.900 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.891+0000 7fb1c052d700 1 -- start start 2023-10-14T10:37:12.904 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.892+0000 7fb1c052d700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fb1b8071920 0x7fb1b80a9240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:12.904 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.892+0000 7fb1c052d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb1b80cef00 0x7fb1b80c8900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:12.904 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.892+0000 7fb1c052d700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb1b815a6f0 0x7fb1b80aba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:12.904 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.892+0000 7fb1c052d700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b8159670 con 0x7fb1b80cef00 2023-10-14T10:37:12.905 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.892+0000 7fb1c052d700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b80c9830 con 0x7fb1b815a6f0 2023-10-14T10:37:12.905 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.892+0000 7fb1c052d700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b80c8f90 con 0x7fb1b8071920 2023-10-14T10:37:12.905 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.892+0000 7fb1bdac8700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb1b80cef00 0x7fb1b80c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:12.906 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.892+0000 7fb1bdac8700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb1b80cef00 0x7fb1b80c8900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46140/0 (socket says 172.21.15.67:46140) 2023-10-14T10:37:12.906 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.892+0000 7fb1be2c9700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fb1b8071920 0x7fb1b80a9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:12.906 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.892+0000 7fb1bdac8700 1 -- 172.21.15.67:0/3191923585 learned_addr learned my addr 172.21.15.67:0/3191923585 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:37:12.906 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.892+0000 7fb1beaca700 1 --2- 172.21.15.67:0/3191923585 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb1b815a6f0 0x7fb1b80aba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:12.907 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.892+0000 7fb1bdac8700 1 -- 172.21.15.67:0/3191923585 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fb1b8071920 msgr2=0x7fb1b80a9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:12.907 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.892+0000 7fb1bdac8700 1 --2- 172.21.15.67:0/3191923585 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fb1b8071920 0x7fb1b80a9240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:12.907 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.892+0000 7fb1bdac8700 1 -- 172.21.15.67:0/3191923585 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb1b815a6f0 msgr2=0x7fb1b80aba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:12.908 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.892+0000 7fb1bdac8700 1 --2- 172.21.15.67:0/3191923585 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb1b815a6f0 0x7fb1b80aba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:12.908 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.892+0000 7fb1bdac8700 1 -- 172.21.15.67:0/3191923585 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1b40094a0 con 0x7fb1b80cef00 2023-10-14T10:37:12.908 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.892+0000 7fb1bdac8700 1 --2- 172.21.15.67:0/3191923585 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb1b80cef00 0x7fb1b80c8900 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fb1b4009bc0 tx=0x7fb1b4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:12.908 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.892+0000 7fb1aeffd700 1 -- 172.21.15.67:0/3191923585 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b401a6e0 con 0x7fb1b80cef00 2023-10-14T10:37:12.909 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.892+0000 7fb1aeffd700 1 -- 172.21.15.67:0/3191923585 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1b401a840 con 0x7fb1b80cef00 2023-10-14T10:37:12.909 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.892+0000 7fb1c052d700 1 -- 172.21.15.67:0/3191923585 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1b80c9270 con 0x7fb1b80cef00 2023-10-14T10:37:12.909 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.892+0000 7fb1aeffd700 1 -- 172.21.15.67:0/3191923585 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b401abf0 con 0x7fb1b80cef00 2023-10-14T10:37:12.910 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.893+0000 7fb1c052d700 1 -- 172.21.15.67:0/3191923585 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1b8148e60 con 0x7fb1b80cef00 2023-10-14T10:37:12.910 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.893+0000 7fb1c052d700 1 -- 172.21.15.67:0/3191923585 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb1b80cdc60 con 0x7fb1b80cef00 2023-10-14T10:37:12.910 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.894+0000 7fb1aeffd700 1 -- 172.21.15.67:0/3191923585 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fb1b4022070 con 0x7fb1b80cef00 2023-10-14T10:37:12.910 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.894+0000 7fb1aeffd700 1 --2- 172.21.15.67:0/3191923585 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fb19c06af00 0x7fb19c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:12.911 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.894+0000 7fb1aeffd700 1 -- 172.21.15.67:0/3191923585 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5729+0+0 (secure 0 0 0) 0x7fb1b408a530 con 0x7fb1b80cef00 2023-10-14T10:37:12.911 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.894+0000 7fb1aeffd700 1 --2- 172.21.15.67:0/3191923585 >> [v2:172.21.15.72:6802/1891230785,v1:172.21.15.72:6804/1891230785] conn(0x7fb19c073100 0x7fb19c075540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:12.911 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.895+0000 7fb1aeffd700 1 -- 172.21.15.67:0/3191923585 --> [v2:172.21.15.72:6802/1891230785,v1:172.21.15.72:6804/1891230785] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb19c075c10 con 0x7fb19c073100 2023-10-14T10:37:12.912 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.895+0000 7fb1aeffd700 1 -- 172.21.15.67:0/3191923585 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb1b408cbf0 con 0x7fb1b80cef00 2023-10-14T10:37:12.912 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.895+0000 7fb1be2c9700 1 --2- 172.21.15.67:0/3191923585 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fb19c06af00 0x7fb19c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:12.912 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.895+0000 7fb1beaca700 1 --2- 172.21.15.67:0/3191923585 >> [v2:172.21.15.72:6802/1891230785,v1:172.21.15.72:6804/1891230785] conn(0x7fb19c073100 0x7fb19c075540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:12.913 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.895+0000 7fb1be2c9700 1 --2- 172.21.15.67:0/3191923585 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fb19c06af00 0x7fb19c06d3c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb1a8007fd0 tx=0x7fb1a8007e00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:12.913 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.895+0000 7fb1beaca700 1 --2- 172.21.15.67:0/3191923585 >> [v2:172.21.15.72:6802/1891230785,v1:172.21.15.72:6804/1891230785] conn(0x7fb19c073100 0x7fb19c075540 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:12.913 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.896+0000 7fb1aeffd700 1 -- 172.21.15.67:0/3191923585 <== osd.4 v2:172.21.15.72:6802/1891230785 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7fb19c075c10 con 0x7fb19c073100 2023-10-14T10:37:12.914 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.911+0000 7fb1c052d700 1 -- 172.21.15.67:0/3191923585 --> [v2:172.21.15.72:6802/1891230785,v1:172.21.15.72:6804/1891230785] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fb1b80cd980 con 0x7fb19c073100 2023-10-14T10:37:12.914 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.911+0000 7fb1aeffd700 1 -- 172.21.15.67:0/3191923585 <== osd.4 v2:172.21.15.72:6802/1891230785 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fb1b0001fe0 con 0x7fb19c073100 2023-10-14T10:37:12.915 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2023-10-14T10:37:12.915 INFO:teuthology.orchestra.run.smithi067.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-14T10:37:12.915 INFO:teuthology.orchestra.run.smithi067.stdout:} 2023-10-14T10:37:12.916 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.911+0000 7fb1c052d700 1 -- 172.21.15.67:0/3191923585 >> [v2:172.21.15.72:6802/1891230785,v1:172.21.15.72:6804/1891230785] conn(0x7fb19c073100 msgr2=0x7fb19c075540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:12.917 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.911+0000 7fb1c052d700 1 --2- 172.21.15.67:0/3191923585 >> [v2:172.21.15.72:6802/1891230785,v1:172.21.15.72:6804/1891230785] conn(0x7fb19c073100 0x7fb19c075540 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:12.917 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.912+0000 7fb1c052d700 1 -- 172.21.15.67:0/3191923585 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fb19c06af00 msgr2=0x7fb19c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:12.917 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.912+0000 7fb1c052d700 1 --2- 172.21.15.67:0/3191923585 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fb19c06af00 0x7fb19c06d3c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb1a8007fd0 tx=0x7fb1a8007e00 comp rx=0 tx=0).stop 2023-10-14T10:37:12.918 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.912+0000 7fb1c052d700 1 -- 172.21.15.67:0/3191923585 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb1b80cef00 msgr2=0x7fb1b80c8900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:12.918 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.912+0000 7fb1c052d700 1 --2- 172.21.15.67:0/3191923585 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb1b80cef00 0x7fb1b80c8900 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fb1b4009bc0 tx=0x7fb1b4004750 comp rx=0 tx=0).stop 2023-10-14T10:37:12.918 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.912+0000 7fb1beaca700 1 -- 172.21.15.67:0/3191923585 reap_dead start 2023-10-14T10:37:12.919 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.912+0000 7fb1c052d700 1 -- 172.21.15.67:0/3191923585 shutdown_connections 2023-10-14T10:37:12.919 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.912+0000 7fb1c052d700 1 -- 172.21.15.67:0/3191923585 >> 172.21.15.67:0/3191923585 conn(0x7fb1b80cfc10 msgr2=0x7fb1b815bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:37:12.919 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.912+0000 7fb1c052d700 1 -- 172.21.15.67:0/3191923585 shutdown_connections 2023-10-14T10:37:12.919 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:12.912+0000 7fb1c052d700 1 -- 172.21.15.67:0/3191923585 wait complete. 2023-10-14T10:37:12.925 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-10-14T10:37:13.052 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.050+0000 7f49da5a2700 1 Processor -- start 2023-10-14T10:37:13.053 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.050+0000 7f49da5a2700 1 -- start start 2023-10-14T10:37:13.056 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.051+0000 7f49da5a2700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f49d415dce0 0x7f49d40b03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:13.057 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.051+0000 7f49da5a2700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f49d40cfc00 0x7f49d40cc090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:13.057 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.051+0000 7f49da5a2700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49d406cb70 con 0x7f49d415e0c0 2023-10-14T10:37:13.057 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.051+0000 7f49da5a2700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49d40cbc40 con 0x7f49d40cfc00 2023-10-14T10:37:13.058 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.051+0000 7f49da5a2700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49d406ce50 con 0x7f49d415dce0 2023-10-14T10:37:13.058 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.051+0000 7f49d37fe700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f49d415e0c0 0x7f49d40cf4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:53918/0 (socket says 172.21.15.67:53918) 2023-10-14T10:37:13.058 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.051+0000 7f49d37fe700 1 -- 172.21.15.67:0/3969761458 learned_addr learned my addr 172.21.15.67:0/3969761458 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:37:13.058 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.051+0000 7f49d8b3f700 1 --2- 172.21.15.67:0/3969761458 >> v2:172.21.15.72:3300/0 conn(0x7f49d40cfc00 0x7f49d40cc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:13.059 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.051+0000 7f49d3fff700 1 --2- 172.21.15.67:0/3969761458 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f49d415dce0 0x7f49d40b03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:13.059 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.051+0000 7f49d2ffd700 1 -- 172.21.15.67:0/3969761458 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2526257031 0 0) 0x7f49d406cb70 con 0x7f49d415e0c0 2023-10-14T10:37:13.059 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.051+0000 7f49d2ffd700 1 -- 172.21.15.67:0/3969761458 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49bc0035b0 con 0x7f49d415e0c0 2023-10-14T10:37:13.060 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.052+0000 7f49d2ffd700 1 -- 172.21.15.67:0/3969761458 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1157970651 0 0) 0x7f49bc0035b0 con 0x7f49d415e0c0 2023-10-14T10:37:13.060 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.052+0000 7f49d2ffd700 1 -- 172.21.15.67:0/3969761458 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f49d415dce0 msgr2=0x7f49d40b03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:37:13.060 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.052+0000 7f49d2ffd700 1 --2- 172.21.15.67:0/3969761458 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f49d415dce0 0x7f49d40b03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:13.060 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.052+0000 7f49d2ffd700 1 -- 172.21.15.67:0/3969761458 >> v2:172.21.15.72:3300/0 conn(0x7f49d40cfc00 msgr2=0x7f49d40cc090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:37:13.061 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.052+0000 7f49d2ffd700 1 --2- 172.21.15.67:0/3969761458 >> v2:172.21.15.72:3300/0 conn(0x7f49d40cfc00 0x7f49d40cc090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:13.061 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.052+0000 7f49d2ffd700 1 -- 172.21.15.67:0/3969761458 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49d406d130 con 0x7f49d415e0c0 2023-10-14T10:37:13.061 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.052+0000 7f49d2ffd700 1 -- 172.21.15.67:0/3969761458 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3180043755 0 0) 0x7f49c4001670 con 0x7f49d415e0c0 2023-10-14T10:37:13.061 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.052+0000 7f49d8b3f700 1 --2- 172.21.15.67:0/3969761458 >> v2:172.21.15.72:3300/0 conn(0x7f49d40cfc00 0x7f49d40cc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-14T10:37:13.062 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.052+0000 7f49d2ffd700 1 -- 172.21.15.67:0/3969761458 >> v1:172.21.15.67:6789/0 conn(0x7f49d415e0c0 legacy=0x7f49d40cf4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:13.062 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.052+0000 7f49d2ffd700 1 --2- 172.21.15.67:0/3969761458 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f49bc003a00 0x7f49bc003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:13.062 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.052+0000 7f49d2ffd700 1 --2- 172.21.15.67:0/3969761458 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f49d415dce0 0x7f49bc004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:13.063 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.052+0000 7f49d2ffd700 1 --2- 172.21.15.67:0/3969761458 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f49bc004d90 0x7f49bc005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:13.063 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.052+0000 7f49d2ffd700 1 -- 172.21.15.67:0/3969761458 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49d40cbc40 con 0x7f49bc004d90 2023-10-14T10:37:13.063 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.052+0000 7f49d2ffd700 1 -- 172.21.15.67:0/3969761458 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49d406ce50 con 0x7f49bc003a00 2023-10-14T10:37:13.063 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.052+0000 7f49d2ffd700 1 -- 172.21.15.67:0/3969761458 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49bc00ba10 con 0x7f49d415dce0 2023-10-14T10:37:13.064 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.052+0000 7f49d8b3f700 1 --2- 172.21.15.67:0/3969761458 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f49bc004d90 0x7f49bc005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:13.064 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.052+0000 7f49d3fff700 1 --2- 172.21.15.67:0/3969761458 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f49bc003a00 0x7f49bc003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:13.064 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.052+0000 7f49d37fe700 1 --2- 172.21.15.67:0/3969761458 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f49d415dce0 0x7f49bc004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:13.064 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.052+0000 7f49d8b3f700 1 -- 172.21.15.67:0/3969761458 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f49d415dce0 msgr2=0x7f49bc004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:37:13.065 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.052+0000 7f49d8b3f700 1 --2- 172.21.15.67:0/3969761458 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f49d415dce0 0x7f49bc004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:13.065 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.052+0000 7f49d8b3f700 1 -- 172.21.15.67:0/3969761458 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f49bc003a00 msgr2=0x7f49bc003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:37:13.065 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.052+0000 7f49d8b3f700 1 --2- 172.21.15.67:0/3969761458 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f49bc003a00 0x7f49bc003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:13.065 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.052+0000 7f49d8b3f700 1 -- 172.21.15.67:0/3969761458 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49bc0035b0 con 0x7f49bc004d90 2023-10-14T10:37:13.066 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.052+0000 7f49d8b3f700 1 --2- 172.21.15.67:0/3969761458 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f49bc004d90 0x7f49bc005210 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f49cc00e350 tx=0x7f49bc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d8b00604b602f69 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:13.066 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.052+0000 7f49d2ffd700 1 -- 172.21.15.67:0/3969761458 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49cc00edf0 con 0x7f49bc004d90 2023-10-14T10:37:13.066 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.052+0000 7f49d2ffd700 1 -- 172.21.15.67:0/3969761458 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49cc00f040 con 0x7f49bc004d90 2023-10-14T10:37:13.066 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.052+0000 7f49d2ffd700 1 -- 172.21.15.67:0/3969761458 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49cc0146a0 con 0x7f49bc004d90 2023-10-14T10:37:13.067 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.053+0000 7f49da5a2700 1 -- 172.21.15.67:0/3969761458 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f49bc004d90 msgr2=0x7f49bc005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:13.067 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.053+0000 7f49da5a2700 1 --2- 172.21.15.67:0/3969761458 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f49bc004d90 0x7f49bc005210 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f49cc00e350 tx=0x7f49bc005880 comp rx=0 tx=0).stop 2023-10-14T10:37:13.067 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.053+0000 7f49d8b3f700 1 -- 172.21.15.67:0/3969761458 reap_dead start 2023-10-14T10:37:13.068 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.053+0000 7f49da5a2700 1 -- 172.21.15.67:0/3969761458 shutdown_connections 2023-10-14T10:37:13.068 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.053+0000 7f49da5a2700 1 -- 172.21.15.67:0/3969761458 >> 172.21.15.67:0/3969761458 conn(0x7f49d40d15f0 msgr2=0x7f49d40d19d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:37:13.068 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.053+0000 7f49da5a2700 1 -- 172.21.15.67:0/3969761458 shutdown_connections 2023-10-14T10:37:13.068 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.053+0000 7f49da5a2700 1 -- 172.21.15.67:0/3969761458 wait complete. 2023-10-14T10:37:13.069 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.054+0000 7f49da5a2700 1 Processor -- start 2023-10-14T10:37:13.069 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.055+0000 7f49da5a2700 1 -- start start 2023-10-14T10:37:13.072 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.055+0000 7f49da5a2700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f49d415add0 0x7f49d40b3f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:13.072 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.055+0000 7f49da5a2700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f49d415b1b0 0x7f49d409e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:13.072 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.055+0000 7f49da5a2700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f49d409e9d0 0x7f49d406ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:13.073 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.055+0000 7f49da5a2700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49d409b390 con 0x7f49d415add0 2023-10-14T10:37:13.073 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.055+0000 7f49da5a2700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49d409b5e0 con 0x7f49d409e9d0 2023-10-14T10:37:13.073 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.055+0000 7f49da5a2700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49d409b8c0 con 0x7f49d415b1b0 2023-10-14T10:37:13.074 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.055+0000 7f49d3fff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f49d415add0 0x7f49d40b3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:13.074 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.055+0000 7f49d3fff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f49d415add0 0x7f49d40b3f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46160/0 (socket says 172.21.15.67:46160) 2023-10-14T10:37:13.074 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.055+0000 7f49d3fff700 1 -- 172.21.15.67:0/1373722182 learned_addr learned my addr 172.21.15.67:0/1373722182 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:37:13.075 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.055+0000 7f49d8b3f700 1 --2- 172.21.15.67:0/1373722182 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f49d409e9d0 0x7f49d406ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:13.075 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.056+0000 7f49d37fe700 1 --2- 172.21.15.67:0/1373722182 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f49d415b1b0 0x7f49d409e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:13.075 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.056+0000 7f49d3fff700 1 -- 172.21.15.67:0/1373722182 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f49d415b1b0 msgr2=0x7f49d409e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:13.075 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.056+0000 7f49d3fff700 1 --2- 172.21.15.67:0/1373722182 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f49d415b1b0 0x7f49d409e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:13.076 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.056+0000 7f49d3fff700 1 -- 172.21.15.67:0/1373722182 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f49d409e9d0 msgr2=0x7f49d406ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:13.076 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.056+0000 7f49d3fff700 1 --2- 172.21.15.67:0/1373722182 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f49d409e9d0 0x7f49d406ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:13.076 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.056+0000 7f49d3fff700 1 -- 172.21.15.67:0/1373722182 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49d409bba0 con 0x7f49d415add0 2023-10-14T10:37:13.077 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.056+0000 7f49d8b3f700 1 --2- 172.21.15.67:0/1373722182 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f49d409e9d0 0x7f49d406ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-14T10:37:13.077 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.056+0000 7f49d3fff700 1 --2- 172.21.15.67:0/1373722182 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f49d415add0 0x7f49d40b3f00 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f49c80119a0 tx=0x7f49c8011d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:13.077 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.056+0000 7f49d0ff9700 1 -- 172.21.15.67:0/1373722182 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c800f940 con 0x7f49d415add0 2023-10-14T10:37:13.077 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.056+0000 7f49d0ff9700 1 -- 172.21.15.67:0/1373722182 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49c800faa0 con 0x7f49d415add0 2023-10-14T10:37:13.078 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.056+0000 7f49d0ff9700 1 -- 172.21.15.67:0/1373722182 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c800fe30 con 0x7f49d415add0 2023-10-14T10:37:13.078 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.056+0000 7f49da5a2700 1 -- 172.21.15.67:0/1373722182 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49d409be80 con 0x7f49d415add0 2023-10-14T10:37:13.078 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.056+0000 7f49da5a2700 1 -- 172.21.15.67:0/1373722182 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49d406f260 con 0x7f49d415add0 2023-10-14T10:37:13.079 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.057+0000 7f49da5a2700 1 -- 172.21.15.67:0/1373722182 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49d4067d20 con 0x7f49d415add0 2023-10-14T10:37:13.079 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.057+0000 7f49d0ff9700 1 -- 172.21.15.67:0/1373722182 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f49c8014440 con 0x7f49d415add0 2023-10-14T10:37:13.079 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.058+0000 7f49d0ff9700 1 --2- 172.21.15.67:0/1373722182 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f49c006af00 0x7f49c006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:13.079 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.058+0000 7f49d0ff9700 1 -- 172.21.15.67:0/1373722182 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5729+0+0 (secure 0 0 0) 0x7f49c8089cf0 con 0x7f49d415add0 2023-10-14T10:37:13.080 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.058+0000 7f49d37fe700 1 --2- 172.21.15.67:0/1373722182 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f49c006af00 0x7f49c006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:13.080 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.059+0000 7f49d37fe700 1 --2- 172.21.15.67:0/1373722182 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f49c006af00 0x7f49c006d3c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f49c40048d0 tx=0x7f49c4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:13.080 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.062+0000 7f49d0ff9700 1 -- 172.21.15.67:0/1373722182 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f49c808e070 con 0x7f49d415add0 2023-10-14T10:37:13.265 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.263+0000 7f49da5a2700 1 -- 172.21.15.67:0/1373722182 --> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f49d409b750 con 0x7f49c006af00 2023-10-14T10:37:13.266 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.264+0000 7f49d0ff9700 1 -- 172.21.15.67:0/1373722182 <== mgr.4100 v2:172.21.15.72:6832/703488382 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f49d409b750 con 0x7f49c006af00 2023-10-14T10:37:13.268 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.266+0000 7f49da5a2700 1 -- 172.21.15.67:0/1373722182 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f49c006af00 msgr2=0x7f49c006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:13.268 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.266+0000 7f49da5a2700 1 --2- 172.21.15.67:0/1373722182 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f49c006af00 0x7f49c006d3c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f49c40048d0 tx=0x7f49c4009040 comp rx=0 tx=0).stop 2023-10-14T10:37:13.269 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.266+0000 7f49da5a2700 1 -- 172.21.15.67:0/1373722182 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f49d415add0 msgr2=0x7f49d40b3f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:13.269 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.266+0000 7f49da5a2700 1 --2- 172.21.15.67:0/1373722182 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f49d415add0 0x7f49d40b3f00 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f49c80119a0 tx=0x7f49c8011d60 comp rx=0 tx=0).stop 2023-10-14T10:37:13.269 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.266+0000 7f49da5a2700 1 -- 172.21.15.67:0/1373722182 shutdown_connections 2023-10-14T10:37:13.269 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.266+0000 7f49da5a2700 1 --2- 172.21.15.67:0/1373722182 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f49c006af00 0x7f49c006d3c0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:13.270 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.266+0000 7f49da5a2700 1 --2- 172.21.15.67:0/1373722182 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f49d409e9d0 0x7f49d406ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:13.270 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.266+0000 7f49da5a2700 1 --2- 172.21.15.67:0/1373722182 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f49d415add0 0x7f49d40b3f00 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:13.270 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.266+0000 7f49da5a2700 1 --2- 172.21.15.67:0/1373722182 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f49d415b1b0 0x7f49d409e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:13.271 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.266+0000 7f49da5a2700 1 -- 172.21.15.67:0/1373722182 >> 172.21.15.67:0/1373722182 conn(0x7f49d40d15f0 msgr2=0x7f49d415cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:37:13.271 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.266+0000 7f49da5a2700 1 -- 172.21.15.67:0/1373722182 shutdown_connections 2023-10-14T10:37:13.271 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.266+0000 7f49da5a2700 1 -- 172.21.15.67:0/1373722182 wait complete. 2023-10-14T10:37:13.272 INFO:teuthology.orchestra.run.smithi067.stderr:instructed osd(s) 4 to deep-scrub 2023-10-14T10:37:13.280 INFO:tasks.ceph:Scrubbing osd.5 2023-10-14T10:37:13.280 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-10-14T10:37:13.410 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.407+0000 7fd7d611d700 1 Processor -- start 2023-10-14T10:37:13.410 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.408+0000 7fd7d611d700 1 -- start start 2023-10-14T10:37:13.414 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.408+0000 7fd7d611d700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fd7d015d840 0x7fd7d0162070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:13.414 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.408+0000 7fd7d611d700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7d00cee20 con 0x7fd7d00cdfd0 2023-10-14T10:37:13.415 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.408+0000 7fd7d611d700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7d00b2ce0 con 0x7fd7d00cda40 2023-10-14T10:37:13.415 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.408+0000 7fd7d611d700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7d00ceb40 con 0x7fd7d015d840 2023-10-14T10:37:13.415 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.409+0000 7fd7ceffd700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fd7d00cdfd0 0x7fd7d00cd330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:53936/0 (socket says 172.21.15.67:53936) 2023-10-14T10:37:13.415 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.409+0000 7fd7ceffd700 1 -- 172.21.15.67:0/3637011163 learned_addr learned my addr 172.21.15.67:0/3637011163 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:37:13.416 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.409+0000 7fd7cf7fe700 1 --2- 172.21.15.67:0/3637011163 >> v2:172.21.15.72:3300/0 conn(0x7fd7d015d840 0x7fd7d0162070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:13.416 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.409+0000 7fd7ce7fc700 1 -- 172.21.15.67:0/3637011163 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3387982279 0 0) 0x7fd7d00cee20 con 0x7fd7d00cdfd0 2023-10-14T10:37:13.416 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.409+0000 7fd7ce7fc700 1 -- 172.21.15.67:0/3637011163 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7b80035b0 con 0x7fd7d00cdfd0 2023-10-14T10:37:13.417 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.409+0000 7fd7ce7fc700 1 -- 172.21.15.67:0/3637011163 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 591406799 0 0) 0x7fd7b80035b0 con 0x7fd7d00cdfd0 2023-10-14T10:37:13.417 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.409+0000 7fd7ce7fc700 1 -- 172.21.15.67:0/3637011163 >> v2:172.21.15.72:3300/0 conn(0x7fd7d015d840 msgr2=0x7fd7d0162070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:37:13.417 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.409+0000 7fd7ce7fc700 1 --2- 172.21.15.67:0/3637011163 >> v2:172.21.15.72:3300/0 conn(0x7fd7d015d840 0x7fd7d0162070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:13.417 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.409+0000 7fd7ce7fc700 1 -- 172.21.15.67:0/3637011163 >> v1:172.21.15.72:6789/0 conn(0x7fd7d00cda40 legacy=0x7fd7d00caa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:13.418 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.409+0000 7fd7ce7fc700 1 -- 172.21.15.67:0/3637011163 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7d0161590 con 0x7fd7d00cdfd0 2023-10-14T10:37:13.418 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.409+0000 7fd7ce7fc700 1 -- 172.21.15.67:0/3637011163 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3180043755 0 0) 0x7fd7c4001670 con 0x7fd7d00cdfd0 2023-10-14T10:37:13.418 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.409+0000 7fd7ce7fc700 1 -- 172.21.15.67:0/3637011163 >> v1:172.21.15.67:6789/0 conn(0x7fd7d00cdfd0 legacy=0x7fd7d00cd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:13.418 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.409+0000 7fd7ce7fc700 1 --2- 172.21.15.67:0/3637011163 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd7b8003a00 0x7fd7b8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:13.419 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.409+0000 7fd7ce7fc700 1 --2- 172.21.15.67:0/3637011163 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd7b8004850 0x7fd7b8004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:13.419 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.409+0000 7fd7ce7fc700 1 --2- 172.21.15.67:0/3637011163 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fd7b8005210 0x7fd7b800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:13.419 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.409+0000 7fd7ce7fc700 1 -- 172.21.15.67:0/3637011163 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d00ceb40 con 0x7fd7b8004850 2023-10-14T10:37:13.420 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.409+0000 7fd7ce7fc700 1 -- 172.21.15.67:0/3637011163 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b800c0b0 con 0x7fd7b8003a00 2023-10-14T10:37:13.420 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.409+0000 7fd7ce7fc700 1 -- 172.21.15.67:0/3637011163 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b800c270 con 0x7fd7b8005210 2023-10-14T10:37:13.420 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.409+0000 7fd7ceffd700 1 --2- 172.21.15.67:0/3637011163 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd7b8004850 0x7fd7b8004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:13.420 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.409+0000 7fd7cf7fe700 1 --2- 172.21.15.67:0/3637011163 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd7b8003a00 0x7fd7b8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:13.421 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.410+0000 7fd7cffff700 1 --2- 172.21.15.67:0/3637011163 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fd7b8005210 0x7fd7b800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:13.421 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.410+0000 7fd7ceffd700 1 -- 172.21.15.67:0/3637011163 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fd7b8005210 msgr2=0x7fd7b800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:37:13.421 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.410+0000 7fd7ceffd700 1 --2- 172.21.15.67:0/3637011163 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fd7b8005210 0x7fd7b800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:13.421 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.410+0000 7fd7ceffd700 1 -- 172.21.15.67:0/3637011163 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd7b8003a00 msgr2=0x7fd7b8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:37:13.422 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.410+0000 7fd7ceffd700 1 --2- 172.21.15.67:0/3637011163 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd7b8003a00 0x7fd7b8003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:13.422 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.410+0000 7fd7ceffd700 1 -- 172.21.15.67:0/3637011163 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7b80035b0 con 0x7fd7b8004850 2023-10-14T10:37:13.422 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.410+0000 7fd7cffff700 1 --2- 172.21.15.67:0/3637011163 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fd7b8005210 0x7fd7b800ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-14T10:37:13.423 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.410+0000 7fd7cf7fe700 1 --2- 172.21.15.67:0/3637011163 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd7b8003a00 0x7fd7b8003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-14T10:37:13.423 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.410+0000 7fd7cffff700 1 -- 172.21.15.67:0/3637011163 reap_dead start 2023-10-14T10:37:13.423 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.410+0000 7fd7ceffd700 1 --2- 172.21.15.67:0/3637011163 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd7b8004850 0x7fd7b8004cd0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fd7c400c2a0 tx=0x7fd7b8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c4b4965f4532aa1 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:13.423 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.410+0000 7fd7ce7fc700 1 -- 172.21.15.67:0/3637011163 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c4007c10 con 0x7fd7b8004850 2023-10-14T10:37:13.424 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.410+0000 7fd7ce7fc700 1 -- 172.21.15.67:0/3637011163 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7c400cdc0 con 0x7fd7b8004850 2023-10-14T10:37:13.424 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.410+0000 7fd7ce7fc700 1 -- 172.21.15.67:0/3637011163 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c4005ba0 con 0x7fd7b8004850 2023-10-14T10:37:13.424 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.410+0000 7fd7d611d700 1 -- 172.21.15.67:0/3637011163 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd7b8004850 msgr2=0x7fd7b8004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:13.424 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.410+0000 7fd7d611d700 1 --2- 172.21.15.67:0/3637011163 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd7b8004850 0x7fd7b8004cd0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fd7c400c2a0 tx=0x7fd7b8005880 comp rx=0 tx=0).stop 2023-10-14T10:37:13.425 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.411+0000 7fd7d611d700 1 -- 172.21.15.67:0/3637011163 shutdown_connections 2023-10-14T10:37:13.425 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.411+0000 7fd7d611d700 1 --2- 172.21.15.67:0/3637011163 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd7b8004850 0x7fd7b8004cd0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:13.425 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.411+0000 7fd7d611d700 1 -- 172.21.15.67:0/3637011163 >> 172.21.15.67:0/3637011163 conn(0x7fd7d006bd70 msgr2=0x7fd7d006c180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:37:13.426 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.411+0000 7fd7d611d700 1 -- 172.21.15.67:0/3637011163 shutdown_connections 2023-10-14T10:37:13.426 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.411+0000 7fd7d611d700 1 -- 172.21.15.67:0/3637011163 wait complete. 2023-10-14T10:37:13.426 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.412+0000 7fd7d611d700 1 Processor -- start 2023-10-14T10:37:13.426 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.412+0000 7fd7d611d700 1 -- start start 2023-10-14T10:37:13.430 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.413+0000 7fd7d611d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd7b8004850 0x7fd7d0152a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:13.430 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.413+0000 7fd7d611d700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fd7d0158680 0x7fd7d00b57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:13.430 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.413+0000 7fd7d611d700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd7d00b30b0 0x7fd7d00aec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:13.431 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.413+0000 7fd7d611d700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d01576d0 con 0x7fd7b8004850 2023-10-14T10:37:13.431 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.413+0000 7fd7d611d700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d01578e0 con 0x7fd7d00b30b0 2023-10-14T10:37:13.431 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.413+0000 7fd7d611d700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d0157bc0 con 0x7fd7d0158680 2023-10-14T10:37:13.431 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.413+0000 7fd7cf7fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd7b8004850 0x7fd7d0152a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:13.432 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.413+0000 7fd7cf7fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd7b8004850 0x7fd7d0152a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46180/0 (socket says 172.21.15.67:46180) 2023-10-14T10:37:13.432 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.413+0000 7fd7cf7fe700 1 -- 172.21.15.67:0/2816355393 learned_addr learned my addr 172.21.15.67:0/2816355393 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:37:13.432 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.413+0000 7fd7ceffd700 1 --2- 172.21.15.67:0/2816355393 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fd7d0158680 0x7fd7d00b57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:13.433 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.413+0000 7fd7cffff700 1 --2- 172.21.15.67:0/2816355393 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd7d00b30b0 0x7fd7d00aec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:13.433 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.413+0000 7fd7cf7fe700 1 -- 172.21.15.67:0/2816355393 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fd7d0158680 msgr2=0x7fd7d00b57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:13.433 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.413+0000 7fd7cf7fe700 1 --2- 172.21.15.67:0/2816355393 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fd7d0158680 0x7fd7d00b57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:13.434 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.413+0000 7fd7cf7fe700 1 -- 172.21.15.67:0/2816355393 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd7d00b30b0 msgr2=0x7fd7d00aec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:13.434 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.413+0000 7fd7cf7fe700 1 --2- 172.21.15.67:0/2816355393 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd7d00b30b0 0x7fd7d00aec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:13.434 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.413+0000 7fd7cf7fe700 1 -- 172.21.15.67:0/2816355393 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7d0157ea0 con 0x7fd7b8004850 2023-10-14T10:37:13.434 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.413+0000 7fd7cffff700 1 --2- 172.21.15.67:0/2816355393 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd7d00b30b0 0x7fd7d00aec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-14T10:37:13.435 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.413+0000 7fd7ceffd700 1 --2- 172.21.15.67:0/2816355393 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fd7d0158680 0x7fd7d00b57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-14T10:37:13.435 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.413+0000 7fd7cf7fe700 1 --2- 172.21.15.67:0/2816355393 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd7b8004850 0x7fd7d0152a80 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fd7c0011c30 tx=0x7fd7c000f240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:13.435 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.413+0000 7fd7b7fff700 1 -- 172.21.15.67:0/2816355393 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c000fb80 con 0x7fd7b8004850 2023-10-14T10:37:13.436 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.413+0000 7fd7b7fff700 1 -- 172.21.15.67:0/2816355393 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7c000fce0 con 0x7fd7b8004850 2023-10-14T10:37:13.436 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.413+0000 7fd7d611d700 1 -- 172.21.15.67:0/2816355393 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7d0158180 con 0x7fd7b8004850 2023-10-14T10:37:13.436 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.413+0000 7fd7b7fff700 1 -- 172.21.15.67:0/2816355393 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c0004260 con 0x7fd7b8004850 2023-10-14T10:37:13.436 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.414+0000 7fd7d611d700 1 -- 172.21.15.67:0/2816355393 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7d00af1b0 con 0x7fd7b8004850 2023-10-14T10:37:13.437 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.415+0000 7fd7d611d700 1 -- 172.21.15.67:0/2816355393 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd7d0161b50 con 0x7fd7b8004850 2023-10-14T10:37:13.437 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.415+0000 7fd7b7fff700 1 -- 172.21.15.67:0/2816355393 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd7c00043c0 con 0x7fd7b8004850 2023-10-14T10:37:13.437 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.415+0000 7fd7b7fff700 1 --2- 172.21.15.67:0/2816355393 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fd7bc06af30 0x7fd7bc06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:13.437 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.415+0000 7fd7b7fff700 1 -- 172.21.15.67:0/2816355393 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5729+0+0 (secure 0 0 0) 0x7fd7c0089c90 con 0x7fd7b8004850 2023-10-14T10:37:13.438 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.415+0000 7fd7b7fff700 1 --2- 172.21.15.67:0/2816355393 >> [v2:172.21.15.72:6800/569686991,v1:172.21.15.72:6801/569686991] conn(0x7fd7bc073130 0x7fd7bc075570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:13.438 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.415+0000 7fd7b7fff700 1 -- 172.21.15.67:0/2816355393 --> [v2:172.21.15.72:6800/569686991,v1:172.21.15.72:6801/569686991] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd7bc075c40 con 0x7fd7bc073130 2023-10-14T10:37:13.438 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.415+0000 7fd7b7fff700 1 -- 172.21.15.67:0/2816355393 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd7c008c350 con 0x7fd7b8004850 2023-10-14T10:37:13.438 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.416+0000 7fd7ceffd700 1 --2- 172.21.15.67:0/2816355393 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fd7bc06af30 0x7fd7bc06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:13.439 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.416+0000 7fd7cffff700 1 --2- 172.21.15.67:0/2816355393 >> [v2:172.21.15.72:6800/569686991,v1:172.21.15.72:6801/569686991] conn(0x7fd7bc073130 0x7fd7bc075570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:13.439 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.416+0000 7fd7ceffd700 1 --2- 172.21.15.67:0/2816355393 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fd7bc06af30 0x7fd7bc06d3f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd7c4001930 tx=0x7fd7c4007000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:13.439 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.416+0000 7fd7cffff700 1 --2- 172.21.15.67:0/2816355393 >> [v2:172.21.15.72:6800/569686991,v1:172.21.15.72:6801/569686991] conn(0x7fd7bc073130 0x7fd7bc075570 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:13.440 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.417+0000 7fd7b7fff700 1 -- 172.21.15.67:0/2816355393 <== osd.5 v2:172.21.15.72:6800/569686991 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7fd7bc075c40 con 0x7fd7bc073130 2023-10-14T10:37:13.440 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.432+0000 7fd7d611d700 1 -- 172.21.15.67:0/2816355393 --> [v2:172.21.15.72:6800/569686991,v1:172.21.15.72:6801/569686991] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fd7d0161870 con 0x7fd7bc073130 2023-10-14T10:37:13.441 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.432+0000 7fd7b7fff700 1 -- 172.21.15.67:0/2816355393 <== osd.5 v2:172.21.15.72:6800/569686991 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fd7c8001bc0 con 0x7fd7bc073130 2023-10-14T10:37:13.441 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2023-10-14T10:37:13.441 INFO:teuthology.orchestra.run.smithi067.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-14T10:37:13.442 INFO:teuthology.orchestra.run.smithi067.stdout:} 2023-10-14T10:37:13.443 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.432+0000 7fd7d611d700 1 -- 172.21.15.67:0/2816355393 >> [v2:172.21.15.72:6800/569686991,v1:172.21.15.72:6801/569686991] conn(0x7fd7bc073130 msgr2=0x7fd7bc075570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:13.443 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.432+0000 7fd7d611d700 1 --2- 172.21.15.67:0/2816355393 >> [v2:172.21.15.72:6800/569686991,v1:172.21.15.72:6801/569686991] conn(0x7fd7bc073130 0x7fd7bc075570 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:13.443 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.432+0000 7fd7d611d700 1 -- 172.21.15.67:0/2816355393 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fd7bc06af30 msgr2=0x7fd7bc06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:13.444 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.432+0000 7fd7d611d700 1 --2- 172.21.15.67:0/2816355393 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fd7bc06af30 0x7fd7bc06d3f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd7c4001930 tx=0x7fd7c4007000 comp rx=0 tx=0).stop 2023-10-14T10:37:13.444 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.432+0000 7fd7d611d700 1 -- 172.21.15.67:0/2816355393 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd7b8004850 msgr2=0x7fd7d0152a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:13.444 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.432+0000 7fd7d611d700 1 --2- 172.21.15.67:0/2816355393 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd7b8004850 0x7fd7d0152a80 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fd7c0011c30 tx=0x7fd7c000f240 comp rx=0 tx=0).stop 2023-10-14T10:37:13.445 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.432+0000 7fd7cffff700 1 -- 172.21.15.67:0/2816355393 reap_dead start 2023-10-14T10:37:13.445 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.432+0000 7fd7d611d700 1 -- 172.21.15.67:0/2816355393 shutdown_connections 2023-10-14T10:37:13.445 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.432+0000 7fd7d611d700 1 -- 172.21.15.67:0/2816355393 >> 172.21.15.67:0/2816355393 conn(0x7fd7d006bd70 msgr2=0x7fd7d00cd300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:37:13.445 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.433+0000 7fd7d611d700 1 -- 172.21.15.67:0/2816355393 shutdown_connections 2023-10-14T10:37:13.446 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.433+0000 7fd7d611d700 1 -- 172.21.15.67:0/2816355393 wait complete. 2023-10-14T10:37:13.447 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-10-14T10:37:13.575 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.570+0000 7f40e93f5700 1 Processor -- start 2023-10-14T10:37:13.575 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.570+0000 7f40e93f5700 1 -- start start 2023-10-14T10:37:13.576 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.571+0000 7f40e93f5700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f40e40d3fc0 0x7f40e415dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:13.576 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.571+0000 7f40e93f5700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f40e40b7050 0x7f40e40d21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:13.576 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.571+0000 7f40e93f5700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40e415e4e0 con 0x7f40e415dbc0 2023-10-14T10:37:13.576 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.571+0000 7f40e93f5700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40e40b7560 con 0x7f40e40b7050 2023-10-14T10:37:13.577 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.571+0000 7f40e93f5700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40e40b7840 con 0x7f40e40d3fc0 2023-10-14T10:37:13.577 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.571+0000 7f40e37fe700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f40e40b7050 0x7f40e40d21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:13.577 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.571+0000 7f40e37fe700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f40e40b7050 0x7f40e40d21d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46194/0 (socket says 172.21.15.67:46194) 2023-10-14T10:37:13.578 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.571+0000 7f40e37fe700 1 -- 172.21.15.67:0/3281321029 learned_addr learned my addr 172.21.15.67:0/3281321029 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:37:13.578 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.571+0000 7f40e27fc700 1 --2- 172.21.15.67:0/3281321029 >> v2:172.21.15.72:3300/0 conn(0x7f40e40d3fc0 0x7f40e415dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:13.578 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.571+0000 7f40e37fe700 1 -- 172.21.15.67:0/3281321029 >> v2:172.21.15.72:3300/0 conn(0x7f40e40d3fc0 msgr2=0x7f40e415dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:37:13.578 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.571+0000 7f40e37fe700 1 --2- 172.21.15.67:0/3281321029 >> v2:172.21.15.72:3300/0 conn(0x7f40e40d3fc0 0x7f40e415dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:13.579 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.571+0000 7f40e37fe700 1 -- 172.21.15.67:0/3281321029 >> v1:172.21.15.72:6789/0 conn(0x7f40e415dbc0 legacy=0x7f40e40d1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:37:13.579 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.571+0000 7f40e37fe700 1 -- 172.21.15.67:0/3281321029 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40e40b7b20 con 0x7f40e40b7050 2023-10-14T10:37:13.579 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.572+0000 7f40e37fe700 1 --2- 172.21.15.67:0/3281321029 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f40e40b7050 0x7f40e40d21d0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f40e40d14e0 tx=0x7f40dc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beee3a9f98dfa34a server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:13.580 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.572+0000 7f40e1ffb700 1 -- 172.21.15.67:0/3281321029 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40dc00e070 con 0x7f40e40b7050 2023-10-14T10:37:13.580 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.572+0000 7f40e1ffb700 1 -- 172.21.15.67:0/3281321029 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40dc003a20 con 0x7f40e40b7050 2023-10-14T10:37:13.580 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.572+0000 7f40e1ffb700 1 -- 172.21.15.67:0/3281321029 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40dc003dd0 con 0x7f40e40b7050 2023-10-14T10:37:13.580 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.572+0000 7f40e93f5700 1 -- 172.21.15.67:0/3281321029 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f40e40b7050 msgr2=0x7f40e40d21d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:13.581 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.572+0000 7f40e93f5700 1 --2- 172.21.15.67:0/3281321029 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f40e40b7050 0x7f40e40d21d0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f40e40d14e0 tx=0x7f40dc00b330 comp rx=0 tx=0).stop 2023-10-14T10:37:13.581 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.572+0000 7f40e93f5700 1 -- 172.21.15.67:0/3281321029 shutdown_connections 2023-10-14T10:37:13.581 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.572+0000 7f40e93f5700 1 --2- 172.21.15.67:0/3281321029 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f40e40b7050 0x7f40e40d21d0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:13.581 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.572+0000 7f40e93f5700 1 --2- 172.21.15.67:0/3281321029 >> v2:172.21.15.72:3300/0 conn(0x7f40e40d3fc0 0x7f40e415dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:13.582 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.572+0000 7f40e93f5700 1 -- 172.21.15.67:0/3281321029 >> 172.21.15.67:0/3281321029 conn(0x7f40e409cb40 msgr2=0x7f40e40d3820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:37:13.582 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.572+0000 7f40e93f5700 1 -- 172.21.15.67:0/3281321029 shutdown_connections 2023-10-14T10:37:13.582 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.572+0000 7f40e93f5700 1 -- 172.21.15.67:0/3281321029 wait complete. 2023-10-14T10:37:13.585 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.574+0000 7f40e93f5700 1 Processor -- start 2023-10-14T10:37:13.585 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.574+0000 7f40e93f5700 1 -- start start 2023-10-14T10:37:13.586 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.575+0000 7f40e93f5700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f40e40b7050 0x7f40e4154e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:13.586 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.575+0000 7f40e93f5700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f40e40d3fc0 0x7f40e41f3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:13.586 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.575+0000 7f40e93f5700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f40e415dbc0 0x7f40e40cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:13.587 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.575+0000 7f40e93f5700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40e41f3d60 con 0x7f40e415dbc0 2023-10-14T10:37:13.587 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.575+0000 7f40e93f5700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40e40cd1f0 con 0x7f40e40d3fc0 2023-10-14T10:37:13.587 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.575+0000 7f40e93f5700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40e40cbc10 con 0x7f40e40b7050 2023-10-14T10:37:13.587 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.575+0000 7f40e37fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f40e415dbc0 0x7f40e40cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:13.588 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.575+0000 7f40e37fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f40e415dbc0 0x7f40e40cccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46200/0 (socket says 172.21.15.67:46200) 2023-10-14T10:37:13.588 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.575+0000 7f40e37fe700 1 -- 172.21.15.67:0/3837187674 learned_addr learned my addr 172.21.15.67:0/3837187674 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:37:13.588 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.575+0000 7f40e27fc700 1 --2- 172.21.15.67:0/3837187674 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f40e40d3fc0 0x7f40e41f3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:13.589 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.575+0000 7f40e2ffd700 1 --2- 172.21.15.67:0/3837187674 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f40e40b7050 0x7f40e4154e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:13.589 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.575+0000 7f40e37fe700 1 -- 172.21.15.67:0/3837187674 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f40e40b7050 msgr2=0x7f40e4154e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:13.589 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.575+0000 7f40e37fe700 1 --2- 172.21.15.67:0/3837187674 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f40e40b7050 0x7f40e4154e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:13.589 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.575+0000 7f40e37fe700 1 -- 172.21.15.67:0/3837187674 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f40e40d3fc0 msgr2=0x7f40e41f3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:13.590 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.575+0000 7f40e37fe700 1 --2- 172.21.15.67:0/3837187674 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f40e40d3fc0 0x7f40e41f3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:13.590 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.575+0000 7f40e37fe700 1 -- 172.21.15.67:0/3837187674 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40dc009b30 con 0x7f40e415dbc0 2023-10-14T10:37:13.590 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.575+0000 7f40e2ffd700 1 --2- 172.21.15.67:0/3837187674 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f40e40b7050 0x7f40e4154e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-14T10:37:13.590 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.575+0000 7f40e37fe700 1 --2- 172.21.15.67:0/3837187674 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f40e415dbc0 0x7f40e40cccb0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f40dc00b370 tx=0x7f40dc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:13.591 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.575+0000 7f40cb7fe700 1 -- 172.21.15.67:0/3837187674 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40dc004440 con 0x7f40e415dbc0 2023-10-14T10:37:13.591 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.575+0000 7f40cb7fe700 1 -- 172.21.15.67:0/3837187674 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40dc0045a0 con 0x7f40e415dbc0 2023-10-14T10:37:13.591 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.575+0000 7f40e93f5700 1 -- 172.21.15.67:0/3837187674 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40e40cbef0 con 0x7f40e415dbc0 2023-10-14T10:37:13.591 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.576+0000 7f40cb7fe700 1 -- 172.21.15.67:0/3837187674 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40dc004950 con 0x7f40e415dbc0 2023-10-14T10:37:13.592 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.576+0000 7f40e93f5700 1 -- 172.21.15.67:0/3837187674 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40e40cc4b0 con 0x7f40e415dbc0 2023-10-14T10:37:13.592 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.576+0000 7f40e93f5700 1 -- 172.21.15.67:0/3837187674 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40e404ebb0 con 0x7f40e415dbc0 2023-10-14T10:37:13.592 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.578+0000 7f40cb7fe700 1 -- 172.21.15.67:0/3837187674 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f40dc019030 con 0x7f40e415dbc0 2023-10-14T10:37:13.592 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.578+0000 7f40cb7fe700 1 --2- 172.21.15.67:0/3837187674 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f40d006af00 0x7f40d006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:13.593 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.578+0000 7f40cb7fe700 1 -- 172.21.15.67:0/3837187674 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5729+0+0 (secure 0 0 0) 0x7f40dc08ab20 con 0x7f40e415dbc0 2023-10-14T10:37:13.593 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.578+0000 7f40e2ffd700 1 --2- 172.21.15.67:0/3837187674 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f40d006af00 0x7f40d006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:13.593 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.579+0000 7f40e2ffd700 1 --2- 172.21.15.67:0/3837187674 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f40d006af00 0x7f40d006d3c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f40d8006fd0 tx=0x7f40d8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:13.594 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.582+0000 7f40cb7fe700 1 -- 172.21.15.67:0/3837187674 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f40dc055d20 con 0x7f40e415dbc0 2023-10-14T10:37:13.785 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.783+0000 7f40e93f5700 1 -- 172.21.15.67:0/3837187674 --> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f40e40b80e0 con 0x7f40d006af00 2023-10-14T10:37:13.785 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.783+0000 7f40cb7fe700 1 -- 172.21.15.67:0/3837187674 <== mgr.4100 v2:172.21.15.72:6832/703488382 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f40d8009600 con 0x7f40d006af00 2023-10-14T10:37:13.787 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.785+0000 7f40e93f5700 1 -- 172.21.15.67:0/3837187674 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f40d006af00 msgr2=0x7f40d006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:13.788 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.785+0000 7f40e93f5700 1 --2- 172.21.15.67:0/3837187674 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f40d006af00 0x7f40d006d3c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f40d8006fd0 tx=0x7f40d8008040 comp rx=0 tx=0).stop 2023-10-14T10:37:13.788 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.785+0000 7f40e93f5700 1 -- 172.21.15.67:0/3837187674 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f40e415dbc0 msgr2=0x7f40e40cccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:13.788 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.785+0000 7f40e93f5700 1 --2- 172.21.15.67:0/3837187674 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f40e415dbc0 0x7f40e40cccb0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f40dc00b370 tx=0x7f40dc004ad0 comp rx=0 tx=0).stop 2023-10-14T10:37:13.789 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.785+0000 7f40e93f5700 1 -- 172.21.15.67:0/3837187674 shutdown_connections 2023-10-14T10:37:13.789 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.785+0000 7f40e93f5700 1 --2- 172.21.15.67:0/3837187674 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f40d006af00 0x7f40d006d3c0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:13.789 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.785+0000 7f40e93f5700 1 --2- 172.21.15.67:0/3837187674 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f40e415dbc0 0x7f40e40cccb0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:13.789 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.785+0000 7f40e93f5700 1 --2- 172.21.15.67:0/3837187674 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f40e40b7050 0x7f40e4154e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:13.790 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.785+0000 7f40e93f5700 1 --2- 172.21.15.67:0/3837187674 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f40e40d3fc0 0x7f40e41f3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:13.790 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.785+0000 7f40e93f5700 1 -- 172.21.15.67:0/3837187674 >> 172.21.15.67:0/3837187674 conn(0x7f40e409cb40 msgr2=0x7f40e406d230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:37:13.790 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.785+0000 7f40e93f5700 1 -- 172.21.15.67:0/3837187674 shutdown_connections 2023-10-14T10:37:13.790 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.785+0000 7f40e93f5700 1 -- 172.21.15.67:0/3837187674 wait complete. 2023-10-14T10:37:13.791 INFO:teuthology.orchestra.run.smithi067.stderr:instructed osd(s) 5 to deep-scrub 2023-10-14T10:37:13.799 INFO:tasks.ceph:Scrubbing osd.6 2023-10-14T10:37:13.800 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-10-14T10:37:13.929 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.927+0000 7ff0043d5700 1 Processor -- start 2023-10-14T10:37:13.929 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.927+0000 7ff0043d5700 1 -- start start 2023-10-14T10:37:13.932 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.928+0000 7ff0043d5700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7feffc0cdf10 0x7feffc0d04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:13.932 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.928+0000 7ff0043d5700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7feffc0cc240 0x7feffc0cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:13.932 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.928+0000 7ff0043d5700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feffc0cbdb0 con 0x7feffc0b0730 2023-10-14T10:37:13.933 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.928+0000 7ff0043d5700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feffc0d0b40 con 0x7feffc0cdf10 2023-10-14T10:37:13.933 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.928+0000 7ff0043d5700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feffc0d0e20 con 0x7feffc0cc240 2023-10-14T10:37:13.933 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.928+0000 7ff002171700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7feffc0cdf10 0x7feffc0d04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:13.933 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.928+0000 7ff002171700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7feffc0cdf10 0x7feffc0d04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46204/0 (socket says 172.21.15.67:46204) 2023-10-14T10:37:13.934 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.928+0000 7ff002171700 1 -- 172.21.15.67:0/26887676 learned_addr learned my addr 172.21.15.67:0/26887676 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:37:13.934 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.928+0000 7ff002171700 1 -- 172.21.15.67:0/26887676 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7feffc0cc240 msgr2=0x7feffc0cc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:37:13.934 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.928+0000 7ff002972700 1 --2- 172.21.15.67:0/26887676 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7feffc0cc240 0x7feffc0cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:13.934 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.928+0000 7ff002171700 1 --2- 172.21.15.67:0/26887676 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7feffc0cc240 0x7feffc0cc6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:13.935 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.928+0000 7ff002171700 1 -- 172.21.15.67:0/26887676 >> v1:172.21.15.72:6789/0 conn(0x7feffc0b0730 legacy=0x7feffc0ce2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:37:13.935 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.928+0000 7ff002171700 1 -- 172.21.15.67:0/26887676 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feffc0d1100 con 0x7feffc0cdf10 2023-10-14T10:37:13.935 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.928+0000 7ff002171700 1 --2- 172.21.15.67:0/26887676 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feffc0cdf10 0x7feffc0d04b0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fefec006fd0 tx=0x7fefec009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da2c84175879a593 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:13.935 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.928+0000 7ff00116f700 1 -- 172.21.15.67:0/26887676 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefec005e30 con 0x7feffc0cdf10 2023-10-14T10:37:13.936 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.929+0000 7ff00116f700 1 -- 172.21.15.67:0/26887676 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefec004030 con 0x7feffc0cdf10 2023-10-14T10:37:13.936 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.929+0000 7ff00116f700 1 -- 172.21.15.67:0/26887676 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefec0043e0 con 0x7feffc0cdf10 2023-10-14T10:37:13.936 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.929+0000 7ff0043d5700 1 -- 172.21.15.67:0/26887676 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feffc0cdf10 msgr2=0x7feffc0d04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:13.937 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.929+0000 7ff0043d5700 1 --2- 172.21.15.67:0/26887676 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feffc0cdf10 0x7feffc0d04b0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fefec006fd0 tx=0x7fefec009bd0 comp rx=0 tx=0).stop 2023-10-14T10:37:13.937 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.929+0000 7ff0043d5700 1 -- 172.21.15.67:0/26887676 shutdown_connections 2023-10-14T10:37:13.937 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.929+0000 7ff0043d5700 1 --2- 172.21.15.67:0/26887676 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7feffc0cc240 0x7feffc0cc6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:13.937 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.929+0000 7ff0043d5700 1 --2- 172.21.15.67:0/26887676 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feffc0cdf10 0x7feffc0d04b0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:13.938 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.929+0000 7ff0043d5700 1 -- 172.21.15.67:0/26887676 >> 172.21.15.67:0/26887676 conn(0x7feffc0d1530 msgr2=0x7feffc0d1940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:37:13.938 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.929+0000 7ff0043d5700 1 -- 172.21.15.67:0/26887676 shutdown_connections 2023-10-14T10:37:13.938 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.929+0000 7ff0043d5700 1 -- 172.21.15.67:0/26887676 wait complete. 2023-10-14T10:37:13.941 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.930+0000 7ff0043d5700 1 Processor -- start 2023-10-14T10:37:13.942 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.930+0000 7ff0043d5700 1 -- start start 2023-10-14T10:37:13.942 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.931+0000 7ff0043d5700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7feffc0b0730 0x7feffc15ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:13.942 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.931+0000 7ff0043d5700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7feffc0cc240 0x7feffc1f2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:13.943 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.931+0000 7ff0043d5700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feffc0cdf10 0x7feffc15c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:13.943 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.931+0000 7ff0043d5700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feffc15b460 con 0x7feffc0cdf10 2023-10-14T10:37:13.943 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.931+0000 7ff0043d5700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feffc15b740 con 0x7feffc0cc240 2023-10-14T10:37:13.943 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.931+0000 7ff0043d5700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feffc15ba20 con 0x7feffc0b0730 2023-10-14T10:37:13.944 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.931+0000 7ff002972700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feffc0cdf10 0x7feffc15c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:13.944 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.931+0000 7ff002972700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feffc0cdf10 0x7feffc15c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46214/0 (socket says 172.21.15.67:46214) 2023-10-14T10:37:13.944 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.931+0000 7ff002972700 1 -- 172.21.15.67:0/3287802706 learned_addr learned my addr 172.21.15.67:0/3287802706 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:37:13.945 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.931+0000 7ff001970700 1 --2- 172.21.15.67:0/3287802706 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7feffc0cc240 0x7feffc1f2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:13.945 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.931+0000 7ff002171700 1 --2- 172.21.15.67:0/3287802706 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7feffc0b0730 0x7feffc15ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:13.945 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.932+0000 7ff002972700 1 -- 172.21.15.67:0/3287802706 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7feffc0b0730 msgr2=0x7feffc15ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:13.945 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.932+0000 7ff002972700 1 --2- 172.21.15.67:0/3287802706 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7feffc0b0730 0x7feffc15ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:13.946 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.932+0000 7ff002972700 1 -- 172.21.15.67:0/3287802706 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7feffc0cc240 msgr2=0x7feffc1f2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:13.946 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.932+0000 7ff002972700 1 --2- 172.21.15.67:0/3287802706 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7feffc0cc240 0x7feffc1f2aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:13.946 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.932+0000 7ff002972700 1 -- 172.21.15.67:0/3287802706 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefec009500 con 0x7feffc0cdf10 2023-10-14T10:37:13.947 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.932+0000 7ff002972700 1 --2- 172.21.15.67:0/3287802706 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feffc0cdf10 0x7feffc15c790 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7feff400cd20 tx=0x7feff400a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:13.947 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.932+0000 7feff2ffd700 1 -- 172.21.15.67:0/3287802706 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff4003a20 con 0x7feffc0cdf10 2023-10-14T10:37:13.947 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.932+0000 7feff2ffd700 1 -- 172.21.15.67:0/3287802706 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feff4003b80 con 0x7feffc0cdf10 2023-10-14T10:37:13.947 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.932+0000 7ff0043d5700 1 -- 172.21.15.67:0/3287802706 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feffc15bd00 con 0x7feffc0cdf10 2023-10-14T10:37:13.948 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.932+0000 7feff2ffd700 1 -- 172.21.15.67:0/3287802706 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff4004020 con 0x7feffc0cdf10 2023-10-14T10:37:13.948 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.932+0000 7ff0043d5700 1 -- 172.21.15.67:0/3287802706 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feffc15c500 con 0x7feffc0cdf10 2023-10-14T10:37:13.948 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.933+0000 7ff0043d5700 1 -- 172.21.15.67:0/3287802706 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7feffc0b0490 con 0x7feffc0cdf10 2023-10-14T10:37:13.948 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.933+0000 7feff2ffd700 1 -- 172.21.15.67:0/3287802706 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7feff4004180 con 0x7feffc0cdf10 2023-10-14T10:37:13.949 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.934+0000 7feff2ffd700 1 --2- 172.21.15.67:0/3287802706 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fefe006af00 0x7fefe006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:13.949 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.934+0000 7feff2ffd700 1 -- 172.21.15.67:0/3287802706 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5729+0+0 (secure 0 0 0) 0x7feff4012070 con 0x7feffc0cdf10 2023-10-14T10:37:13.949 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.934+0000 7feff2ffd700 1 --2- 172.21.15.67:0/3287802706 >> [v2:172.21.15.72:6816/3767685934,v1:172.21.15.72:6817/3767685934] conn(0x7fefe0073100 0x7fefe0075540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:13.950 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.934+0000 7feff2ffd700 1 -- 172.21.15.67:0/3287802706 --> [v2:172.21.15.72:6816/3767685934,v1:172.21.15.72:6817/3767685934] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fefe0075c10 con 0x7fefe0073100 2023-10-14T10:37:13.950 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.934+0000 7feff2ffd700 1 -- 172.21.15.67:0/3287802706 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7feff408e2a0 con 0x7feffc0cdf10 2023-10-14T10:37:13.950 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.934+0000 7ff002171700 1 --2- 172.21.15.67:0/3287802706 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fefe006af00 0x7fefe006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:13.951 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.934+0000 7ff001970700 1 --2- 172.21.15.67:0/3287802706 >> [v2:172.21.15.72:6816/3767685934,v1:172.21.15.72:6817/3767685934] conn(0x7fefe0073100 0x7fefe0075540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:13.951 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.935+0000 7ff002171700 1 --2- 172.21.15.67:0/3287802706 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fefe006af00 0x7fefe006d3c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fefec000c00 tx=0x7fefec008070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:13.951 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.935+0000 7ff001970700 1 --2- 172.21.15.67:0/3287802706 >> [v2:172.21.15.72:6816/3767685934,v1:172.21.15.72:6817/3767685934] conn(0x7fefe0073100 0x7fefe0075540 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:13.951 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.935+0000 7feff2ffd700 1 -- 172.21.15.67:0/3287802706 <== osd.6 v2:172.21.15.72:6816/3767685934 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7fefe0075c10 con 0x7fefe0073100 2023-10-14T10:37:13.952 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.950+0000 7ff0043d5700 1 -- 172.21.15.67:0/3287802706 --> [v2:172.21.15.72:6816/3767685934,v1:172.21.15.72:6817/3767685934] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7feffc15c220 con 0x7fefe0073100 2023-10-14T10:37:13.953 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.951+0000 7feff2ffd700 1 -- 172.21.15.67:0/3287802706 <== osd.6 v2:172.21.15.72:6816/3767685934 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7feffc15c220 con 0x7fefe0073100 2023-10-14T10:37:13.953 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2023-10-14T10:37:13.953 INFO:teuthology.orchestra.run.smithi067.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-14T10:37:13.954 INFO:teuthology.orchestra.run.smithi067.stdout:} 2023-10-14T10:37:13.955 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.951+0000 7ff0043d5700 1 -- 172.21.15.67:0/3287802706 >> [v2:172.21.15.72:6816/3767685934,v1:172.21.15.72:6817/3767685934] conn(0x7fefe0073100 msgr2=0x7fefe0075540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:13.955 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.951+0000 7ff0043d5700 1 --2- 172.21.15.67:0/3287802706 >> [v2:172.21.15.72:6816/3767685934,v1:172.21.15.72:6817/3767685934] conn(0x7fefe0073100 0x7fefe0075540 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:13.956 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.951+0000 7ff0043d5700 1 -- 172.21.15.67:0/3287802706 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fefe006af00 msgr2=0x7fefe006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:13.956 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.951+0000 7ff0043d5700 1 --2- 172.21.15.67:0/3287802706 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fefe006af00 0x7fefe006d3c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fefec000c00 tx=0x7fefec008070 comp rx=0 tx=0).stop 2023-10-14T10:37:13.956 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.951+0000 7ff0043d5700 1 -- 172.21.15.67:0/3287802706 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feffc0cdf10 msgr2=0x7feffc15c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:13.956 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.951+0000 7ff0043d5700 1 --2- 172.21.15.67:0/3287802706 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feffc0cdf10 0x7feffc15c790 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7feff400cd20 tx=0x7feff400a570 comp rx=0 tx=0).stop 2023-10-14T10:37:13.957 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.951+0000 7ff002972700 1 -- 172.21.15.67:0/3287802706 reap_dead start 2023-10-14T10:37:13.957 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.951+0000 7ff0043d5700 1 -- 172.21.15.67:0/3287802706 shutdown_connections 2023-10-14T10:37:13.957 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.951+0000 7ff0043d5700 1 -- 172.21.15.67:0/3287802706 >> 172.21.15.67:0/3287802706 conn(0x7feffc0d1530 msgr2=0x7feffc0cd290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:37:13.958 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.951+0000 7ff0043d5700 1 -- 172.21.15.67:0/3287802706 shutdown_connections 2023-10-14T10:37:13.958 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:13.951+0000 7ff0043d5700 1 -- 172.21.15.67:0/3287802706 wait complete. 2023-10-14T10:37:13.965 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-10-14T10:37:14.091 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.089+0000 7f2d7809d700 1 Processor -- start 2023-10-14T10:37:14.091 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.089+0000 7f2d7809d700 1 -- start start 2023-10-14T10:37:14.095 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.090+0000 7f2d7809d700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f2d700cd4b0 0x7f2d700cbc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:14.098 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.090+0000 7f2d7809d700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d700ce3b0 con 0x7f2d7015a850 2023-10-14T10:37:14.098 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.090+0000 7f2d7809d700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d700b6290 con 0x7f2d700b2940 2023-10-14T10:37:14.098 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.090+0000 7f2d7809d700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d700cc220 con 0x7f2d700cd4b0 2023-10-14T10:37:14.099 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.090+0000 7f2d75e39700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f2d7015a850 0x7f2d700ccda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:53988/0 (socket says 172.21.15.67:53988) 2023-10-14T10:37:14.099 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.090+0000 7f2d75e39700 1 -- 172.21.15.67:0/3267112915 learned_addr learned my addr 172.21.15.67:0/3267112915 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:37:14.099 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.090+0000 7f2d75638700 1 --2- 172.21.15.67:0/3267112915 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f2d700cd4b0 0x7f2d700cbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:14.099 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.090+0000 7f2d74e37700 1 -- 172.21.15.67:0/3267112915 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2984000565 0 0) 0x7f2d700ce3b0 con 0x7f2d7015a850 2023-10-14T10:37:14.100 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.090+0000 7f2d74e37700 1 -- 172.21.15.67:0/3267112915 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d5c0035b0 con 0x7f2d7015a850 2023-10-14T10:37:14.100 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.090+0000 7f2d74e37700 1 -- 172.21.15.67:0/3267112915 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3664560814 0 0) 0x7f2d5c0035b0 con 0x7f2d7015a850 2023-10-14T10:37:14.100 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.090+0000 7f2d74e37700 1 -- 172.21.15.67:0/3267112915 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f2d700cd4b0 msgr2=0x7f2d700cbc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:37:14.101 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.090+0000 7f2d74e37700 1 --2- 172.21.15.67:0/3267112915 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f2d700cd4b0 0x7f2d700cbc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:14.101 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.090+0000 7f2d74e37700 1 -- 172.21.15.67:0/3267112915 >> v1:172.21.15.72:6789/0 conn(0x7f2d700b2940 legacy=0x7f2d700ce840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:14.101 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.090+0000 7f2d74e37700 1 -- 172.21.15.67:0/3267112915 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d700cc500 con 0x7f2d7015a850 2023-10-14T10:37:14.101 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.090+0000 7f2d74e37700 1 -- 172.21.15.67:0/3267112915 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3180043755 0 0) 0x7f2d6c001670 con 0x7f2d7015a850 2023-10-14T10:37:14.102 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.090+0000 7f2d74e37700 1 -- 172.21.15.67:0/3267112915 >> v1:172.21.15.67:6789/0 conn(0x7f2d7015a850 legacy=0x7f2d700ccda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:14.102 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.091+0000 7f2d74e37700 1 --2- 172.21.15.67:0/3267112915 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2d5c003a30 0x7f2d5c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:14.102 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.091+0000 7f2d74e37700 1 --2- 172.21.15.67:0/3267112915 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f2d700cd4b0 0x7f2d5c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:14.102 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.091+0000 7f2d74e37700 1 --2- 172.21.15.67:0/3267112915 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2d5c004d90 0x7f2d5c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:14.103 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.091+0000 7f2d74e37700 1 -- 172.21.15.67:0/3267112915 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d700cc220 con 0x7f2d5c004d90 2023-10-14T10:37:14.103 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.091+0000 7f2d74e37700 1 -- 172.21.15.67:0/3267112915 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d5c00baf0 con 0x7f2d5c003a30 2023-10-14T10:37:14.103 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.091+0000 7f2d74e37700 1 -- 172.21.15.67:0/3267112915 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d5c00bcb0 con 0x7f2d700cd4b0 2023-10-14T10:37:14.104 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.091+0000 7f2d7663a700 1 --2- 172.21.15.67:0/3267112915 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2d5c004d90 0x7f2d5c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:14.104 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.091+0000 7f2d75e39700 1 --2- 172.21.15.67:0/3267112915 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2d5c003a30 0x7f2d5c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:14.104 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.091+0000 7f2d75638700 1 --2- 172.21.15.67:0/3267112915 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f2d700cd4b0 0x7f2d5c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:14.104 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.091+0000 7f2d7663a700 1 -- 172.21.15.67:0/3267112915 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f2d700cd4b0 msgr2=0x7f2d5c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:37:14.105 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.091+0000 7f2d7663a700 1 --2- 172.21.15.67:0/3267112915 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f2d700cd4b0 0x7f2d5c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:14.105 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.091+0000 7f2d7663a700 1 -- 172.21.15.67:0/3267112915 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2d5c003a30 msgr2=0x7f2d5c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:37:14.105 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.091+0000 7f2d7663a700 1 --2- 172.21.15.67:0/3267112915 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2d5c003a30 0x7f2d5c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:14.105 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.091+0000 7f2d7663a700 1 -- 172.21.15.67:0/3267112915 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d5c0035b0 con 0x7f2d5c004d90 2023-10-14T10:37:14.106 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.091+0000 7f2d7663a700 1 --2- 172.21.15.67:0/3267112915 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2d5c004d90 0x7f2d5c005210 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f2d6800c290 tx=0x7f2d5c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9544c7be581a9d39 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:14.106 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.091+0000 7f2d74e37700 1 -- 172.21.15.67:0/3267112915 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d68001e70 con 0x7f2d5c004d90 2023-10-14T10:37:14.106 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.091+0000 7f2d74e37700 1 -- 172.21.15.67:0/3267112915 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d6800ce50 con 0x7f2d5c004d90 2023-10-14T10:37:14.107 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.091+0000 7f2d74e37700 1 -- 172.21.15.67:0/3267112915 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d68012660 con 0x7f2d5c004d90 2023-10-14T10:37:14.107 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.092+0000 7f2d7809d700 1 -- 172.21.15.67:0/3267112915 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2d5c004d90 msgr2=0x7f2d5c005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:14.107 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.092+0000 7f2d7809d700 1 --2- 172.21.15.67:0/3267112915 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2d5c004d90 0x7f2d5c005210 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f2d6800c290 tx=0x7f2d5c005880 comp rx=0 tx=0).stop 2023-10-14T10:37:14.107 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.092+0000 7f2d7663a700 1 -- 172.21.15.67:0/3267112915 reap_dead start 2023-10-14T10:37:14.108 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.092+0000 7f2d7809d700 1 -- 172.21.15.67:0/3267112915 shutdown_connections 2023-10-14T10:37:14.108 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.092+0000 7f2d7809d700 1 -- 172.21.15.67:0/3267112915 >> 172.21.15.67:0/3267112915 conn(0x7f2d7006dac0 msgr2=0x7f2d7006dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:37:14.108 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.092+0000 7f2d7809d700 1 -- 172.21.15.67:0/3267112915 shutdown_connections 2023-10-14T10:37:14.108 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.092+0000 7f2d7809d700 1 -- 172.21.15.67:0/3267112915 wait complete. 2023-10-14T10:37:14.111 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.093+0000 7f2d7809d700 1 Processor -- start 2023-10-14T10:37:14.112 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.093+0000 7f2d7809d700 1 -- start start 2023-10-14T10:37:14.112 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.094+0000 7f2d7809d700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f2d5c004d90 0x7f2d70157d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:14.112 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.094+0000 7f2d7809d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2d700b51f0 0x7f2d700aff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:14.113 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.094+0000 7f2d7809d700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2d70159ad0 0x7f2d700b0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:14.113 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.094+0000 7f2d7809d700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7015dab0 con 0x7f2d700b51f0 2023-10-14T10:37:14.113 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.094+0000 7f2d7809d700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7015d210 con 0x7f2d70159ad0 2023-10-14T10:37:14.113 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.094+0000 7f2d7809d700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7015d4f0 con 0x7f2d5c004d90 2023-10-14T10:37:14.114 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.094+0000 7f2d75638700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2d700b51f0 0x7f2d700aff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:14.114 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.094+0000 7f2d75638700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2d700b51f0 0x7f2d700aff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46234/0 (socket says 172.21.15.67:46234) 2023-10-14T10:37:14.114 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.094+0000 7f2d75638700 1 -- 172.21.15.67:0/3262700343 learned_addr learned my addr 172.21.15.67:0/3262700343 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:37:14.115 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.094+0000 7f2d7663a700 1 --2- 172.21.15.67:0/3262700343 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2d70159ad0 0x7f2d700b0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:14.115 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.094+0000 7f2d75e39700 1 --2- 172.21.15.67:0/3262700343 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f2d5c004d90 0x7f2d70157d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:14.115 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.094+0000 7f2d75638700 1 -- 172.21.15.67:0/3262700343 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f2d5c004d90 msgr2=0x7f2d70157d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:14.115 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.094+0000 7f2d75638700 1 --2- 172.21.15.67:0/3262700343 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f2d5c004d90 0x7f2d70157d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:14.116 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.094+0000 7f2d75638700 1 -- 172.21.15.67:0/3262700343 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2d70159ad0 msgr2=0x7f2d700b0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:14.116 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.094+0000 7f2d75638700 1 --2- 172.21.15.67:0/3262700343 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2d70159ad0 0x7f2d700b0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:14.116 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.094+0000 7f2d75638700 1 -- 172.21.15.67:0/3262700343 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d7015d7d0 con 0x7f2d700b51f0 2023-10-14T10:37:14.117 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.094+0000 7f2d75e39700 1 --2- 172.21.15.67:0/3262700343 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f2d5c004d90 0x7f2d70157d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-14T10:37:14.117 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.095+0000 7f2d75638700 1 --2- 172.21.15.67:0/3262700343 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2d700b51f0 0x7f2d700aff10 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f2d60011aa0 tx=0x7f2d60011db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:14.117 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.095+0000 7f2d667fc700 1 -- 172.21.15.67:0/3262700343 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6000f940 con 0x7f2d700b51f0 2023-10-14T10:37:14.117 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.095+0000 7f2d667fc700 1 -- 172.21.15.67:0/3262700343 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d6000faa0 con 0x7f2d700b51f0 2023-10-14T10:37:14.118 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.095+0000 7f2d7809d700 1 -- 172.21.15.67:0/3262700343 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d700ac9c0 con 0x7f2d700b51f0 2023-10-14T10:37:14.118 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.095+0000 7f2d667fc700 1 -- 172.21.15.67:0/3262700343 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6000fe50 con 0x7f2d700b51f0 2023-10-14T10:37:14.118 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.095+0000 7f2d7809d700 1 -- 172.21.15.67:0/3262700343 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d7015ce70 con 0x7f2d700b51f0 2023-10-14T10:37:14.118 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.096+0000 7f2d7809d700 1 -- 172.21.15.67:0/3262700343 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d700acbe0 con 0x7f2d700b51f0 2023-10-14T10:37:14.119 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.096+0000 7f2d667fc700 1 -- 172.21.15.67:0/3262700343 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f2d60014440 con 0x7f2d700b51f0 2023-10-14T10:37:14.119 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.097+0000 7f2d667fc700 1 --2- 172.21.15.67:0/3262700343 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f2d5406af00 0x7f2d5406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:14.119 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.097+0000 7f2d667fc700 1 -- 172.21.15.67:0/3262700343 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5729+0+0 (secure 0 0 0) 0x7f2d60089d50 con 0x7f2d700b51f0 2023-10-14T10:37:14.120 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.097+0000 7f2d75e39700 1 --2- 172.21.15.67:0/3262700343 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f2d5406af00 0x7f2d5406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:14.120 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.097+0000 7f2d75e39700 1 --2- 172.21.15.67:0/3262700343 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f2d5406af00 0x7f2d5406d3c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c001eb0 tx=0x7f2d6c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:14.120 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.101+0000 7f2d667fc700 1 -- 172.21.15.67:0/3262700343 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2d6008d030 con 0x7f2d700b51f0 2023-10-14T10:37:14.302 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.300+0000 7f2d7809d700 1 -- 172.21.15.67:0/3262700343 --> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f2d700ccac0 con 0x7f2d5406af00 2023-10-14T10:37:14.302 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.300+0000 7f2d667fc700 1 -- 172.21.15.67:0/3262700343 <== mgr.4100 v2:172.21.15.72:6832/703488382 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f2d6c007da0 con 0x7f2d5406af00 2023-10-14T10:37:14.305 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.302+0000 7f2d7809d700 1 -- 172.21.15.67:0/3262700343 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f2d5406af00 msgr2=0x7f2d5406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:14.305 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.302+0000 7f2d7809d700 1 --2- 172.21.15.67:0/3262700343 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f2d5406af00 0x7f2d5406d3c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c001eb0 tx=0x7f2d6c009040 comp rx=0 tx=0).stop 2023-10-14T10:37:14.305 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.302+0000 7f2d7809d700 1 -- 172.21.15.67:0/3262700343 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2d700b51f0 msgr2=0x7f2d700aff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:14.306 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.302+0000 7f2d7809d700 1 --2- 172.21.15.67:0/3262700343 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2d700b51f0 0x7f2d700aff10 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f2d60011aa0 tx=0x7f2d60011db0 comp rx=0 tx=0).stop 2023-10-14T10:37:14.306 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.302+0000 7f2d7809d700 1 -- 172.21.15.67:0/3262700343 shutdown_connections 2023-10-14T10:37:14.306 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.302+0000 7f2d7809d700 1 --2- 172.21.15.67:0/3262700343 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f2d5406af00 0x7f2d5406d3c0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:14.306 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.302+0000 7f2d7809d700 1 --2- 172.21.15.67:0/3262700343 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2d70159ad0 0x7f2d700b0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:14.307 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.302+0000 7f2d7809d700 1 --2- 172.21.15.67:0/3262700343 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f2d5c004d90 0x7f2d70157d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:14.307 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.302+0000 7f2d7809d700 1 --2- 172.21.15.67:0/3262700343 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2d700b51f0 0x7f2d700aff10 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:14.307 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.302+0000 7f2d7809d700 1 -- 172.21.15.67:0/3262700343 >> 172.21.15.67:0/3262700343 conn(0x7f2d7006dac0 msgr2=0x7f2d700ce7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:37:14.308 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.302+0000 7f2d7809d700 1 -- 172.21.15.67:0/3262700343 shutdown_connections 2023-10-14T10:37:14.308 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.302+0000 7f2d7809d700 1 -- 172.21.15.67:0/3262700343 wait complete. 2023-10-14T10:37:14.308 INFO:teuthology.orchestra.run.smithi067.stderr:instructed osd(s) 6 to deep-scrub 2023-10-14T10:37:14.316 INFO:tasks.ceph:Scrubbing osd.7 2023-10-14T10:37:14.317 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-10-14T10:37:14.447 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.445+0000 7f580eecf700 1 Processor -- start 2023-10-14T10:37:14.447 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.445+0000 7f580eecf700 1 -- start start 2023-10-14T10:37:14.449 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.445+0000 7f580eecf700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f58080727e0 0x7f58080b1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:14.450 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.445+0000 7f580eecf700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f580806da30 0x7f580806dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:14.450 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.445+0000 7f580eecf700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58080b2e40 con 0x7f5808151680 2023-10-14T10:37:14.450 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.445+0000 7f580eecf700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f580806e420 con 0x7f58080727e0 2023-10-14T10:37:14.451 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.445+0000 7f580eecf700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f580815e7c0 con 0x7f580806da30 2023-10-14T10:37:14.451 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.446+0000 7f5807fff700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f58080727e0 0x7f58080b1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:14.451 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.446+0000 7f5807fff700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f58080727e0 0x7f58080b1f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46242/0 (socket says 172.21.15.67:46242) 2023-10-14T10:37:14.451 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.446+0000 7f5807fff700 1 -- 172.21.15.67:0/3625045118 learned_addr learned my addr 172.21.15.67:0/3625045118 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:37:14.452 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.446+0000 7f580d46c700 1 --2- 172.21.15.67:0/3625045118 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f580806da30 0x7f580806dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:14.452 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.446+0000 7f5807fff700 1 -- 172.21.15.67:0/3625045118 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f580806da30 msgr2=0x7f580806dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:37:14.452 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.446+0000 7f5807fff700 1 --2- 172.21.15.67:0/3625045118 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f580806da30 0x7f580806dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:14.453 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.446+0000 7f5807fff700 1 -- 172.21.15.67:0/3625045118 >> v1:172.21.15.72:6789/0 conn(0x7f5808151680 legacy=0x7f580815ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:14.453 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.446+0000 7f5807fff700 1 -- 172.21.15.67:0/3625045118 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58080b2880 con 0x7f58080727e0 2023-10-14T10:37:14.453 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.446+0000 7f5807fff700 1 --2- 172.21.15.67:0/3625045118 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58080727e0 0x7f58080b1f10 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f57f400bb90 tx=0x7f57f400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb0a0561d1a9d370 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:14.453 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.446+0000 7f58077fe700 1 -- 172.21.15.67:0/3625045118 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f400dd50 con 0x7f58080727e0 2023-10-14T10:37:14.454 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.446+0000 7f58077fe700 1 -- 172.21.15.67:0/3625045118 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57f4010470 con 0x7f58080727e0 2023-10-14T10:37:14.454 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.446+0000 7f58077fe700 1 -- 172.21.15.67:0/3625045118 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f4010800 con 0x7f58080727e0 2023-10-14T10:37:14.454 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.447+0000 7f580eecf700 1 -- 172.21.15.67:0/3625045118 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58080727e0 msgr2=0x7f58080b1f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:14.454 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.447+0000 7f580eecf700 1 --2- 172.21.15.67:0/3625045118 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58080727e0 0x7f58080b1f10 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f57f400bb90 tx=0x7f57f400bea0 comp rx=0 tx=0).stop 2023-10-14T10:37:14.455 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.447+0000 7f580eecf700 1 -- 172.21.15.67:0/3625045118 shutdown_connections 2023-10-14T10:37:14.455 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.447+0000 7f580eecf700 1 --2- 172.21.15.67:0/3625045118 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f580806da30 0x7f580806dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:14.455 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.447+0000 7f580eecf700 1 --2- 172.21.15.67:0/3625045118 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58080727e0 0x7f58080b1f10 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:14.455 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.447+0000 7f580eecf700 1 -- 172.21.15.67:0/3625045118 >> 172.21.15.67:0/3625045118 conn(0x7f580806bd70 msgr2=0x7f580806c180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:37:14.456 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.447+0000 7f580eecf700 1 -- 172.21.15.67:0/3625045118 shutdown_connections 2023-10-14T10:37:14.456 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.447+0000 7f580eecf700 1 -- 172.21.15.67:0/3625045118 wait complete. 2023-10-14T10:37:14.459 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.448+0000 7f580eecf700 1 Processor -- start 2023-10-14T10:37:14.460 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.448+0000 7f580eecf700 1 -- start start 2023-10-14T10:37:14.460 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.449+0000 7f580eecf700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f580806da30 0x7f58080cf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:14.460 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.449+0000 7f580eecf700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58080727e0 0x7f580813dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:14.460 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.449+0000 7f580eecf700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5808151680 0x7f58081ee7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:14.461 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.449+0000 7f580eecf700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58080d0c10 con 0x7f58080727e0 2023-10-14T10:37:14.461 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.449+0000 7f580eecf700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f580813e290 con 0x7f5808151680 2023-10-14T10:37:14.461 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.449+0000 7f580eecf700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58080d0000 con 0x7f580806da30 2023-10-14T10:37:14.462 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.449+0000 7f5807fff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58080727e0 0x7f580813dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:14.462 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.449+0000 7f5807fff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58080727e0 0x7f580813dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46248/0 (socket says 172.21.15.67:46248) 2023-10-14T10:37:14.462 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.449+0000 7f5807fff700 1 -- 172.21.15.67:0/195649255 learned_addr learned my addr 172.21.15.67:0/195649255 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:37:14.462 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.449+0000 7f580d46c700 1 --2- 172.21.15.67:0/195649255 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5808151680 0x7f58081ee7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:14.463 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.449+0000 7f5807fff700 1 -- 172.21.15.67:0/195649255 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f580806da30 msgr2=0x7f58080cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:14.463 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.449+0000 7f580cc6b700 1 --2- 172.21.15.67:0/195649255 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f580806da30 0x7f58080cf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:14.463 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.449+0000 7f5807fff700 1 --2- 172.21.15.67:0/195649255 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f580806da30 0x7f58080cf9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:14.463 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.449+0000 7f5807fff700 1 -- 172.21.15.67:0/195649255 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5808151680 msgr2=0x7f58081ee7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:14.464 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.449+0000 7f5807fff700 1 --2- 172.21.15.67:0/195649255 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5808151680 0x7f58081ee7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:14.464 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.449+0000 7f5807fff700 1 -- 172.21.15.67:0/195649255 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57f400b7f0 con 0x7f58080727e0 2023-10-14T10:37:14.464 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.449+0000 7f580d46c700 1 --2- 172.21.15.67:0/195649255 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5808151680 0x7f58081ee7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-14T10:37:14.465 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.449+0000 7f580cc6b700 1 --2- 172.21.15.67:0/195649255 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f580806da30 0x7f58080cf9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-14T10:37:14.465 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.450+0000 7f5807fff700 1 --2- 172.21.15.67:0/195649255 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58080727e0 0x7f580813dd50 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f57f400bee0 tx=0x7f57f4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:14.465 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.450+0000 7f58057fa700 1 -- 172.21.15.67:0/195649255 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f401f070 con 0x7f58080727e0 2023-10-14T10:37:14.465 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.450+0000 7f58057fa700 1 -- 172.21.15.67:0/195649255 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57f40038b0 con 0x7f58080727e0 2023-10-14T10:37:14.466 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.450+0000 7f580eecf700 1 -- 172.21.15.67:0/195649255 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58080d02e0 con 0x7f58080727e0 2023-10-14T10:37:14.466 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.450+0000 7f58057fa700 1 -- 172.21.15.67:0/195649255 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f4003c60 con 0x7f58080727e0 2023-10-14T10:37:14.466 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.450+0000 7f580eecf700 1 -- 172.21.15.67:0/195649255 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58080d08a0 con 0x7f58080727e0 2023-10-14T10:37:14.466 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.450+0000 7f580eecf700 1 -- 172.21.15.67:0/195649255 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f580815e450 con 0x7f58080727e0 2023-10-14T10:37:14.467 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.451+0000 7f58057fa700 1 -- 172.21.15.67:0/195649255 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f57f4019400 con 0x7f58080727e0 2023-10-14T10:37:14.467 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.451+0000 7f58057fa700 1 --2- 172.21.15.67:0/195649255 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f57ec06af00 0x7f57ec06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:14.467 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.452+0000 7f58057fa700 1 -- 172.21.15.67:0/195649255 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5729+0+0 (secure 0 0 0) 0x7f57f408d380 con 0x7f58080727e0 2023-10-14T10:37:14.467 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.452+0000 7f58057fa700 1 --2- 172.21.15.67:0/195649255 >> [v2:172.21.15.72:6824/3567465926,v1:172.21.15.72:6825/3567465926] conn(0x7f57ec073100 0x7f57ec075540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:14.468 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.452+0000 7f58057fa700 1 -- 172.21.15.67:0/195649255 --> [v2:172.21.15.72:6824/3567465926,v1:172.21.15.72:6825/3567465926] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f57ec075c10 con 0x7f57ec073100 2023-10-14T10:37:14.468 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.452+0000 7f58057fa700 1 -- 172.21.15.67:0/195649255 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57f408d7d0 con 0x7f58080727e0 2023-10-14T10:37:14.468 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.452+0000 7f580cc6b700 1 --2- 172.21.15.67:0/195649255 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f57ec06af00 0x7f57ec06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:14.469 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.452+0000 7f580d46c700 1 --2- 172.21.15.67:0/195649255 >> [v2:172.21.15.72:6824/3567465926,v1:172.21.15.72:6825/3567465926] conn(0x7f57ec073100 0x7f57ec075540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:14.469 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.452+0000 7f580cc6b700 1 --2- 172.21.15.67:0/195649255 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f57ec06af00 0x7f57ec06d3c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f57fc000f60 tx=0x7f57fc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:14.469 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.452+0000 7f580d46c700 1 --2- 172.21.15.67:0/195649255 >> [v2:172.21.15.72:6824/3567465926,v1:172.21.15.72:6825/3567465926] conn(0x7f57ec073100 0x7f57ec075540 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:14.469 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.453+0000 7f58057fa700 1 -- 172.21.15.67:0/195649255 <== osd.7 v2:172.21.15.72:6824/3567465926 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7f57ec075c10 con 0x7f57ec073100 2023-10-14T10:37:14.470 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.468+0000 7f580eecf700 1 -- 172.21.15.67:0/195649255 --> [v2:172.21.15.72:6824/3567465926,v1:172.21.15.72:6825/3567465926] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f58080b2b60 con 0x7f57ec073100 2023-10-14T10:37:14.470 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.468+0000 7f58057fa700 1 -- 172.21.15.67:0/195649255 <== osd.7 v2:172.21.15.72:6824/3567465926 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f5800003a20 con 0x7f57ec073100 2023-10-14T10:37:14.471 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2023-10-14T10:37:14.471 INFO:teuthology.orchestra.run.smithi067.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-14T10:37:14.472 INFO:teuthology.orchestra.run.smithi067.stdout:} 2023-10-14T10:37:14.473 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.469+0000 7f580eecf700 1 -- 172.21.15.67:0/195649255 >> [v2:172.21.15.72:6824/3567465926,v1:172.21.15.72:6825/3567465926] conn(0x7f57ec073100 msgr2=0x7f57ec075540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:14.473 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.469+0000 7f580eecf700 1 --2- 172.21.15.67:0/195649255 >> [v2:172.21.15.72:6824/3567465926,v1:172.21.15.72:6825/3567465926] conn(0x7f57ec073100 0x7f57ec075540 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:14.473 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.469+0000 7f580eecf700 1 -- 172.21.15.67:0/195649255 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f57ec06af00 msgr2=0x7f57ec06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:14.473 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.469+0000 7f580eecf700 1 --2- 172.21.15.67:0/195649255 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f57ec06af00 0x7f57ec06d3c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f57fc000f60 tx=0x7f57fc009040 comp rx=0 tx=0).stop 2023-10-14T10:37:14.474 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.469+0000 7f580eecf700 1 -- 172.21.15.67:0/195649255 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58080727e0 msgr2=0x7f580813dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:14.474 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.469+0000 7f580eecf700 1 --2- 172.21.15.67:0/195649255 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58080727e0 0x7f580813dd50 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f57f400bee0 tx=0x7f57f4004750 comp rx=0 tx=0).stop 2023-10-14T10:37:14.474 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.469+0000 7f580d46c700 1 -- 172.21.15.67:0/195649255 reap_dead start 2023-10-14T10:37:14.475 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.469+0000 7f580eecf700 1 -- 172.21.15.67:0/195649255 shutdown_connections 2023-10-14T10:37:14.475 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.469+0000 7f580eecf700 1 -- 172.21.15.67:0/195649255 >> 172.21.15.67:0/195649255 conn(0x7f580806bd70 msgr2=0x7f580806c150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:37:14.475 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.469+0000 7f580eecf700 1 -- 172.21.15.67:0/195649255 shutdown_connections 2023-10-14T10:37:14.475 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.469+0000 7f580eecf700 1 -- 172.21.15.67:0/195649255 wait complete. 2023-10-14T10:37:14.483 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-10-14T10:37:14.609 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.606+0000 7f77dd516700 1 Processor -- start 2023-10-14T10:37:14.609 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.606+0000 7f77dd516700 1 -- start start 2023-10-14T10:37:14.615 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.607+0000 7f77dd516700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f77d815ac90 0x7f77d806bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:14.615 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.607+0000 7f77dd516700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77d806c160 con 0x7f77d80cf1a0 2023-10-14T10:37:14.616 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.607+0000 7f77dd516700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77d80ce100 con 0x7f77d80cdb40 2023-10-14T10:37:14.616 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.607+0000 7f77dd516700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77d80ce3e0 con 0x7f77d815ac90 2023-10-14T10:37:14.616 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.607+0000 7f77d77fe700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f77d80cf1a0 0x7f77d8071280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:54026/0 (socket says 172.21.15.67:54026) 2023-10-14T10:37:14.616 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.607+0000 7f77d77fe700 1 -- 172.21.15.67:0/21097440 learned_addr learned my addr 172.21.15.67:0/21097440 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:37:14.617 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.608+0000 7f77d6ffd700 1 --2- 172.21.15.67:0/21097440 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f77d815ac90 0x7f77d806bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:14.617 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.608+0000 7f77d5ffb700 1 -- 172.21.15.67:0/21097440 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2192872274 0 0) 0x7f77d806c160 con 0x7f77d80cf1a0 2023-10-14T10:37:14.617 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.608+0000 7f77d5ffb700 1 -- 172.21.15.67:0/21097440 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f77c00035b0 con 0x7f77d80cf1a0 2023-10-14T10:37:14.617 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.608+0000 7f77d5ffb700 1 -- 172.21.15.67:0/21097440 <== mon.1 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3699514452 0 0) 0x7f77d80ce100 con 0x7f77d80cdb40 2023-10-14T10:37:14.618 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.608+0000 7f77d5ffb700 1 -- 172.21.15.67:0/21097440 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f77d806c160 con 0x7f77d80cdb40 2023-10-14T10:37:14.618 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.608+0000 7f77d5ffb700 1 -- 172.21.15.67:0/21097440 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 436862269 0 0) 0x7f77c00035b0 con 0x7f77d80cf1a0 2023-10-14T10:37:14.618 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.608+0000 7f77d5ffb700 1 -- 172.21.15.67:0/21097440 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f77d815ac90 msgr2=0x7f77d806bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:37:14.619 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.608+0000 7f77d5ffb700 1 --2- 172.21.15.67:0/21097440 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f77d815ac90 0x7f77d806bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:14.619 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.608+0000 7f77d5ffb700 1 -- 172.21.15.67:0/21097440 >> v1:172.21.15.72:6789/0 conn(0x7f77d80cdb40 legacy=0x7f77d80b2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:14.619 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.608+0000 7f77d5ffb700 1 -- 172.21.15.67:0/21097440 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77d80ce6c0 con 0x7f77d80cf1a0 2023-10-14T10:37:14.619 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.608+0000 7f77d5ffb700 1 -- 172.21.15.67:0/21097440 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3180043755 0 0) 0x7f77d00019f0 con 0x7f77d80cf1a0 2023-10-14T10:37:14.620 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.608+0000 7f77d6ffd700 1 --2- 172.21.15.67:0/21097440 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f77d815ac90 0x7f77d806bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-14T10:37:14.620 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.608+0000 7f77d5ffb700 1 -- 172.21.15.67:0/21097440 >> v1:172.21.15.67:6789/0 conn(0x7f77d80cf1a0 legacy=0x7f77d8071280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:14.620 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.608+0000 7f77d5ffb700 1 --2- 172.21.15.67:0/21097440 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77c0003bc0 0x7f77c0003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:14.620 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.608+0000 7f77d5ffb700 1 --2- 172.21.15.67:0/21097440 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f77d815ac90 0x7f77c0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:14.621 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.608+0000 7f77d5ffb700 1 --2- 172.21.15.67:0/21097440 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f77c0004d90 0x7f77c0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:14.621 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.608+0000 7f77d5ffb700 1 -- 172.21.15.67:0/21097440 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d80ce3e0 con 0x7f77c0003bc0 2023-10-14T10:37:14.621 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.608+0000 7f77d5ffb700 1 -- 172.21.15.67:0/21097440 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77c000bac0 con 0x7f77c0004d90 2023-10-14T10:37:14.621 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.608+0000 7f77d5ffb700 1 -- 172.21.15.67:0/21097440 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77c000bc00 con 0x7f77d815ac90 2023-10-14T10:37:14.622 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.608+0000 7f77d6ffd700 1 --2- 172.21.15.67:0/21097440 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77c0003bc0 0x7f77c0003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:14.622 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.608+0000 7f77d67fc700 1 --2- 172.21.15.67:0/21097440 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f77d815ac90 0x7f77c0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:14.622 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.608+0000 7f77d77fe700 1 --2- 172.21.15.67:0/21097440 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f77c0004d90 0x7f77c0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:14.623 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.609+0000 7f77d6ffd700 1 -- 172.21.15.67:0/21097440 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f77d815ac90 msgr2=0x7f77c0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:37:14.623 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.609+0000 7f77d6ffd700 1 --2- 172.21.15.67:0/21097440 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f77d815ac90 0x7f77c0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:14.623 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.609+0000 7f77d6ffd700 1 -- 172.21.15.67:0/21097440 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f77c0004d90 msgr2=0x7f77c0005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:37:14.623 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.609+0000 7f77d6ffd700 1 --2- 172.21.15.67:0/21097440 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f77c0004d90 0x7f77c0005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:14.624 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.609+0000 7f77d6ffd700 1 -- 172.21.15.67:0/21097440 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77cc0094a0 con 0x7f77c0003bc0 2023-10-14T10:37:14.624 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.609+0000 7f77d6ffd700 1 --2- 172.21.15.67:0/21097440 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77c0003bc0 0x7f77c0003fc0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f77cc009d60 tx=0x7f77c0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9df29e2ccf3be68 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:14.624 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.609+0000 7f77d5ffb700 1 -- 172.21.15.67:0/21097440 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77cc00e9a0 con 0x7f77c0003bc0 2023-10-14T10:37:14.624 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.609+0000 7f77d5ffb700 1 -- 172.21.15.67:0/21097440 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77cc00eb00 con 0x7f77c0003bc0 2023-10-14T10:37:14.625 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.609+0000 7f77d5ffb700 1 -- 172.21.15.67:0/21097440 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77cc00ee90 con 0x7f77c0003bc0 2023-10-14T10:37:14.625 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.609+0000 7f77dd516700 1 -- 172.21.15.67:0/21097440 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77c0003bc0 msgr2=0x7f77c0003fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:14.625 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.609+0000 7f77dd516700 1 --2- 172.21.15.67:0/21097440 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77c0003bc0 0x7f77c0003fc0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f77cc009d60 tx=0x7f77c0005880 comp rx=0 tx=0).stop 2023-10-14T10:37:14.626 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.609+0000 7f77d77fe700 1 -- 172.21.15.67:0/21097440 reap_dead start 2023-10-14T10:37:14.626 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.609+0000 7f77dd516700 1 -- 172.21.15.67:0/21097440 shutdown_connections 2023-10-14T10:37:14.626 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.609+0000 7f77dd516700 1 -- 172.21.15.67:0/21097440 >> 172.21.15.67:0/21097440 conn(0x7f77d80cfc40 msgr2=0x7f77d80d0020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:37:14.626 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.610+0000 7f77dd516700 1 -- 172.21.15.67:0/21097440 shutdown_connections 2023-10-14T10:37:14.627 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.610+0000 7f77dd516700 1 -- 172.21.15.67:0/21097440 wait complete. 2023-10-14T10:37:14.627 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.611+0000 7f77dd516700 1 Processor -- start 2023-10-14T10:37:14.627 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.611+0000 7f77dd516700 1 -- start start 2023-10-14T10:37:14.627 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.612+0000 7f77dd516700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f77c0003bc0 0x7f77d814b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:14.628 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.612+0000 7f77dd516700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f77d80c6d10 0x7f77d80c8f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:14.628 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.612+0000 7f77dd516700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77d80c7190 0x7f77d80c9480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:14.628 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.612+0000 7f77dd516700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d80c8790 con 0x7f77d80c7190 2023-10-14T10:37:14.628 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.612+0000 7f77dd516700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d80c76f0 con 0x7f77c0003bc0 2023-10-14T10:37:14.629 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.612+0000 7f77dd516700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77d80c79d0 con 0x7f77d80c6d10 2023-10-14T10:37:14.629 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.612+0000 7f77d77fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77d80c7190 0x7f77d80c9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:14.629 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.612+0000 7f77d77fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77d80c7190 0x7f77d80c9480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46270/0 (socket says 172.21.15.67:46270) 2023-10-14T10:37:14.630 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.612+0000 7f77d6ffd700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f77c0003bc0 0x7f77d814b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:14.630 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.612+0000 7f77d77fe700 1 -- 172.21.15.67:0/318348308 learned_addr learned my addr 172.21.15.67:0/318348308 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:37:14.630 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.612+0000 7f77d67fc700 1 --2- 172.21.15.67:0/318348308 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f77d80c6d10 0x7f77d80c8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:14.630 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.612+0000 7f77d77fe700 1 -- 172.21.15.67:0/318348308 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f77d80c6d10 msgr2=0x7f77d80c8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:14.631 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.612+0000 7f77d77fe700 1 --2- 172.21.15.67:0/318348308 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f77d80c6d10 0x7f77d80c8f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:14.631 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.612+0000 7f77d77fe700 1 -- 172.21.15.67:0/318348308 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f77c0003bc0 msgr2=0x7f77d814b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:14.631 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.612+0000 7f77d77fe700 1 --2- 172.21.15.67:0/318348308 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f77c0003bc0 0x7f77d814b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:14.631 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.612+0000 7f77d77fe700 1 -- 172.21.15.67:0/318348308 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77d80c7cb0 con 0x7f77d80c7190 2023-10-14T10:37:14.632 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.612+0000 7f77d77fe700 1 --2- 172.21.15.67:0/318348308 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77d80c7190 0x7f77d80c9480 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f77d0000f80 tx=0x7f77d000a5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:14.632 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.612+0000 7f77bf7fe700 1 -- 172.21.15.67:0/318348308 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d000ad20 con 0x7f77d80c7190 2023-10-14T10:37:14.632 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.613+0000 7f77bf7fe700 1 -- 172.21.15.67:0/318348308 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77d000ae80 con 0x7f77d80c7190 2023-10-14T10:37:14.632 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.613+0000 7f77dd516700 1 -- 172.21.15.67:0/318348308 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77d80c7f90 con 0x7f77d80c7190 2023-10-14T10:37:14.633 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.613+0000 7f77bf7fe700 1 -- 172.21.15.67:0/318348308 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d0013630 con 0x7f77d80c7190 2023-10-14T10:37:14.633 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.613+0000 7f77dd516700 1 -- 172.21.15.67:0/318348308 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77d814bb00 con 0x7f77d80c7190 2023-10-14T10:37:14.634 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.613+0000 7f77dd516700 1 -- 172.21.15.67:0/318348308 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77d80c84b0 con 0x7f77d80c7190 2023-10-14T10:37:14.635 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.614+0000 7f77bf7fe700 1 -- 172.21.15.67:0/318348308 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f77d0013790 con 0x7f77d80c7190 2023-10-14T10:37:14.635 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.615+0000 7f77bf7fe700 1 --2- 172.21.15.67:0/318348308 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f77c406af00 0x7f77c406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:14.635 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.615+0000 7f77bf7fe700 1 -- 172.21.15.67:0/318348308 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5729+0+0 (secure 0 0 0) 0x7f77d008a6e0 con 0x7f77d80c7190 2023-10-14T10:37:14.635 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.615+0000 7f77d6ffd700 1 --2- 172.21.15.67:0/318348308 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f77c406af00 0x7f77c406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:14.636 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.615+0000 7f77d6ffd700 1 --2- 172.21.15.67:0/318348308 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f77c406af00 0x7f77c406d3c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f77cc0090b0 tx=0x7f77cc008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:14.636 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.619+0000 7f77bf7fe700 1 -- 172.21.15.67:0/318348308 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f77d00557c0 con 0x7f77d80c7190 2023-10-14T10:37:14.821 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.819+0000 7f77dd516700 1 -- 172.21.15.67:0/318348308 --> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f77d80cec80 con 0x7f77c406af00 2023-10-14T10:37:14.822 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.820+0000 7f77bf7fe700 1 -- 172.21.15.67:0/318348308 <== mgr.4100 v2:172.21.15.72:6832/703488382 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f77cc022040 con 0x7f77c406af00 2023-10-14T10:37:14.824 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.821+0000 7f77dd516700 1 -- 172.21.15.67:0/318348308 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f77c406af00 msgr2=0x7f77c406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:14.824 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.821+0000 7f77dd516700 1 --2- 172.21.15.67:0/318348308 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f77c406af00 0x7f77c406d3c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f77cc0090b0 tx=0x7f77cc008030 comp rx=0 tx=0).stop 2023-10-14T10:37:14.824 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.821+0000 7f77dd516700 1 -- 172.21.15.67:0/318348308 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77d80c7190 msgr2=0x7f77d80c9480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:14.825 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.821+0000 7f77dd516700 1 --2- 172.21.15.67:0/318348308 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77d80c7190 0x7f77d80c9480 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f77d0000f80 tx=0x7f77d000a5b0 comp rx=0 tx=0).stop 2023-10-14T10:37:14.825 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.821+0000 7f77dd516700 1 -- 172.21.15.67:0/318348308 shutdown_connections 2023-10-14T10:37:14.825 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.821+0000 7f77dd516700 1 --2- 172.21.15.67:0/318348308 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f77c406af00 0x7f77c406d3c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:14.826 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.821+0000 7f77dd516700 1 --2- 172.21.15.67:0/318348308 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77d80c7190 0x7f77d80c9480 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:14.826 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.821+0000 7f77dd516700 1 --2- 172.21.15.67:0/318348308 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f77c0003bc0 0x7f77d814b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:14.826 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.821+0000 7f77dd516700 1 --2- 172.21.15.67:0/318348308 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f77d80c6d10 0x7f77d80c8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:14.826 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.821+0000 7f77dd516700 1 -- 172.21.15.67:0/318348308 >> 172.21.15.67:0/318348308 conn(0x7f77d80cfc40 msgr2=0x7f77d80cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:37:14.827 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.822+0000 7f77dd516700 1 -- 172.21.15.67:0/318348308 shutdown_connections 2023-10-14T10:37:14.827 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.822+0000 7f77dd516700 1 -- 172.21.15.67:0/318348308 wait complete. 2023-10-14T10:37:14.828 INFO:teuthology.orchestra.run.smithi067.stderr:instructed osd(s) 7 to deep-scrub 2023-10-14T10:37:14.836 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T10:37:14.962 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.960+0000 7f6fde012700 1 Processor -- start 2023-10-14T10:37:14.962 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.960+0000 7f6fde012700 1 -- start start 2023-10-14T10:37:14.966 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.960+0000 7f6fde012700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f6fd80b2980 0x7f6fd806fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:14.967 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.960+0000 7f6fde012700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f6fd80728e0 0x7f6fd815e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:14.967 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.960+0000 7f6fde012700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fd8070040 con 0x7f6fd815a8e0 2023-10-14T10:37:14.967 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.960+0000 7f6fde012700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fd80d0420 con 0x7f6fd80b2980 2023-10-14T10:37:14.967 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.960+0000 7f6fde012700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd80afce0 con 0x7f6fd80728e0 2023-10-14T10:37:14.968 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.961+0000 7f6fd77fe700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f6fd815a8e0 0x7f6fd80b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:54042/0 (socket says 172.21.15.67:54042) 2023-10-14T10:37:14.968 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.961+0000 7f6fd77fe700 1 -- 172.21.15.67:0/393863184 learned_addr learned my addr 172.21.15.67:0/393863184 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:37:14.968 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.961+0000 7f6fd7fff700 1 --2- 172.21.15.67:0/393863184 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f6fd80728e0 0x7f6fd815e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:14.968 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.961+0000 7f6fd67fc700 1 -- 172.21.15.67:0/393863184 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2366399247 0 0) 0x7f6fd8070040 con 0x7f6fd815a8e0 2023-10-14T10:37:14.969 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.961+0000 7f6fd6ffd700 1 --2- 172.21.15.67:0/393863184 >> v2:172.21.15.72:3300/0 conn(0x7f6fd80b2980 0x7f6fd806fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:14.969 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.961+0000 7f6fd67fc700 1 -- 172.21.15.67:0/393863184 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fc00035b0 con 0x7f6fd815a8e0 2023-10-14T10:37:14.969 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.961+0000 7f6fd67fc700 1 -- 172.21.15.67:0/393863184 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3962551515 0 0) 0x7f6fc00035b0 con 0x7f6fd815a8e0 2023-10-14T10:37:14.970 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.961+0000 7f6fd67fc700 1 -- 172.21.15.67:0/393863184 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f6fd80728e0 msgr2=0x7f6fd815e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:37:14.970 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.961+0000 7f6fd67fc700 1 --2- 172.21.15.67:0/393863184 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f6fd80728e0 0x7f6fd815e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:14.970 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.961+0000 7f6fd67fc700 1 -- 172.21.15.67:0/393863184 >> v2:172.21.15.72:3300/0 conn(0x7f6fd80b2980 msgr2=0x7f6fd806fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:37:14.971 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.961+0000 7f6fd67fc700 1 --2- 172.21.15.67:0/393863184 >> v2:172.21.15.72:3300/0 conn(0x7f6fd80b2980 0x7f6fd806fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:14.971 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.961+0000 7f6fd67fc700 1 -- 172.21.15.67:0/393863184 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fd8071e00 con 0x7f6fd815a8e0 2023-10-14T10:37:14.971 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.961+0000 7f6fd67fc700 1 -- 172.21.15.67:0/393863184 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3180043755 0 0) 0x7f6fc80017f0 con 0x7f6fd815a8e0 2023-10-14T10:37:14.971 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.961+0000 7f6fd67fc700 1 -- 172.21.15.67:0/393863184 >> v1:172.21.15.67:6789/0 conn(0x7f6fd815a8e0 legacy=0x7f6fd80b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:14.972 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.961+0000 7f6fd67fc700 1 --2- 172.21.15.67:0/393863184 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6fc0003a00 0x7f6fc0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:14.972 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.961+0000 7f6fd67fc700 1 --2- 172.21.15.67:0/393863184 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f6fd80728e0 0x7f6fc0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:14.972 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.961+0000 7f6fd67fc700 1 --2- 172.21.15.67:0/393863184 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6fc0004d90 0x7f6fc0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:14.973 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.961+0000 7f6fd67fc700 1 -- 172.21.15.67:0/393863184 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd80d0420 con 0x7f6fc0004d90 2023-10-14T10:37:14.973 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.961+0000 7f6fd67fc700 1 -- 172.21.15.67:0/393863184 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd80afce0 con 0x7f6fc0003a00 2023-10-14T10:37:14.973 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.961+0000 7f6fd67fc700 1 -- 172.21.15.67:0/393863184 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc000ba10 con 0x7f6fd80728e0 2023-10-14T10:37:14.973 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.961+0000 7f6fd7fff700 1 --2- 172.21.15.67:0/393863184 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6fc0004d90 0x7f6fc0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:14.974 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.961+0000 7f6fd77fe700 1 --2- 172.21.15.67:0/393863184 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6fc0003a00 0x7f6fc0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:14.974 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.961+0000 7f6fd6ffd700 1 --2- 172.21.15.67:0/393863184 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f6fd80728e0 0x7f6fc0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:14.974 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.962+0000 7f6fd7fff700 1 -- 172.21.15.67:0/393863184 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f6fd80728e0 msgr2=0x7f6fc0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:37:14.975 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.962+0000 7f6fd7fff700 1 --2- 172.21.15.67:0/393863184 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f6fd80728e0 0x7f6fc0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:14.975 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.962+0000 7f6fd7fff700 1 -- 172.21.15.67:0/393863184 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6fc0003a00 msgr2=0x7f6fc0003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:37:14.975 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.962+0000 7f6fd7fff700 1 --2- 172.21.15.67:0/393863184 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6fc0003a00 0x7f6fc0003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:14.976 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.962+0000 7f6fd7fff700 1 -- 172.21.15.67:0/393863184 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fc00035b0 con 0x7f6fc0004d90 2023-10-14T10:37:14.976 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.962+0000 7f6fd7fff700 1 --2- 172.21.15.67:0/393863184 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6fc0004d90 0x7f6fc0005210 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f6fd0009bd0 tx=0x7f6fc0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f8b455e02c2e1d5 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:14.976 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.962+0000 7f6fd67fc700 1 -- 172.21.15.67:0/393863184 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd000eb10 con 0x7f6fc0004d90 2023-10-14T10:37:14.976 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.962+0000 7f6fd67fc700 1 -- 172.21.15.67:0/393863184 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fd000f040 con 0x7f6fc0004d90 2023-10-14T10:37:14.977 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.962+0000 7f6fd67fc700 1 -- 172.21.15.67:0/393863184 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd000eeb0 con 0x7f6fc0004d90 2023-10-14T10:37:14.977 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.962+0000 7f6fde012700 1 -- 172.21.15.67:0/393863184 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6fc0004d90 msgr2=0x7f6fc0005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:14.977 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.962+0000 7f6fde012700 1 --2- 172.21.15.67:0/393863184 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6fc0004d90 0x7f6fc0005210 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f6fd0009bd0 tx=0x7f6fc0005880 comp rx=0 tx=0).stop 2023-10-14T10:37:14.978 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.962+0000 7f6fd7fff700 1 -- 172.21.15.67:0/393863184 reap_dead start 2023-10-14T10:37:14.978 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.962+0000 7f6fde012700 1 -- 172.21.15.67:0/393863184 shutdown_connections 2023-10-14T10:37:14.978 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.962+0000 7f6fde012700 1 -- 172.21.15.67:0/393863184 >> 172.21.15.67:0/393863184 conn(0x7f6fd80cd270 msgr2=0x7f6fd80cd650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:37:14.978 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.963+0000 7f6fde012700 1 -- 172.21.15.67:0/393863184 shutdown_connections 2023-10-14T10:37:14.979 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.963+0000 7f6fde012700 1 -- 172.21.15.67:0/393863184 wait complete. 2023-10-14T10:37:14.979 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.964+0000 7f6fde012700 1 Processor -- start 2023-10-14T10:37:14.979 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.964+0000 7f6fde012700 1 -- start start 2023-10-14T10:37:14.982 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.965+0000 7f6fde012700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f6fd80c9940 0x7f6fd80c9d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:14.983 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.965+0000 7f6fde012700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6fd80c7830 0x7f6fd815bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:14.983 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.965+0000 7f6fde012700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6fd815c2a0 0x7f6fd80b41d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:14.983 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.965+0000 7f6fde012700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd80c87d0 con 0x7f6fd815c2a0 2023-10-14T10:37:14.983 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.965+0000 7f6fde012700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd80c8210 con 0x7f6fd80c7830 2023-10-14T10:37:14.984 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.965+0000 7f6fde012700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd80c84f0 con 0x7f6fd80c9940 2023-10-14T10:37:14.984 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.965+0000 7f6fd7fff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6fd815c2a0 0x7f6fd80b41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:14.984 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.965+0000 7f6fd7fff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6fd815c2a0 0x7f6fd80b41d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46290/0 (socket says 172.21.15.67:46290) 2023-10-14T10:37:14.985 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.965+0000 7f6fd7fff700 1 -- 172.21.15.67:0/1006326183 learned_addr learned my addr 172.21.15.67:0/1006326183 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:37:14.985 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.966+0000 7f6fd6ffd700 1 --2- 172.21.15.67:0/1006326183 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6fd80c7830 0x7f6fd815bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:14.985 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.966+0000 7f6fd77fe700 1 --2- 172.21.15.67:0/1006326183 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f6fd80c9940 0x7f6fd80c9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:14.986 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.966+0000 7f6fd7fff700 1 -- 172.21.15.67:0/1006326183 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f6fd80c9940 msgr2=0x7f6fd80c9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:14.986 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.966+0000 7f6fd7fff700 1 --2- 172.21.15.67:0/1006326183 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f6fd80c9940 0x7f6fd80c9d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:14.986 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.966+0000 7f6fd7fff700 1 -- 172.21.15.67:0/1006326183 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6fd80c7830 msgr2=0x7f6fd815bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:14.986 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.966+0000 7f6fd7fff700 1 --2- 172.21.15.67:0/1006326183 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6fd80c7830 0x7f6fd815bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:14.987 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.966+0000 7f6fd7fff700 1 -- 172.21.15.67:0/1006326183 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fd815b560 con 0x7f6fd815c2a0 2023-10-14T10:37:14.987 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.966+0000 7f6fd6ffd700 1 --2- 172.21.15.67:0/1006326183 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6fd80c7830 0x7f6fd815bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-14T10:37:14.987 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.966+0000 7f6fd7fff700 1 --2- 172.21.15.67:0/1006326183 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6fd815c2a0 0x7f6fd80b41d0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f6fc0004710 tx=0x7f6fd0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:14.988 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.966+0000 7f6fbffff700 1 -- 172.21.15.67:0/1006326183 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd0009410 con 0x7f6fd815c2a0 2023-10-14T10:37:14.988 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.966+0000 7f6fbffff700 1 -- 172.21.15.67:0/1006326183 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fd0003a20 con 0x7f6fd815c2a0 2023-10-14T10:37:14.988 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.966+0000 7f6fde012700 1 -- 172.21.15.67:0/1006326183 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fd815b840 con 0x7f6fd815c2a0 2023-10-14T10:37:14.988 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.966+0000 7f6fbffff700 1 -- 172.21.15.67:0/1006326183 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd0003dd0 con 0x7f6fd815c2a0 2023-10-14T10:37:14.989 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.966+0000 7f6fde012700 1 -- 172.21.15.67:0/1006326183 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fd815dde0 con 0x7f6fd815c2a0 2023-10-14T10:37:14.989 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.967+0000 7f6fde012700 1 -- 172.21.15.67:0/1006326183 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fd815b3c0 con 0x7f6fd815c2a0 2023-10-14T10:37:14.989 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.968+0000 7f6fbffff700 1 -- 172.21.15.67:0/1006326183 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f6fd0004030 con 0x7f6fd815c2a0 2023-10-14T10:37:14.990 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.968+0000 7f6fbffff700 1 --2- 172.21.15.67:0/1006326183 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f6fc406af00 0x7f6fc406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:14.990 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.968+0000 7f6fbffff700 1 -- 172.21.15.67:0/1006326183 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5729+0+0 (secure 0 0 0) 0x7f6fd008e4d0 con 0x7f6fd815c2a0 2023-10-14T10:37:14.990 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.968+0000 7f6fd77fe700 1 --2- 172.21.15.67:0/1006326183 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f6fc406af00 0x7f6fc406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:14.990 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.969+0000 7f6fd77fe700 1 --2- 172.21.15.67:0/1006326183 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f6fc406af00 0x7f6fc406d3c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6fc8001e40 tx=0x7f6fc8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:14.991 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:14.972+0000 7f6fbffff700 1 -- 172.21.15.67:0/1006326183 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6fd815b3c0 con 0x7f6fd815c2a0 2023-10-14T10:37:15.172 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:15.170+0000 7f6fde012700 1 -- 172.21.15.67:0/1006326183 --> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6fd80723c0 con 0x7f6fc406af00 2023-10-14T10:37:15.181 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:15.179+0000 7f6fbffff700 1 -- 172.21.15.67:0/1006326183 <== mgr.4100 v2:172.21.15.72:6832/703488382 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+288621 (secure 0 0 0) 0x7f6fc8007e00 con 0x7f6fc406af00 2023-10-14T10:37:15.185 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-14T10:37:15.187 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:15.182+0000 7f6fde012700 1 -- 172.21.15.67:0/1006326183 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f6fc406af00 msgr2=0x7f6fc406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:15.187 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:15.182+0000 7f6fde012700 1 --2- 172.21.15.67:0/1006326183 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f6fc406af00 0x7f6fc406d3c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6fc8001e40 tx=0x7f6fc8009040 comp rx=0 tx=0).stop 2023-10-14T10:37:15.188 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:15.182+0000 7f6fde012700 1 -- 172.21.15.67:0/1006326183 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6fd815c2a0 msgr2=0x7f6fd80b41d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:15.188 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:15.182+0000 7f6fde012700 1 --2- 172.21.15.67:0/1006326183 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6fd815c2a0 0x7f6fd80b41d0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f6fc0004710 tx=0x7f6fd0009750 comp rx=0 tx=0).stop 2023-10-14T10:37:15.188 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:15.182+0000 7f6fde012700 1 -- 172.21.15.67:0/1006326183 shutdown_connections 2023-10-14T10:37:15.188 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:15.182+0000 7f6fde012700 1 --2- 172.21.15.67:0/1006326183 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f6fc406af00 0x7f6fc406d3c0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:15.189 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:15.182+0000 7f6fde012700 1 --2- 172.21.15.67:0/1006326183 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6fd815c2a0 0x7f6fd80b41d0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:15.189 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:15.182+0000 7f6fde012700 1 --2- 172.21.15.67:0/1006326183 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f6fd80c9940 0x7f6fd80c9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:15.189 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:15.182+0000 7f6fde012700 1 --2- 172.21.15.67:0/1006326183 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6fd80c7830 0x7f6fd815bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:15.189 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:15.182+0000 7f6fde012700 1 -- 172.21.15.67:0/1006326183 >> 172.21.15.67:0/1006326183 conn(0x7f6fd80cd270 msgr2=0x7f6fd80b0e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:37:15.190 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:15.183+0000 7f6fde012700 1 -- 172.21.15.67:0/1006326183 shutdown_connections 2023-10-14T10:37:15.190 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:15.183+0000 7f6fde012700 1 -- 172.21.15.67:0/1006326183 wait complete. 2023-10-14T10:37:15.190 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2023-10-14T10:37:15.197 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":65,"stamp":"2023-10-14T10:37:13.772184+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220280,"kb_used_data":4104,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754440,"statfs":{"total":773094113280,"available":772868546560,"internally_reserved":0,"allocated":4202496,"data_stored":2322644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12742,"internal_metadata":221236794},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003071"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.787560+0000","last_change":"2023-10-14T10:37:00.075102+0000","last_active":"2023-10-14T10:37:01.787560+0000","last_peered":"2023-10-14T10:37:01.787560+0000","last_clean":"2023-10-14T10:37:01.787560+0000","last_became_active":"2023-10-14T10:37:00.074851+0000","last_became_peered":"2023-10-14T10:37:00.074851+0000","last_unstale":"2023-10-14T10:37:01.787560+0000","last_undegraded":"2023-10-14T10:37:01.787560+0000","last_fullsized":"2023-10-14T10:37:01.787560+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T21:39:35.587728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788402+0000","last_change":"2023-10-14T10:37:00.068871+0000","last_active":"2023-10-14T10:37:01.788402+0000","last_peered":"2023-10-14T10:37:01.788402+0000","last_clean":"2023-10-14T10:37:01.788402+0000","last_became_active":"2023-10-14T10:37:00.068586+0000","last_became_peered":"2023-10-14T10:37:00.068586+0000","last_unstale":"2023-10-14T10:37:01.788402+0000","last_undegraded":"2023-10-14T10:37:01.788402+0000","last_fullsized":"2023-10-14T10:37:01.788402+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T12:47:24.725885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.062215+0000","last_change":"2023-10-14T10:37:00.080988+0000","last_active":"2023-10-14T10:37:01.062215+0000","last_peered":"2023-10-14T10:37:01.062215+0000","last_clean":"2023-10-14T10:37:01.062215+0000","last_became_active":"2023-10-14T10:37:00.080476+0000","last_became_peered":"2023-10-14T10:37:00.080476+0000","last_unstale":"2023-10-14T10:37:01.062215+0000","last_undegraded":"2023-10-14T10:37:01.062215+0000","last_fullsized":"2023-10-14T10:37:01.062215+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T10:48:03.043383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.820430+0000","last_change":"2023-10-14T10:37:00.077598+0000","last_active":"2023-10-14T10:37:01.820430+0000","last_peered":"2023-10-14T10:37:01.820430+0000","last_clean":"2023-10-14T10:37:01.820430+0000","last_became_active":"2023-10-14T10:37:00.077322+0000","last_became_peered":"2023-10-14T10:37:00.077322+0000","last_unstale":"2023-10-14T10:37:01.820430+0000","last_undegraded":"2023-10-14T10:37:01.820430+0000","last_fullsized":"2023-10-14T10:37:01.820430+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788360+0000","last_change":"2023-10-14T10:37:00.068342+0000","last_active":"2023-10-14T10:37:01.788360+0000","last_peered":"2023-10-14T10:37:01.788360+0000","last_clean":"2023-10-14T10:37:01.788360+0000","last_became_active":"2023-10-14T10:37:00.068055+0000","last_became_peered":"2023-10-14T10:37:00.068055+0000","last_unstale":"2023-10-14T10:37:01.788360+0000","last_undegraded":"2023-10-14T10:37:01.788360+0000","last_fullsized":"2023-10-14T10:37:01.788360+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T12:52:16.372161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788917+0000","last_change":"2023-10-14T10:37:00.072629+0000","last_active":"2023-10-14T10:37:01.788917+0000","last_peered":"2023-10-14T10:37:01.788917+0000","last_clean":"2023-10-14T10:37:01.788917+0000","last_became_active":"2023-10-14T10:37:00.072345+0000","last_became_peered":"2023-10-14T10:37:00.072345+0000","last_unstale":"2023-10-14T10:37:01.788917+0000","last_undegraded":"2023-10-14T10:37:01.788917+0000","last_fullsized":"2023-10-14T10:37:01.788917+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T17:09:21.565688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.061635+0000","last_change":"2023-10-14T10:37:00.073019+0000","last_active":"2023-10-14T10:37:01.061635+0000","last_peered":"2023-10-14T10:37:01.061635+0000","last_clean":"2023-10-14T10:37:01.061635+0000","last_became_active":"2023-10-14T10:37:00.072368+0000","last_became_peered":"2023-10-14T10:37:00.072368+0000","last_unstale":"2023-10-14T10:37:01.061635+0000","last_undegraded":"2023-10-14T10:37:01.061635+0000","last_fullsized":"2023-10-14T10:37:01.061635+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:31:16.127752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.062210+0000","last_change":"2023-10-14T10:37:00.074091+0000","last_active":"2023-10-14T10:37:01.062210+0000","last_peered":"2023-10-14T10:37:01.062210+0000","last_clean":"2023-10-14T10:37:01.062210+0000","last_became_active":"2023-10-14T10:37:00.073792+0000","last_became_peered":"2023-10-14T10:37:00.073792+0000","last_unstale":"2023-10-14T10:37:01.062210+0000","last_undegraded":"2023-10-14T10:37:01.062210+0000","last_fullsized":"2023-10-14T10:37:01.062210+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T17:27:50.406084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.489588+0000","last_change":"2023-10-14T10:37:00.081745+0000","last_active":"2023-10-14T10:37:01.489588+0000","last_peered":"2023-10-14T10:37:01.489588+0000","last_clean":"2023-10-14T10:37:01.489588+0000","last_became_active":"2023-10-14T10:37:00.081451+0000","last_became_peered":"2023-10-14T10:37:00.081451+0000","last_unstale":"2023-10-14T10:37:01.489588+0000","last_undegraded":"2023-10-14T10:37:01.489588+0000","last_fullsized":"2023-10-14T10:37:01.489588+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:18:45.297555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.820490+0000","last_change":"2023-10-14T10:37:00.071138+0000","last_active":"2023-10-14T10:37:01.820490+0000","last_peered":"2023-10-14T10:37:01.820490+0000","last_clean":"2023-10-14T10:37:01.820490+0000","last_became_active":"2023-10-14T10:37:00.070667+0000","last_became_peered":"2023-10-14T10:37:00.070667+0000","last_unstale":"2023-10-14T10:37:01.820490+0000","last_undegraded":"2023-10-14T10:37:01.820490+0000","last_fullsized":"2023-10-14T10:37:01.820490+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.062228+0000","last_change":"2023-10-14T10:37:00.073322+0000","last_active":"2023-10-14T10:37:01.062228+0000","last_peered":"2023-10-14T10:37:01.062228+0000","last_clean":"2023-10-14T10:37:01.062228+0000","last_became_active":"2023-10-14T10:37:00.072639+0000","last_became_peered":"2023-10-14T10:37:00.072639+0000","last_unstale":"2023-10-14T10:37:01.062228+0000","last_undegraded":"2023-10-14T10:37:01.062228+0000","last_fullsized":"2023-10-14T10:37:01.062228+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T18:57:50.136605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.061858+0000","last_change":"2023-10-14T10:37:00.069681+0000","last_active":"2023-10-14T10:37:01.061858+0000","last_peered":"2023-10-14T10:37:01.061858+0000","last_clean":"2023-10-14T10:37:01.061858+0000","last_became_active":"2023-10-14T10:37:00.069373+0000","last_became_peered":"2023-10-14T10:37:00.069373+0000","last_unstale":"2023-10-14T10:37:01.061858+0000","last_undegraded":"2023-10-14T10:37:01.061858+0000","last_fullsized":"2023-10-14T10:37:01.061858+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:31:16.127752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788687+0000","last_change":"2023-10-14T10:37:00.074917+0000","last_active":"2023-10-14T10:37:01.788687+0000","last_peered":"2023-10-14T10:37:01.788687+0000","last_clean":"2023-10-14T10:37:01.788687+0000","last_became_active":"2023-10-14T10:37:00.074473+0000","last_became_peered":"2023-10-14T10:37:00.074473+0000","last_unstale":"2023-10-14T10:37:01.788687+0000","last_undegraded":"2023-10-14T10:37:01.788687+0000","last_fullsized":"2023-10-14T10:37:01.788687+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T20:20:02.187767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788602+0000","last_change":"2023-10-14T10:37:00.073986+0000","last_active":"2023-10-14T10:37:01.788602+0000","last_peered":"2023-10-14T10:37:01.788602+0000","last_clean":"2023-10-14T10:37:01.788602+0000","last_became_active":"2023-10-14T10:37:00.073651+0000","last_became_peered":"2023-10-14T10:37:00.073651+0000","last_unstale":"2023-10-14T10:37:01.788602+0000","last_undegraded":"2023-10-14T10:37:01.788602+0000","last_fullsized":"2023-10-14T10:37:01.788602+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T10:48:03.043383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.061624+0000","last_change":"2023-10-14T10:37:00.072277+0000","last_active":"2023-10-14T10:37:01.061624+0000","last_peered":"2023-10-14T10:37:01.061624+0000","last_clean":"2023-10-14T10:37:01.061624+0000","last_became_active":"2023-10-14T10:37:00.071999+0000","last_became_peered":"2023-10-14T10:37:00.071999+0000","last_unstale":"2023-10-14T10:37:01.061624+0000","last_undegraded":"2023-10-14T10:37:01.061624+0000","last_fullsized":"2023-10-14T10:37:01.061624+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:18:45.297555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.062458+0000","last_change":"2023-10-14T10:37:00.072492+0000","last_active":"2023-10-14T10:37:01.062458+0000","last_peered":"2023-10-14T10:37:01.062458+0000","last_clean":"2023-10-14T10:37:01.062458+0000","last_became_active":"2023-10-14T10:37:00.071509+0000","last_became_peered":"2023-10-14T10:37:00.071509+0000","last_unstale":"2023-10-14T10:37:01.062458+0000","last_undegraded":"2023-10-14T10:37:01.062458+0000","last_fullsized":"2023-10-14T10:37:01.062458+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:51:00.167498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.062381+0000","last_change":"2023-10-14T10:37:00.072597+0000","last_active":"2023-10-14T10:37:01.062381+0000","last_peered":"2023-10-14T10:37:01.062381+0000","last_clean":"2023-10-14T10:37:01.062381+0000","last_became_active":"2023-10-14T10:37:00.071946+0000","last_became_peered":"2023-10-14T10:37:00.071946+0000","last_unstale":"2023-10-14T10:37:01.062381+0000","last_undegraded":"2023-10-14T10:37:01.062381+0000","last_fullsized":"2023-10-14T10:37:01.062381+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:06:28.391046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788267+0000","last_change":"2023-10-14T10:37:00.080581+0000","last_active":"2023-10-14T10:37:01.788267+0000","last_peered":"2023-10-14T10:37:01.788267+0000","last_clean":"2023-10-14T10:37:01.788267+0000","last_became_active":"2023-10-14T10:37:00.080288+0000","last_became_peered":"2023-10-14T10:37:00.080288+0000","last_unstale":"2023-10-14T10:37:01.788267+0000","last_undegraded":"2023-10-14T10:37:01.788267+0000","last_fullsized":"2023-10-14T10:37:01.788267+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T18:57:50.136605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.787886+0000","last_change":"2023-10-14T10:37:00.080484+0000","last_active":"2023-10-14T10:37:01.787886+0000","last_peered":"2023-10-14T10:37:01.787886+0000","last_clean":"2023-10-14T10:37:01.787886+0000","last_became_active":"2023-10-14T10:37:00.080186+0000","last_became_peered":"2023-10-14T10:37:00.080186+0000","last_unstale":"2023-10-14T10:37:01.787886+0000","last_undegraded":"2023-10-14T10:37:01.787886+0000","last_fullsized":"2023-10-14T10:37:01.787886+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:17:54.730152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788501+0000","last_change":"2023-10-14T10:37:00.072056+0000","last_active":"2023-10-14T10:37:01.788501+0000","last_peered":"2023-10-14T10:37:01.788501+0000","last_clean":"2023-10-14T10:37:01.788501+0000","last_became_active":"2023-10-14T10:37:00.071796+0000","last_became_peered":"2023-10-14T10:37:00.071796+0000","last_unstale":"2023-10-14T10:37:01.788501+0000","last_undegraded":"2023-10-14T10:37:01.788501+0000","last_fullsized":"2023-10-14T10:37:01.788501+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T20:39:07.030985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.489206+0000","last_change":"2023-10-14T10:37:00.080796+0000","last_active":"2023-10-14T10:37:01.489206+0000","last_peered":"2023-10-14T10:37:01.489206+0000","last_clean":"2023-10-14T10:37:01.489206+0000","last_became_active":"2023-10-14T10:37:00.080526+0000","last_became_peered":"2023-10-14T10:37:00.080526+0000","last_unstale":"2023-10-14T10:37:01.489206+0000","last_undegraded":"2023-10-14T10:37:01.489206+0000","last_fullsized":"2023-10-14T10:37:01.489206+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:45:25.303185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.489168+0000","last_change":"2023-10-14T10:37:00.080350+0000","last_active":"2023-10-14T10:37:01.489168+0000","last_peered":"2023-10-14T10:37:01.489168+0000","last_clean":"2023-10-14T10:37:01.489168+0000","last_became_active":"2023-10-14T10:37:00.080062+0000","last_became_peered":"2023-10-14T10:37:00.080062+0000","last_unstale":"2023-10-14T10:37:01.489168+0000","last_undegraded":"2023-10-14T10:37:01.489168+0000","last_fullsized":"2023-10-14T10:37:01.489168+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:10:46.473503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.820936+0000","last_change":"2023-10-14T10:37:00.071237+0000","last_active":"2023-10-14T10:37:01.820936+0000","last_peered":"2023-10-14T10:37:01.820936+0000","last_clean":"2023-10-14T10:37:01.820936+0000","last_became_active":"2023-10-14T10:37:00.070897+0000","last_became_peered":"2023-10-14T10:37:00.070897+0000","last_unstale":"2023-10-14T10:37:01.820936+0000","last_undegraded":"2023-10-14T10:37:01.820936+0000","last_fullsized":"2023-10-14T10:37:01.820936+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.061365+0000","last_change":"2023-10-14T10:37:00.069525+0000","last_active":"2023-10-14T10:37:01.061365+0000","last_peered":"2023-10-14T10:37:01.061365+0000","last_clean":"2023-10-14T10:37:01.061365+0000","last_became_active":"2023-10-14T10:37:00.069246+0000","last_became_peered":"2023-10-14T10:37:00.069246+0000","last_unstale":"2023-10-14T10:37:01.061365+0000","last_undegraded":"2023-10-14T10:37:01.061365+0000","last_fullsized":"2023-10-14T10:37:01.061365+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:52:37.894720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.820501+0000","last_change":"2023-10-14T10:37:00.070144+0000","last_active":"2023-10-14T10:37:01.820501+0000","last_peered":"2023-10-14T10:37:01.820501+0000","last_clean":"2023-10-14T10:37:01.820501+0000","last_became_active":"2023-10-14T10:37:00.069871+0000","last_became_peered":"2023-10-14T10:37:00.069871+0000","last_unstale":"2023-10-14T10:37:01.820501+0000","last_undegraded":"2023-10-14T10:37:01.820501+0000","last_fullsized":"2023-10-14T10:37:01.820501+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788109+0000","last_change":"2023-10-14T10:37:00.076474+0000","last_active":"2023-10-14T10:37:01.788109+0000","last_peered":"2023-10-14T10:37:01.788109+0000","last_clean":"2023-10-14T10:37:01.788109+0000","last_became_active":"2023-10-14T10:37:00.076169+0000","last_became_peered":"2023-10-14T10:37:00.076169+0000","last_unstale":"2023-10-14T10:37:01.788109+0000","last_undegraded":"2023-10-14T10:37:01.788109+0000","last_fullsized":"2023-10-14T10:37:01.788109+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T14:52:18.060480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.820465+0000","last_change":"2023-10-14T10:37:00.076568+0000","last_active":"2023-10-14T10:37:01.820465+0000","last_peered":"2023-10-14T10:37:01.820465+0000","last_clean":"2023-10-14T10:37:01.820465+0000","last_became_active":"2023-10-14T10:37:00.076263+0000","last_became_peered":"2023-10-14T10:37:00.076263+0000","last_unstale":"2023-10-14T10:37:01.820465+0000","last_undegraded":"2023-10-14T10:37:01.820465+0000","last_fullsized":"2023-10-14T10:37:01.820465+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.061994+0000","last_change":"2023-10-14T10:37:00.079439+0000","last_active":"2023-10-14T10:37:01.061994+0000","last_peered":"2023-10-14T10:37:01.061994+0000","last_clean":"2023-10-14T10:37:01.061994+0000","last_became_active":"2023-10-14T10:37:00.079178+0000","last_became_peered":"2023-10-14T10:37:00.079178+0000","last_unstale":"2023-10-14T10:37:01.061994+0000","last_undegraded":"2023-10-14T10:37:01.061994+0000","last_fullsized":"2023-10-14T10:37:01.061994+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:17:54.730152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.489206+0000","last_change":"2023-10-14T10:37:00.080675+0000","last_active":"2023-10-14T10:37:01.489206+0000","last_peered":"2023-10-14T10:37:01.489206+0000","last_clean":"2023-10-14T10:37:01.489206+0000","last_became_active":"2023-10-14T10:37:00.080171+0000","last_became_peered":"2023-10-14T10:37:00.080171+0000","last_unstale":"2023-10-14T10:37:01.489206+0000","last_undegraded":"2023-10-14T10:37:01.489206+0000","last_fullsized":"2023-10-14T10:37:01.489206+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:28:04.536948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.061814+0000","last_change":"2023-10-14T10:37:00.072155+0000","last_active":"2023-10-14T10:37:01.061814+0000","last_peered":"2023-10-14T10:37:01.061814+0000","last_clean":"2023-10-14T10:37:01.061814+0000","last_became_active":"2023-10-14T10:37:00.071882+0000","last_became_peered":"2023-10-14T10:37:00.071882+0000","last_unstale":"2023-10-14T10:37:01.061814+0000","last_undegraded":"2023-10-14T10:37:01.061814+0000","last_fullsized":"2023-10-14T10:37:01.061814+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T10:48:03.043383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.787458+0000","last_change":"2023-10-14T10:37:00.081816+0000","last_active":"2023-10-14T10:37:01.787458+0000","last_peered":"2023-10-14T10:37:01.787458+0000","last_clean":"2023-10-14T10:37:01.787458+0000","last_became_active":"2023-10-14T10:37:00.081312+0000","last_became_peered":"2023-10-14T10:37:00.081312+0000","last_unstale":"2023-10-14T10:37:01.787458+0000","last_undegraded":"2023-10-14T10:37:01.787458+0000","last_fullsized":"2023-10-14T10:37:01.787458+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:58:56.558710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.061373+0000","last_change":"2023-10-14T10:37:00.078914+0000","last_active":"2023-10-14T10:37:01.061373+0000","last_peered":"2023-10-14T10:37:01.061373+0000","last_clean":"2023-10-14T10:37:01.061373+0000","last_became_active":"2023-10-14T10:37:00.078441+0000","last_became_peered":"2023-10-14T10:37:00.078441+0000","last_unstale":"2023-10-14T10:37:01.061373+0000","last_undegraded":"2023-10-14T10:37:01.061373+0000","last_fullsized":"2023-10-14T10:37:01.061373+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:22:32.154663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.821051+0000","last_change":"2023-10-14T10:37:00.071246+0000","last_active":"2023-10-14T10:37:01.821051+0000","last_peered":"2023-10-14T10:37:01.821051+0000","last_clean":"2023-10-14T10:37:01.821051+0000","last_became_active":"2023-10-14T10:37:00.070945+0000","last_became_peered":"2023-10-14T10:37:00.070945+0000","last_unstale":"2023-10-14T10:37:01.821051+0000","last_undegraded":"2023-10-14T10:37:01.821051+0000","last_fullsized":"2023-10-14T10:37:01.821051+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.787645+0000","last_change":"2023-10-14T10:36:59.066772+0000","last_active":"2023-10-14T10:37:01.787645+0000","last_peered":"2023-10-14T10:37:01.787645+0000","last_clean":"2023-10-14T10:37:01.787645+0000","last_became_active":"2023-10-14T10:36:59.066488+0000","last_became_peered":"2023-10-14T10:36:59.066488+0000","last_unstale":"2023-10-14T10:37:01.787645+0000","last_undegraded":"2023-10-14T10:37:01.787645+0000","last_fullsized":"2023-10-14T10:37:01.787645+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:39:01.994329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.787569+0000","last_change":"2023-10-14T10:37:00.077997+0000","last_active":"2023-10-14T10:37:01.787569+0000","last_peered":"2023-10-14T10:37:01.787569+0000","last_clean":"2023-10-14T10:37:01.787569+0000","last_became_active":"2023-10-14T10:37:00.077734+0000","last_became_peered":"2023-10-14T10:37:00.077734+0000","last_unstale":"2023-10-14T10:37:01.787569+0000","last_undegraded":"2023-10-14T10:37:01.787569+0000","last_fullsized":"2023-10-14T10:37:01.787569+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:28:45.821377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.821077+0000","last_change":"2023-10-14T10:36:59.070768+0000","last_active":"2023-10-14T10:37:01.821077+0000","last_peered":"2023-10-14T10:37:01.821077+0000","last_clean":"2023-10-14T10:37:01.821077+0000","last_became_active":"2023-10-14T10:36:59.070463+0000","last_became_peered":"2023-10-14T10:36:59.070463+0000","last_unstale":"2023-10-14T10:37:01.821077+0000","last_undegraded":"2023-10-14T10:37:01.821077+0000","last_fullsized":"2023-10-14T10:37:01.821077+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.489357+0000","last_change":"2023-10-14T10:37:00.080768+0000","last_active":"2023-10-14T10:37:01.489357+0000","last_peered":"2023-10-14T10:37:01.489357+0000","last_clean":"2023-10-14T10:37:01.489357+0000","last_became_active":"2023-10-14T10:37:00.080160+0000","last_became_peered":"2023-10-14T10:37:00.080160+0000","last_unstale":"2023-10-14T10:37:01.489357+0000","last_undegraded":"2023-10-14T10:37:01.489357+0000","last_fullsized":"2023-10-14T10:37:01.489357+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T20:27:28.798883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788006+0000","last_change":"2023-10-14T10:36:59.067900+0000","last_active":"2023-10-14T10:37:01.788006+0000","last_peered":"2023-10-14T10:37:01.788006+0000","last_clean":"2023-10-14T10:37:01.788006+0000","last_became_active":"2023-10-14T10:36:59.067616+0000","last_became_peered":"2023-10-14T10:36:59.067616+0000","last_unstale":"2023-10-14T10:37:01.788006+0000","last_undegraded":"2023-10-14T10:37:01.788006+0000","last_fullsized":"2023-10-14T10:37:01.788006+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:28:44.815010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.820479+0000","last_change":"2023-10-14T10:37:00.079200+0000","last_active":"2023-10-14T10:37:01.820479+0000","last_peered":"2023-10-14T10:37:01.820479+0000","last_clean":"2023-10-14T10:37:01.820479+0000","last_became_active":"2023-10-14T10:37:00.078829+0000","last_became_peered":"2023-10-14T10:37:00.078829+0000","last_unstale":"2023-10-14T10:37:01.820479+0000","last_undegraded":"2023-10-14T10:37:01.820479+0000","last_fullsized":"2023-10-14T10:37:01.820479+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788507+0000","last_change":"2023-10-14T10:36:59.065882+0000","last_active":"2023-10-14T10:37:01.788507+0000","last_peered":"2023-10-14T10:37:01.788507+0000","last_clean":"2023-10-14T10:37:01.788507+0000","last_became_active":"2023-10-14T10:36:59.065608+0000","last_became_peered":"2023-10-14T10:36:59.065608+0000","last_unstale":"2023-10-14T10:37:01.788507+0000","last_undegraded":"2023-10-14T10:37:01.788507+0000","last_fullsized":"2023-10-14T10:37:01.788507+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T21:18:46.547776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.062326+0000","last_change":"2023-10-14T10:37:00.077115+0000","last_active":"2023-10-14T10:37:01.062326+0000","last_peered":"2023-10-14T10:37:01.062326+0000","last_clean":"2023-10-14T10:37:01.062326+0000","last_became_active":"2023-10-14T10:37:00.076840+0000","last_became_peered":"2023-10-14T10:37:00.076840+0000","last_unstale":"2023-10-14T10:37:01.062326+0000","last_undegraded":"2023-10-14T10:37:01.062326+0000","last_fullsized":"2023-10-14T10:37:01.062326+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T12:47:24.725885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.489315+0000","last_change":"2023-10-14T10:36:59.069533+0000","last_active":"2023-10-14T10:37:01.489315+0000","last_peered":"2023-10-14T10:37:01.489315+0000","last_clean":"2023-10-14T10:37:01.489315+0000","last_became_active":"2023-10-14T10:36:59.069192+0000","last_became_peered":"2023-10-14T10:36:59.069192+0000","last_unstale":"2023-10-14T10:37:01.489315+0000","last_undegraded":"2023-10-14T10:37:01.489315+0000","last_fullsized":"2023-10-14T10:37:01.489315+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T21:39:34.581362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.061634+0000","last_change":"2023-10-14T10:37:00.068786+0000","last_active":"2023-10-14T10:37:01.061634+0000","last_peered":"2023-10-14T10:37:01.061634+0000","last_clean":"2023-10-14T10:37:01.061634+0000","last_became_active":"2023-10-14T10:37:00.068517+0000","last_became_peered":"2023-10-14T10:37:00.068517+0000","last_unstale":"2023-10-14T10:37:01.061634+0000","last_undegraded":"2023-10-14T10:37:01.061634+0000","last_fullsized":"2023-10-14T10:37:01.061634+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:22:32.154663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.489303+0000","last_change":"2023-10-14T10:36:59.069445+0000","last_active":"2023-10-14T10:37:01.489303+0000","last_peered":"2023-10-14T10:37:01.489303+0000","last_clean":"2023-10-14T10:37:01.489303+0000","last_became_active":"2023-10-14T10:36:59.069151+0000","last_became_peered":"2023-10-14T10:36:59.069151+0000","last_unstale":"2023-10-14T10:37:01.489303+0000","last_undegraded":"2023-10-14T10:37:01.489303+0000","last_fullsized":"2023-10-14T10:37:01.489303+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:54:15.011628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.488953+0000","last_change":"2023-10-14T10:37:00.076984+0000","last_active":"2023-10-14T10:37:01.488953+0000","last_peered":"2023-10-14T10:37:01.488953+0000","last_clean":"2023-10-14T10:37:01.488953+0000","last_became_active":"2023-10-14T10:37:00.076706+0000","last_became_peered":"2023-10-14T10:37:00.076706+0000","last_unstale":"2023-10-14T10:37:01.488953+0000","last_undegraded":"2023-10-14T10:37:01.488953+0000","last_fullsized":"2023-10-14T10:37:01.488953+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:30:57.784792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.787709+0000","last_change":"2023-10-14T10:36:59.062324+0000","last_active":"2023-10-14T10:37:01.787709+0000","last_peered":"2023-10-14T10:37:01.787709+0000","last_clean":"2023-10-14T10:37:01.787709+0000","last_became_active":"2023-10-14T10:36:59.062075+0000","last_became_peered":"2023-10-14T10:36:59.062075+0000","last_unstale":"2023-10-14T10:37:01.787709+0000","last_undegraded":"2023-10-14T10:37:01.787709+0000","last_fullsized":"2023-10-14T10:37:01.787709+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:45:50.919718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.820360+0000","last_change":"2023-10-14T10:37:00.067486+0000","last_active":"2023-10-14T10:37:01.820360+0000","last_peered":"2023-10-14T10:37:01.820360+0000","last_clean":"2023-10-14T10:37:01.820360+0000","last_became_active":"2023-10-14T10:37:00.067187+0000","last_became_peered":"2023-10-14T10:37:00.067187+0000","last_unstale":"2023-10-14T10:37:01.820360+0000","last_undegraded":"2023-10-14T10:37:01.820360+0000","last_fullsized":"2023-10-14T10:37:01.820360+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.488998+0000","last_change":"2023-10-14T10:36:59.066330+0000","last_active":"2023-10-14T10:37:01.488998+0000","last_peered":"2023-10-14T10:37:01.488998+0000","last_clean":"2023-10-14T10:37:01.488998+0000","last_became_active":"2023-10-14T10:36:59.064990+0000","last_became_peered":"2023-10-14T10:36:59.064990+0000","last_unstale":"2023-10-14T10:37:01.488998+0000","last_undegraded":"2023-10-14T10:37:01.488998+0000","last_fullsized":"2023-10-14T10:37:01.488998+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T12:15:16.823069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.061974+0000","last_change":"2023-10-14T10:37:00.077465+0000","last_active":"2023-10-14T10:37:01.061974+0000","last_peered":"2023-10-14T10:37:01.061974+0000","last_clean":"2023-10-14T10:37:01.061974+0000","last_became_active":"2023-10-14T10:37:00.077152+0000","last_became_peered":"2023-10-14T10:37:00.077152+0000","last_unstale":"2023-10-14T10:37:01.061974+0000","last_undegraded":"2023-10-14T10:37:01.061974+0000","last_fullsized":"2023-10-14T10:37:01.061974+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:51:00.167498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788575+0000","last_change":"2023-10-14T10:36:59.061903+0000","last_active":"2023-10-14T10:37:01.788575+0000","last_peered":"2023-10-14T10:37:01.788575+0000","last_clean":"2023-10-14T10:37:01.788575+0000","last_became_active":"2023-10-14T10:36:59.061597+0000","last_became_peered":"2023-10-14T10:36:59.061597+0000","last_unstale":"2023-10-14T10:37:01.788575+0000","last_undegraded":"2023-10-14T10:37:01.788575+0000","last_fullsized":"2023-10-14T10:37:01.788575+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:22:31.148297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788727+0000","last_change":"2023-10-14T10:37:00.076213+0000","last_active":"2023-10-14T10:37:01.788727+0000","last_peered":"2023-10-14T10:37:01.788727+0000","last_clean":"2023-10-14T10:37:01.788727+0000","last_became_active":"2023-10-14T10:37:00.075889+0000","last_became_peered":"2023-10-14T10:37:00.075889+0000","last_unstale":"2023-10-14T10:37:01.788727+0000","last_undegraded":"2023-10-14T10:37:01.788727+0000","last_fullsized":"2023-10-14T10:37:01.788727+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T14:19:22.310866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.821108+0000","last_change":"2023-10-14T10:36:59.070183+0000","last_active":"2023-10-14T10:37:01.821108+0000","last_peered":"2023-10-14T10:37:01.821108+0000","last_clean":"2023-10-14T10:37:01.821108+0000","last_became_active":"2023-10-14T10:36:59.069731+0000","last_became_peered":"2023-10-14T10:36:59.069731+0000","last_unstale":"2023-10-14T10:37:01.821108+0000","last_undegraded":"2023-10-14T10:37:01.821108+0000","last_fullsized":"2023-10-14T10:37:01.821108+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788210+0000","last_change":"2023-10-14T10:37:00.072592+0000","last_active":"2023-10-14T10:37:01.788210+0000","last_peered":"2023-10-14T10:37:01.788210+0000","last_clean":"2023-10-14T10:37:01.788210+0000","last_became_active":"2023-10-14T10:37:00.072288+0000","last_became_peered":"2023-10-14T10:37:00.072288+0000","last_unstale":"2023-10-14T10:37:01.788210+0000","last_undegraded":"2023-10-14T10:37:01.788210+0000","last_fullsized":"2023-10-14T10:37:01.788210+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T17:27:50.406084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.062406+0000","last_change":"2023-10-14T10:36:59.062120+0000","last_active":"2023-10-14T10:37:01.062406+0000","last_peered":"2023-10-14T10:37:01.062406+0000","last_clean":"2023-10-14T10:37:01.062406+0000","last_became_active":"2023-10-14T10:36:59.061700+0000","last_became_peered":"2023-10-14T10:36:59.061700+0000","last_unstale":"2023-10-14T10:37:01.062406+0000","last_undegraded":"2023-10-14T10:37:01.062406+0000","last_fullsized":"2023-10-14T10:37:01.062406+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:45:24.296819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.062364+0000","last_change":"2023-10-14T10:37:00.074977+0000","last_active":"2023-10-14T10:37:01.062364+0000","last_peered":"2023-10-14T10:37:01.062364+0000","last_clean":"2023-10-14T10:37:01.062364+0000","last_became_active":"2023-10-14T10:37:00.074700+0000","last_became_peered":"2023-10-14T10:37:00.074700+0000","last_unstale":"2023-10-14T10:37:01.062364+0000","last_undegraded":"2023-10-14T10:37:01.062364+0000","last_fullsized":"2023-10-14T10:37:01.062364+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T18:57:50.136605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788323+0000","last_change":"2023-10-14T10:36:59.069400+0000","last_active":"2023-10-14T10:37:01.788323+0000","last_peered":"2023-10-14T10:37:01.788323+0000","last_clean":"2023-10-14T10:37:01.788323+0000","last_became_active":"2023-10-14T10:36:59.069043+0000","last_became_peered":"2023-10-14T10:36:59.069043+0000","last_unstale":"2023-10-14T10:37:01.788323+0000","last_undegraded":"2023-10-14T10:37:01.788323+0000","last_fullsized":"2023-10-14T10:37:01.788323+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:10:45.467136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.820378+0000","last_change":"2023-10-14T10:37:00.068613+0000","last_active":"2023-10-14T10:37:01.820378+0000","last_peered":"2023-10-14T10:37:01.820378+0000","last_clean":"2023-10-14T10:37:01.820378+0000","last_became_active":"2023-10-14T10:37:00.065345+0000","last_became_peered":"2023-10-14T10:37:00.065345+0000","last_unstale":"2023-10-14T10:37:01.820378+0000","last_undegraded":"2023-10-14T10:37:01.820378+0000","last_fullsized":"2023-10-14T10:37:01.820378+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.489387+0000","last_change":"2023-10-14T10:36:59.065846+0000","last_active":"2023-10-14T10:37:01.489387+0000","last_peered":"2023-10-14T10:37:01.489387+0000","last_clean":"2023-10-14T10:37:01.489387+0000","last_became_active":"2023-10-14T10:36:59.065569+0000","last_became_peered":"2023-10-14T10:36:59.065569+0000","last_unstale":"2023-10-14T10:37:01.489387+0000","last_undegraded":"2023-10-14T10:37:01.489387+0000","last_fullsized":"2023-10-14T10:37:01.489387+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:28:44.815010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788692+0000","last_change":"2023-10-14T10:37:00.079137+0000","last_active":"2023-10-14T10:37:01.788692+0000","last_peered":"2023-10-14T10:37:01.788692+0000","last_clean":"2023-10-14T10:37:01.788692+0000","last_became_active":"2023-10-14T10:37:00.078844+0000","last_became_peered":"2023-10-14T10:37:00.078844+0000","last_unstale":"2023-10-14T10:37:01.788692+0000","last_undegraded":"2023-10-14T10:37:01.788692+0000","last_fullsized":"2023-10-14T10:37:01.788692+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:30:51.914014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.820396+0000","last_change":"2023-10-14T10:36:59.067245+0000","last_active":"2023-10-14T10:37:01.820396+0000","last_peered":"2023-10-14T10:37:01.820396+0000","last_clean":"2023-10-14T10:37:01.820396+0000","last_became_active":"2023-10-14T10:36:59.066952+0000","last_became_peered":"2023-10-14T10:36:59.066952+0000","last_unstale":"2023-10-14T10:37:01.820396+0000","last_undegraded":"2023-10-14T10:37:01.820396+0000","last_fullsized":"2023-10-14T10:37:01.820396+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.062454+0000","last_change":"2023-10-14T10:37:00.078646+0000","last_active":"2023-10-14T10:37:01.062454+0000","last_peered":"2023-10-14T10:37:01.062454+0000","last_clean":"2023-10-14T10:37:01.062454+0000","last_became_active":"2023-10-14T10:37:00.078353+0000","last_became_peered":"2023-10-14T10:37:00.078353+0000","last_unstale":"2023-10-14T10:37:01.062454+0000","last_undegraded":"2023-10-14T10:37:01.062454+0000","last_fullsized":"2023-10-14T10:37:01.062454+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:30:51.914014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788938+0000","last_change":"2023-10-14T10:36:59.063118+0000","last_active":"2023-10-14T10:37:01.788938+0000","last_peered":"2023-10-14T10:37:01.788938+0000","last_clean":"2023-10-14T10:37:01.788938+0000","last_became_active":"2023-10-14T10:36:59.062380+0000","last_became_peered":"2023-10-14T10:36:59.062380+0000","last_unstale":"2023-10-14T10:37:01.788938+0000","last_undegraded":"2023-10-14T10:37:01.788938+0000","last_fullsized":"2023-10-14T10:37:01.788938+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T21:39:34.581362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.787507+0000","last_change":"2023-10-14T10:37:00.078077+0000","last_active":"2023-10-14T10:37:01.787507+0000","last_peered":"2023-10-14T10:37:01.787507+0000","last_clean":"2023-10-14T10:37:01.787507+0000","last_became_active":"2023-10-14T10:37:00.077806+0000","last_became_peered":"2023-10-14T10:37:00.077806+0000","last_unstale":"2023-10-14T10:37:01.787507+0000","last_undegraded":"2023-10-14T10:37:01.787507+0000","last_fullsized":"2023-10-14T10:37:01.787507+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:45:51.926085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788260+0000","last_change":"2023-10-14T10:36:59.068012+0000","last_active":"2023-10-14T10:37:01.788260+0000","last_peered":"2023-10-14T10:37:01.788260+0000","last_clean":"2023-10-14T10:37:01.788260+0000","last_became_active":"2023-10-14T10:36:59.067727+0000","last_became_peered":"2023-10-14T10:36:59.067727+0000","last_unstale":"2023-10-14T10:37:01.788260+0000","last_undegraded":"2023-10-14T10:37:01.788260+0000","last_fullsized":"2023-10-14T10:37:01.788260+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:44:49.671921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.489522+0000","last_change":"2023-10-14T10:37:00.076926+0000","last_active":"2023-10-14T10:37:01.489522+0000","last_peered":"2023-10-14T10:37:01.489522+0000","last_clean":"2023-10-14T10:37:01.489522+0000","last_became_active":"2023-10-14T10:37:00.076658+0000","last_became_peered":"2023-10-14T10:37:00.076658+0000","last_unstale":"2023-10-14T10:37:01.489522+0000","last_undegraded":"2023-10-14T10:37:01.489522+0000","last_fullsized":"2023-10-14T10:37:01.489522+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:52:37.894720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.820550+0000","last_change":"2023-10-14T10:36:59.071714+0000","last_active":"2023-10-14T10:37:01.820550+0000","last_peered":"2023-10-14T10:37:01.820550+0000","last_clean":"2023-10-14T10:37:01.820550+0000","last_became_active":"2023-10-14T10:36:59.071406+0000","last_became_peered":"2023-10-14T10:36:59.071406+0000","last_unstale":"2023-10-14T10:37:01.820550+0000","last_undegraded":"2023-10-14T10:37:01.820550+0000","last_fullsized":"2023-10-14T10:37:01.820550+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.062345+0000","last_change":"2023-10-14T10:37:00.080060+0000","last_active":"2023-10-14T10:37:01.062345+0000","last_peered":"2023-10-14T10:37:01.062345+0000","last_clean":"2023-10-14T10:37:01.062345+0000","last_became_active":"2023-10-14T10:37:00.079795+0000","last_became_peered":"2023-10-14T10:37:00.079795+0000","last_unstale":"2023-10-14T10:37:01.062345+0000","last_undegraded":"2023-10-14T10:37:01.062345+0000","last_fullsized":"2023-10-14T10:37:01.062345+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T17:27:50.406084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.489554+0000","last_change":"2023-10-14T10:36:59.065271+0000","last_active":"2023-10-14T10:37:01.489554+0000","last_peered":"2023-10-14T10:37:01.489554+0000","last_clean":"2023-10-14T10:37:01.489554+0000","last_became_active":"2023-10-14T10:36:59.064959+0000","last_became_peered":"2023-10-14T10:36:59.064959+0000","last_unstale":"2023-10-14T10:37:01.489554+0000","last_undegraded":"2023-10-14T10:37:01.489554+0000","last_fullsized":"2023-10-14T10:37:01.489554+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T21:56:55.137838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.820453+0000","last_change":"2023-10-14T10:37:00.077751+0000","last_active":"2023-10-14T10:37:01.820453+0000","last_peered":"2023-10-14T10:37:01.820453+0000","last_clean":"2023-10-14T10:37:01.820453+0000","last_became_active":"2023-10-14T10:37:00.077459+0000","last_became_peered":"2023-10-14T10:37:00.077459+0000","last_unstale":"2023-10-14T10:37:01.820453+0000","last_undegraded":"2023-10-14T10:37:01.820453+0000","last_fullsized":"2023-10-14T10:37:01.820453+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.489473+0000","last_change":"2023-10-14T10:36:59.058893+0000","last_active":"2023-10-14T10:37:01.489473+0000","last_peered":"2023-10-14T10:37:01.489473+0000","last_clean":"2023-10-14T10:37:01.489473+0000","last_became_active":"2023-10-14T10:36:59.058558+0000","last_became_peered":"2023-10-14T10:36:59.058558+0000","last_unstale":"2023-10-14T10:37:01.489473+0000","last_undegraded":"2023-10-14T10:37:01.489473+0000","last_fullsized":"2023-10-14T10:37:01.489473+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:45:50.919718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788693+0000","last_change":"2023-10-14T10:37:00.071398+0000","last_active":"2023-10-14T10:37:01.788693+0000","last_peered":"2023-10-14T10:37:01.788693+0000","last_clean":"2023-10-14T10:37:01.788693+0000","last_became_active":"2023-10-14T10:37:00.071167+0000","last_became_peered":"2023-10-14T10:37:00.071167+0000","last_unstale":"2023-10-14T10:37:01.788693+0000","last_undegraded":"2023-10-14T10:37:01.788693+0000","last_fullsized":"2023-10-14T10:37:01.788693+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:31:12.123956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.489473+0000","last_change":"2023-10-14T10:36:59.069420+0000","last_active":"2023-10-14T10:37:01.489473+0000","last_peered":"2023-10-14T10:37:01.489473+0000","last_clean":"2023-10-14T10:37:01.489473+0000","last_became_active":"2023-10-14T10:36:59.069110+0000","last_became_peered":"2023-10-14T10:36:59.069110+0000","last_unstale":"2023-10-14T10:37:01.489473+0000","last_undegraded":"2023-10-14T10:37:01.489473+0000","last_fullsized":"2023-10-14T10:37:01.489473+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:09:59.812757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788586+0000","last_change":"2023-10-14T10:37:00.077550+0000","last_active":"2023-10-14T10:37:01.788586+0000","last_peered":"2023-10-14T10:37:01.788586+0000","last_clean":"2023-10-14T10:37:01.788586+0000","last_became_active":"2023-10-14T10:37:00.077175+0000","last_became_peered":"2023-10-14T10:37:00.077175+0000","last_unstale":"2023-10-14T10:37:01.788586+0000","last_undegraded":"2023-10-14T10:37:01.788586+0000","last_fullsized":"2023-10-14T10:37:01.788586+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T14:51:19.968923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.820991+0000","last_change":"2023-10-14T10:36:59.067621+0000","last_active":"2023-10-14T10:37:01.820991+0000","last_peered":"2023-10-14T10:37:01.820991+0000","last_clean":"2023-10-14T10:37:01.820991+0000","last_became_active":"2023-10-14T10:36:59.067345+0000","last_became_peered":"2023-10-14T10:36:59.067345+0000","last_unstale":"2023-10-14T10:37:01.820991+0000","last_undegraded":"2023-10-14T10:37:01.820991+0000","last_fullsized":"2023-10-14T10:37:01.820991+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788501+0000","last_change":"2023-10-14T10:37:00.077744+0000","last_active":"2023-10-14T10:37:01.788501+0000","last_peered":"2023-10-14T10:37:01.788501+0000","last_clean":"2023-10-14T10:37:01.788501+0000","last_became_active":"2023-10-14T10:37:00.077465+0000","last_became_peered":"2023-10-14T10:37:00.077465+0000","last_unstale":"2023-10-14T10:37:01.788501+0000","last_undegraded":"2023-10-14T10:37:01.788501+0000","last_fullsized":"2023-10-14T10:37:01.788501+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T20:36:23.971821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.787432+0000","last_change":"2023-10-14T10:36:59.069707+0000","last_active":"2023-10-14T10:37:01.787432+0000","last_peered":"2023-10-14T10:37:01.787432+0000","last_clean":"2023-10-14T10:37:01.787432+0000","last_became_active":"2023-10-14T10:36:59.069269+0000","last_became_peered":"2023-10-14T10:36:59.069269+0000","last_unstale":"2023-10-14T10:37:01.787432+0000","last_undegraded":"2023-10-14T10:37:01.787432+0000","last_fullsized":"2023-10-14T10:37:01.787432+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T12:15:16.823069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788221+0000","last_change":"2023-10-14T10:37:00.068887+0000","last_active":"2023-10-14T10:37:01.788221+0000","last_peered":"2023-10-14T10:37:01.788221+0000","last_clean":"2023-10-14T10:37:01.788221+0000","last_became_active":"2023-10-14T10:37:00.068620+0000","last_became_peered":"2023-10-14T10:37:00.068620+0000","last_unstale":"2023-10-14T10:37:01.788221+0000","last_undegraded":"2023-10-14T10:37:01.788221+0000","last_fullsized":"2023-10-14T10:37:01.788221+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T18:04:59.338050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.489058+0000","last_change":"2023-10-14T10:36:59.061947+0000","last_active":"2023-10-14T10:37:01.489058+0000","last_peered":"2023-10-14T10:37:01.489058+0000","last_clean":"2023-10-14T10:37:01.489058+0000","last_became_active":"2023-10-14T10:36:59.061678+0000","last_became_peered":"2023-10-14T10:36:59.061678+0000","last_unstale":"2023-10-14T10:37:01.489058+0000","last_undegraded":"2023-10-14T10:37:01.489058+0000","last_fullsized":"2023-10-14T10:37:01.489058+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:39:01.994329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.061998+0000","last_change":"2023-10-14T10:37:00.073204+0000","last_active":"2023-10-14T10:37:01.061998+0000","last_peered":"2023-10-14T10:37:01.061998+0000","last_clean":"2023-10-14T10:37:01.061998+0000","last_became_active":"2023-10-14T10:37:00.071692+0000","last_became_peered":"2023-10-14T10:37:00.071692+0000","last_unstale":"2023-10-14T10:37:01.061998+0000","last_undegraded":"2023-10-14T10:37:01.061998+0000","last_fullsized":"2023-10-14T10:37:01.061998+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:17:54.730152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.061467+0000","last_change":"2023-10-14T10:36:59.063545+0000","last_active":"2023-10-14T10:37:01.061467+0000","last_peered":"2023-10-14T10:37:01.061467+0000","last_clean":"2023-10-14T10:37:01.061467+0000","last_became_active":"2023-10-14T10:36:59.063332+0000","last_became_peered":"2023-10-14T10:36:59.063332+0000","last_unstale":"2023-10-14T10:37:01.061467+0000","last_undegraded":"2023-10-14T10:37:01.061467+0000","last_fullsized":"2023-10-14T10:37:01.061467+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:10:45.467136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788558+0000","last_change":"2023-10-14T10:37:00.075249+0000","last_active":"2023-10-14T10:37:01.788558+0000","last_peered":"2023-10-14T10:37:01.788558+0000","last_clean":"2023-10-14T10:37:01.788558+0000","last_became_active":"2023-10-14T10:37:00.074859+0000","last_became_peered":"2023-10-14T10:37:00.074859+0000","last_unstale":"2023-10-14T10:37:01.788558+0000","last_undegraded":"2023-10-14T10:37:01.788558+0000","last_fullsized":"2023-10-14T10:37:01.788558+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:04:51.088268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.820521+0000","last_change":"2023-10-14T10:36:59.069418+0000","last_active":"2023-10-14T10:37:01.820521+0000","last_peered":"2023-10-14T10:37:01.820521+0000","last_clean":"2023-10-14T10:37:01.820521+0000","last_became_active":"2023-10-14T10:36:59.069037+0000","last_became_peered":"2023-10-14T10:36:59.069037+0000","last_unstale":"2023-10-14T10:37:01.820521+0000","last_undegraded":"2023-10-14T10:37:01.820521+0000","last_fullsized":"2023-10-14T10:37:01.820521+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.062253+0000","last_change":"2023-10-14T10:37:00.075731+0000","last_active":"2023-10-14T10:37:01.062253+0000","last_peered":"2023-10-14T10:37:01.062253+0000","last_clean":"2023-10-14T10:37:01.062253+0000","last_became_active":"2023-10-14T10:37:00.075491+0000","last_became_peered":"2023-10-14T10:37:00.075491+0000","last_unstale":"2023-10-14T10:37:01.062253+0000","last_undegraded":"2023-10-14T10:37:01.062253+0000","last_fullsized":"2023-10-14T10:37:01.062253+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T14:52:18.060480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"33'1","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:02.185459+0000","last_change":"2023-10-14T10:36:59.066461+0000","last_active":"2023-10-14T10:37:02.185459+0000","last_peered":"2023-10-14T10:37:02.185459+0000","last_clean":"2023-10-14T10:37:02.185459+0000","last_became_active":"2023-10-14T10:36:59.066146+0000","last_became_peered":"2023-10-14T10:36:59.066146+0000","last_unstale":"2023-10-14T10:37:02.185459+0000","last_undegraded":"2023-10-14T10:37:02.185459+0000","last_fullsized":"2023-10-14T10:37:02.185459+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.821034+0000","last_change":"2023-10-14T10:37:00.082851+0000","last_active":"2023-10-14T10:37:01.821034+0000","last_peered":"2023-10-14T10:37:01.821034+0000","last_clean":"2023-10-14T10:37:01.821034+0000","last_became_active":"2023-10-14T10:37:00.082570+0000","last_became_peered":"2023-10-14T10:37:00.082570+0000","last_unstale":"2023-10-14T10:37:01.821034+0000","last_undegraded":"2023-10-14T10:37:01.821034+0000","last_fullsized":"2023-10-14T10:37:01.821034+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.820442+0000","last_change":"2023-10-14T10:36:59.061936+0000","last_active":"2023-10-14T10:37:01.820442+0000","last_peered":"2023-10-14T10:37:01.820442+0000","last_clean":"2023-10-14T10:37:01.820442+0000","last_became_active":"2023-10-14T10:36:59.061642+0000","last_became_peered":"2023-10-14T10:36:59.061642+0000","last_unstale":"2023-10-14T10:37:01.820442+0000","last_undegraded":"2023-10-14T10:37:01.820442+0000","last_fullsized":"2023-10-14T10:37:01.820442+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.787449+0000","last_change":"2023-10-14T10:37:00.071551+0000","last_active":"2023-10-14T10:37:01.787449+0000","last_peered":"2023-10-14T10:37:01.787449+0000","last_clean":"2023-10-14T10:37:01.787449+0000","last_became_active":"2023-10-14T10:37:00.071244+0000","last_became_peered":"2023-10-14T10:37:00.071244+0000","last_unstale":"2023-10-14T10:37:01.787449+0000","last_undegraded":"2023-10-14T10:37:01.787449+0000","last_fullsized":"2023-10-14T10:37:01.787449+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:08:06.341692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.820967+0000","last_change":"2023-10-14T10:36:59.066839+0000","last_active":"2023-10-14T10:37:01.820967+0000","last_peered":"2023-10-14T10:37:01.820967+0000","last_clean":"2023-10-14T10:37:01.820967+0000","last_became_active":"2023-10-14T10:36:59.066592+0000","last_became_peered":"2023-10-14T10:36:59.066592+0000","last_unstale":"2023-10-14T10:37:01.820967+0000","last_undegraded":"2023-10-14T10:37:01.820967+0000","last_fullsized":"2023-10-14T10:37:01.820967+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.821014+0000","last_change":"2023-10-14T10:37:00.076076+0000","last_active":"2023-10-14T10:37:01.821014+0000","last_peered":"2023-10-14T10:37:01.821014+0000","last_clean":"2023-10-14T10:37:01.821014+0000","last_became_active":"2023-10-14T10:37:00.075794+0000","last_became_peered":"2023-10-14T10:37:00.075794+0000","last_unstale":"2023-10-14T10:37:01.821014+0000","last_undegraded":"2023-10-14T10:37:01.821014+0000","last_fullsized":"2023-10-14T10:37:01.821014+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.820406+0000","last_change":"2023-10-14T10:36:59.065626+0000","last_active":"2023-10-14T10:37:01.820406+0000","last_peered":"2023-10-14T10:37:01.820406+0000","last_clean":"2023-10-14T10:37:01.820406+0000","last_became_active":"2023-10-14T10:36:59.065163+0000","last_became_peered":"2023-10-14T10:36:59.065163+0000","last_unstale":"2023-10-14T10:37:01.820406+0000","last_undegraded":"2023-10-14T10:37:01.820406+0000","last_fullsized":"2023-10-14T10:37:01.820406+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788427+0000","last_change":"2023-10-14T10:37:00.078024+0000","last_active":"2023-10-14T10:37:01.788427+0000","last_peered":"2023-10-14T10:37:01.788427+0000","last_clean":"2023-10-14T10:37:01.788427+0000","last_became_active":"2023-10-14T10:37:00.077621+0000","last_became_peered":"2023-10-14T10:37:00.077621+0000","last_unstale":"2023-10-14T10:37:01.788427+0000","last_undegraded":"2023-10-14T10:37:01.788427+0000","last_fullsized":"2023-10-14T10:37:01.788427+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:06:28.391046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.489094+0000","last_change":"2023-10-14T10:36:59.064632+0000","last_active":"2023-10-14T10:37:01.489094+0000","last_peered":"2023-10-14T10:37:01.489094+0000","last_clean":"2023-10-14T10:37:01.489094+0000","last_became_active":"2023-10-14T10:36:59.064336+0000","last_became_peered":"2023-10-14T10:36:59.064336+0000","last_unstale":"2023-10-14T10:37:01.489094+0000","last_undegraded":"2023-10-14T10:37:01.489094+0000","last_fullsized":"2023-10-14T10:37:01.489094+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:08:05.335325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.488971+0000","last_change":"2023-10-14T10:37:00.081042+0000","last_active":"2023-10-14T10:37:01.488971+0000","last_peered":"2023-10-14T10:37:01.488971+0000","last_clean":"2023-10-14T10:37:01.488971+0000","last_became_active":"2023-10-14T10:37:00.080340+0000","last_became_peered":"2023-10-14T10:37:00.080340+0000","last_unstale":"2023-10-14T10:37:01.488971+0000","last_undegraded":"2023-10-14T10:37:01.488971+0000","last_fullsized":"2023-10-14T10:37:01.488971+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:44:50.678288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.788029+0000","last_change":"2023-10-14T10:36:59.063568+0000","last_active":"2023-10-14T10:37:01.788029+0000","last_peered":"2023-10-14T10:37:01.788029+0000","last_clean":"2023-10-14T10:37:01.788029+0000","last_became_active":"2023-10-14T10:36:59.063130+0000","last_became_peered":"2023-10-14T10:36:59.063130+0000","last_unstale":"2023-10-14T10:37:01.788029+0000","last_undegraded":"2023-10-14T10:37:01.788029+0000","last_fullsized":"2023-10-14T10:37:01.788029+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:59:06.202215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.061998+0000","last_change":"2023-10-14T10:37:00.068870+0000","last_active":"2023-10-14T10:37:01.061998+0000","last_peered":"2023-10-14T10:37:01.061998+0000","last_clean":"2023-10-14T10:37:01.061998+0000","last_became_active":"2023-10-14T10:37:00.068556+0000","last_became_peered":"2023-10-14T10:37:00.068556+0000","last_unstale":"2023-10-14T10:37:01.061998+0000","last_undegraded":"2023-10-14T10:37:01.061998+0000","last_fullsized":"2023-10-14T10:37:01.061998+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T14:52:18.060480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.787484+0000","last_change":"2023-10-14T10:36:59.066703+0000","last_active":"2023-10-14T10:37:01.787484+0000","last_peered":"2023-10-14T10:37:01.787484+0000","last_clean":"2023-10-14T10:37:01.787484+0000","last_became_active":"2023-10-14T10:36:59.066424+0000","last_became_peered":"2023-10-14T10:36:59.066424+0000","last_unstale":"2023-10-14T10:37:01.787484+0000","last_undegraded":"2023-10-14T10:37:01.787484+0000","last_fullsized":"2023-10-14T10:37:01.787484+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:54:15.011628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":176,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-14T10:37:00.060617+0000","last_change":"2023-10-14T10:35:59.770163+0000","last_active":"2023-10-14T10:37:00.060617+0000","last_peered":"2023-10-14T10:37:00.060617+0000","last_clean":"2023-10-14T10:37:00.060617+0000","last_became_active":"2023-10-14T10:35:59.769199+0000","last_became_peered":"2023-10-14T10:35:59.769199+0000","last_unstale":"2023-10-14T10:37:00.060617+0000","last_undegraded":"2023-10-14T10:37:00.060617+0000","last_fullsized":"2023-10-14T10:37:00.060617+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:35:58.675572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:35:58.675572+0000","last_clean_scrub_stamp":"2023-10-14T10:35:58.675572+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:03:25.281187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542163,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27972,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343868,"statfs":{"total":96636764160,"available":96608120832,"internally_reserved":0,"allocated":966656,"data_stored":732594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 10:36:59 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":2.4649999999999999,"5min":2.4649999999999999,"15min":2.4649999999999999},"last":0.628},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.39700000000000002}]},{"osd":1,"last update":"Sat Oct 14 10:36:59 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":2.4289999999999998,"5min":2.4289999999999998,"15min":2.4289999999999998},"last":0.52100000000000002}]},{"osd":2,"last update":"Sat Oct 14 10:36:59 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.496}]},{"osd":3,"last update":"Sat Oct 14 10:36:59 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.405,"5min":1.405,"15min":1.405},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.45100000000000001}]},{"osd":4,"last update":"Sat Oct 14 10:36:59 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.65600000000000003}]},{"osd":5,"last update":"Sat Oct 14 10:36:59 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.60299999999999998}]},{"osd":6,"last update":"Sat Oct 14 10:36:59 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.68799999999999994}]}]},{"osd":6,"up_from":14,"seq":60129542164,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27388,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344452,"statfs":{"total":96636764160,"available":96608718848,"internally_reserved":0,"allocated":376832,"data_stored":142752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 10:36:57 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.42799999999999999}]},{"osd":1,"last update":"Sat Oct 14 10:36:57 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.389,"5min":1.389,"15min":1.389},"last":0.62}]},{"osd":2,"last update":"Sat Oct 14 10:36:57 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.59599999999999997}]},{"osd":3,"last update":"Sat Oct 14 10:36:57 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.36799999999999999}]},{"osd":4,"last update":"Sat Oct 14 10:36:57 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.49299999999999999}]},{"osd":5,"last update":"Sat Oct 14 10:36:57 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.63100000000000001}]},{"osd":7,"last update":"Sat Oct 14 10:36:57 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.51000000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542163,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":376,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":385024,"data_stored":143770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.254,"15min":1.254},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":16.855,"5min":16.855,"15min":16.855},"last":0.378},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":19.353000000000002,"5min":19.353000000000002,"15min":19.353000000000002},"last":0.35699999999999998}]},{"osd":2,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":3.0670000000000002,"5min":3.0670000000000002,"15min":3.0670000000000002},"last":0.26700000000000002},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.442}]},{"osd":3,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":10.005000000000001,"5min":10.005000000000001,"15min":10.005000000000001},"last":0.41699999999999998}]},{"osd":4,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":2.9500000000000002,"5min":2.9500000000000002,"15min":2.9500000000000002},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":5.7270000000000003,"5min":5.7270000000000003,"15min":5.7270000000000003},"last":0.79700000000000004}]},{"osd":5,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.129,"15min":1.129},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":9.8800000000000008,"5min":9.8800000000000008,"15min":9.8800000000000008},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":3.0350000000000001,"5min":3.0350000000000001,"15min":3.0350000000000001},"last":0.78200000000000003}]},{"osd":6,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":5.681,"5min":5.681,"15min":5.681},"last":0.75600000000000001},{"interface":"front","average":{"1min":1.173,"5min":1.173,"15min":1.173},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":9.9380000000000006,"5min":9.9380000000000006,"15min":9.9380000000000006},"last":0.58999999999999997}]},{"osd":7,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":9.9730000000000008,"5min":9.9730000000000008,"15min":9.9730000000000008},"last":0.626},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":3.0009999999999999,"5min":3.0009999999999999,"15min":3.0009999999999999},"last":0.73799999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542162,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27960,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343880,"statfs":{"total":96636764160,"available":96608133120,"internally_reserved":0,"allocated":962560,"data_stored":732504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.62,"5min":0.62,"15min":0.62},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.495,"5min":0.495,"15min":0.495},"last":0.28000000000000003}]},{"osd":2,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"last":0.377}]},{"osd":3,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"last":0.433}]},{"osd":4,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.496}]},{"osd":5,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.621}]},{"osd":6,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.47399999999999998}]},{"osd":7,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.53900000000000003}]}]},{"osd":2,"up_from":14,"seq":60129542163,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":372736,"data_stored":142206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":1.429,"5min":1.429,"15min":1.429},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.36199999999999999}]},{"osd":1,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":1.5269999999999999,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.40500000000000003}]},{"osd":3,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":4.0629999999999997,"5min":4.0629999999999997,"15min":4.0629999999999997},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.381,"5min":1.381,"15min":1.381},"last":0.42899999999999999}]},{"osd":4,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.58399999999999996}]},{"osd":5,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":3.8599999999999999,"5min":3.8599999999999999,"15min":3.8599999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.464,"5min":1.464,"15min":1.464},"last":0.68899999999999995}]},{"osd":6,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.753},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":3.9820000000000002,"5min":3.9820000000000002,"15min":3.9820000000000002},"last":0.63600000000000001}]},{"osd":7,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":3.9049999999999998,"5min":3.9049999999999998,"15min":3.9049999999999998},"last":0.61699999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542163,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":376,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":385024,"data_stored":143770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.308},{"interface":"front","average":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"last":0.26800000000000002}]},{"osd":1,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"last":0.35899999999999999}]},{"osd":2,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"last":0.371},{"interface":"front","average":{"1min":0.437,"5min":0.437,"15min":0.437},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.379}]},{"osd":4,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":6.0350000000000001,"5min":6.0350000000000001,"15min":6.0350000000000001},"last":0.59899999999999998}]},{"osd":5,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.628},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":6.0540000000000003,"5min":6.0540000000000003,"15min":6.0540000000000003},"last":0.58899999999999997}]},{"osd":6,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.67000000000000004}]},{"osd":7,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":6.0819999999999999,"5min":6.0819999999999999,"15min":6.0819999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.61299999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542162,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27388,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344452,"statfs":{"total":96636764160,"available":96608718848,"internally_reserved":0,"allocated":376832,"data_stored":142240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 10:36:56 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.47699999999999998}]},{"osd":1,"last update":"Sat Oct 14 10:36:56 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.39000000000000001}]},{"osd":2,"last update":"Sat Oct 14 10:36:56 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.42699999999999999}]},{"osd":3,"last update":"Sat Oct 14 10:36:56 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.48999999999999999}]},{"osd":5,"last update":"Sat Oct 14 10:36:56 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.60299999999999998}]},{"osd":6,"last update":"Sat Oct 14 10:36:56 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.61899999999999999}]},{"osd":7,"last update":"Sat Oct 14 10:36:56 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.66300000000000003}]}]},{"osd":5,"up_from":14,"seq":60129542163,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27388,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344452,"statfs":{"total":96636764160,"available":96608718848,"internally_reserved":0,"allocated":376832,"data_stored":142808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.377}]},{"osd":1,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.623},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.48099999999999998}]},{"osd":2,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.58399999999999996}]},{"osd":3,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.66000000000000003}]},{"osd":4,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.60899999999999999}]},{"osd":6,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.59799999999999998}]},{"osd":7,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.67700000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-14T10:37:15.205 INFO:tasks.ceph:pgid 5.3f last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.205 INFO:tasks.ceph:pgid 5.3e last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.206 INFO:tasks.ceph:pgid 5.3d last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.206 INFO:tasks.ceph:pgid 5.3c last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.207 INFO:tasks.ceph:pgid 5.3b last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.207 INFO:tasks.ceph:pgid 5.3a last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.207 INFO:tasks.ceph:pgid 5.39 last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.208 INFO:tasks.ceph:pgid 5.38 last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.208 INFO:tasks.ceph:pgid 5.37 last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.208 INFO:tasks.ceph:pgid 5.36 last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.209 INFO:tasks.ceph:pgid 5.35 last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.209 INFO:tasks.ceph:pgid 5.34 last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.210 INFO:tasks.ceph:pgid 5.33 last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.210 INFO:tasks.ceph:pgid 5.32 last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.210 INFO:tasks.ceph:pgid 5.31 last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.211 INFO:tasks.ceph:pgid 5.30 last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.211 INFO:tasks.ceph:pgid 5.2f last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.211 INFO:tasks.ceph:pgid 5.2e last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.212 INFO:tasks.ceph:pgid 5.2d last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.212 INFO:tasks.ceph:pgid 5.2c last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.213 INFO:tasks.ceph:pgid 5.2b last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.213 INFO:tasks.ceph:pgid 5.2a last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.213 INFO:tasks.ceph:pgid 5.29 last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.214 INFO:tasks.ceph:pgid 5.28 last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.214 INFO:tasks.ceph:pgid 5.27 last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.214 INFO:tasks.ceph:pgid 5.26 last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.215 INFO:tasks.ceph:pgid 5.25 last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.215 INFO:tasks.ceph:pgid 5.24 last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.216 INFO:tasks.ceph:pgid 5.23 last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.216 INFO:tasks.ceph:pgid 5.22 last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.216 INFO:tasks.ceph:pgid 5.21 last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.217 INFO:tasks.ceph:pgid 5.20 last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.217 INFO:tasks.ceph:pgid 5.1e last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.217 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2023-10-14T10:36:58.035327+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=58, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.218 INFO:tasks.ceph:pgid 5.1f last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.218 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2023-10-14T10:36:58.035327+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=58, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.219 INFO:tasks.ceph:pgid 5.1c last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.219 INFO:tasks.ceph:pgid 4.1d last_scrub_stamp 2023-10-14T10:36:58.035327+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=58, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.219 INFO:tasks.ceph:pgid 5.1d last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.220 INFO:tasks.ceph:pgid 4.1c last_scrub_stamp 2023-10-14T10:36:58.035327+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=58, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.220 INFO:tasks.ceph:pgid 5.1a last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.220 INFO:tasks.ceph:pgid 4.1b last_scrub_stamp 2023-10-14T10:36:58.035327+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=58, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.221 INFO:tasks.ceph:pgid 5.1b last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.221 INFO:tasks.ceph:pgid 4.1a last_scrub_stamp 2023-10-14T10:36:58.035327+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=58, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.222 INFO:tasks.ceph:pgid 5.18 last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.222 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2023-10-14T10:36:58.035327+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=58, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.222 INFO:tasks.ceph:pgid 5.19 last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.223 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2023-10-14T10:36:58.035327+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=58, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.223 INFO:tasks.ceph:pgid 5.16 last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.223 INFO:tasks.ceph:pgid 4.17 last_scrub_stamp 2023-10-14T10:36:58.035327+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=58, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.224 INFO:tasks.ceph:pgid 5.17 last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.224 INFO:tasks.ceph:pgid 4.16 last_scrub_stamp 2023-10-14T10:36:58.035327+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=58, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.224 INFO:tasks.ceph:pgid 5.14 last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.225 INFO:tasks.ceph:pgid 4.15 last_scrub_stamp 2023-10-14T10:36:58.035327+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=58, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.225 INFO:tasks.ceph:pgid 5.15 last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.226 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2023-10-14T10:36:58.035327+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=58, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.226 INFO:tasks.ceph:pgid 5.12 last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.226 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2023-10-14T10:36:58.035327+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=58, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.227 INFO:tasks.ceph:pgid 5.13 last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.227 INFO:tasks.ceph:pgid 4.12 last_scrub_stamp 2023-10-14T10:36:58.035327+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=58, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.227 INFO:tasks.ceph:pgid 5.10 last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.228 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2023-10-14T10:36:58.035327+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=58, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.228 INFO:tasks.ceph:pgid 5.11 last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.228 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2023-10-14T10:36:58.035327+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=58, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.229 INFO:tasks.ceph:pgid 5.e last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.229 INFO:tasks.ceph:pgid 4.f last_scrub_stamp 2023-10-14T10:36:58.035327+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=58, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.230 INFO:tasks.ceph:pgid 5.f last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.230 INFO:tasks.ceph:pgid 4.e last_scrub_stamp 2023-10-14T10:36:58.035327+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=58, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.230 INFO:tasks.ceph:pgid 5.c last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.231 INFO:tasks.ceph:pgid 4.d last_scrub_stamp 2023-10-14T10:36:58.035327+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=58, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.231 INFO:tasks.ceph:pgid 5.d last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.232 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2023-10-14T10:36:58.035327+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=58, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.232 INFO:tasks.ceph:pgid 5.a last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.232 INFO:tasks.ceph:pgid 4.b last_scrub_stamp 2023-10-14T10:36:58.035327+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=58, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.233 INFO:tasks.ceph:pgid 5.b last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.233 INFO:tasks.ceph:pgid 4.a last_scrub_stamp 2023-10-14T10:36:58.035327+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=58, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.233 INFO:tasks.ceph:pgid 5.8 last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.234 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2023-10-14T10:36:58.035327+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=58, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.234 INFO:tasks.ceph:pgid 5.9 last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.234 INFO:tasks.ceph:pgid 4.8 last_scrub_stamp 2023-10-14T10:36:58.035327+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=58, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.235 INFO:tasks.ceph:pgid 5.6 last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.235 INFO:tasks.ceph:pgid 4.7 last_scrub_stamp 2023-10-14T10:36:58.035327+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=58, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.236 INFO:tasks.ceph:pgid 5.7 last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.236 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2023-10-14T10:36:58.035327+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=58, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.236 INFO:tasks.ceph:pgid 5.5 last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.237 INFO:tasks.ceph:pgid 4.4 last_scrub_stamp 2023-10-14T10:36:58.035327+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=58, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.237 INFO:tasks.ceph:pgid 5.2 last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.237 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2023-10-14T10:36:58.035327+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=58, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.238 INFO:tasks.ceph:pgid 5.3 last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.238 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2023-10-14T10:36:58.035327+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=58, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.239 INFO:tasks.ceph:pgid 5.0 last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.239 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2023-10-14T10:36:58.035327+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=58, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.239 INFO:tasks.ceph:pgid 5.1 last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.240 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2023-10-14T10:36:58.035327+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=58, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.240 INFO:tasks.ceph:pgid 5.4 last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.240 INFO:tasks.ceph:pgid 4.5 last_scrub_stamp 2023-10-14T10:36:58.035327+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=58, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.241 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-10-14T10:35:58.675572+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=35, tm_sec=58, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:15.241 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-14T10:37:35.242 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T10:37:35.383 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:35.381+0000 7fd06e4a0700 1 Processor -- start 2023-10-14T10:37:35.383 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:35.381+0000 7fd06e4a0700 1 -- start start 2023-10-14T10:37:35.385 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:35.381+0000 7fd06e4a0700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd06815f130 0x7fd0680cede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:35.385 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:35.381+0000 7fd06e4a0700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fd0680b28b0 0x7fd06806a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:35.386 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:35.381+0000 7fd06e4a0700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0680b3530 con 0x7fd06806ad80 2023-10-14T10:37:35.386 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:35.381+0000 7fd06e4a0700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0680cde00 con 0x7fd06815f130 2023-10-14T10:37:35.386 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:35.381+0000 7fd06e4a0700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd06806b1d0 con 0x7fd0680b28b0 2023-10-14T10:37:35.387 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:35.382+0000 7fd067fff700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd06815f130 0x7fd0680cede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:35.387 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:35.382+0000 7fd067fff700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd06815f130 0x7fd0680cede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46294/0 (socket says 172.21.15.67:46294) 2023-10-14T10:37:35.387 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:35.382+0000 7fd067fff700 1 -- 172.21.15.67:0/3389016239 learned_addr learned my addr 172.21.15.67:0/3389016239 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:37:35.387 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:35.382+0000 7fd0677fe700 1 --2- 172.21.15.67:0/3389016239 >> v2:172.21.15.72:3300/0 conn(0x7fd0680b28b0 0x7fd06806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:35.388 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:35.382+0000 7fd067fff700 1 -- 172.21.15.67:0/3389016239 >> v2:172.21.15.72:3300/0 conn(0x7fd0680b28b0 msgr2=0x7fd06806a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:37:35.388 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:35.382+0000 7fd067fff700 1 --2- 172.21.15.67:0/3389016239 >> v2:172.21.15.72:3300/0 conn(0x7fd0680b28b0 0x7fd06806a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:35.388 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:35.382+0000 7fd067fff700 1 -- 172.21.15.67:0/3389016239 >> v1:172.21.15.72:6789/0 conn(0x7fd06806ad80 legacy=0x7fd068071af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:37:35.389 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:35.382+0000 7fd067fff700 1 -- 172.21.15.67:0/3389016239 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0680cd180 con 0x7fd06815f130 2023-10-14T10:37:35.389 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:35.382+0000 7fd067fff700 1 --2- 172.21.15.67:0/3389016239 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd06815f130 0x7fd0680cede0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fd0680c3f30 tx=0x7fd058009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dd0d179398e7c54 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:35.389 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:35.382+0000 7fd066ffd700 1 -- 172.21.15.67:0/3389016239 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd058004230 con 0x7fd06815f130 2023-10-14T10:37:35.389 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:35.382+0000 7fd066ffd700 1 -- 172.21.15.67:0/3389016239 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd058004390 con 0x7fd06815f130 2023-10-14T10:37:35.390 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:35.382+0000 7fd066ffd700 1 -- 172.21.15.67:0/3389016239 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0580036a0 con 0x7fd06815f130 2023-10-14T10:37:35.390 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:35.383+0000 7fd06e4a0700 1 -- 172.21.15.67:0/3389016239 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd06815f130 msgr2=0x7fd0680cede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:35.390 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:35.383+0000 7fd06e4a0700 1 --2- 172.21.15.67:0/3389016239 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd06815f130 0x7fd0680cede0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fd0680c3f30 tx=0x7fd058009bd0 comp rx=0 tx=0).stop 2023-10-14T10:37:35.390 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:35.383+0000 7fd06e4a0700 1 -- 172.21.15.67:0/3389016239 shutdown_connections 2023-10-14T10:37:35.391 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:35.383+0000 7fd06e4a0700 1 --2- 172.21.15.67:0/3389016239 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd06815f130 0x7fd0680cede0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:35.391 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:35.383+0000 7fd06e4a0700 1 --2- 172.21.15.67:0/3389016239 >> v2:172.21.15.72:3300/0 conn(0x7fd0680b28b0 0x7fd06806a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:35.391 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:35.383+0000 7fd06e4a0700 1 -- 172.21.15.67:0/3389016239 >> 172.21.15.67:0/3389016239 conn(0x7fd06806dae0 msgr2=0x7fd06806def0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:37:35.392 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:35.383+0000 7fd06e4a0700 1 -- 172.21.15.67:0/3389016239 shutdown_connections 2023-10-14T10:37:35.392 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:35.383+0000 7fd06e4a0700 1 -- 172.21.15.67:0/3389016239 wait complete. 2023-10-14T10:37:35.396 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:35.384+0000 7fd06e4a0700 1 Processor -- start 2023-10-14T10:37:35.396 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:35.384+0000 7fd06e4a0700 1 -- start start 2023-10-14T10:37:35.397 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:35.385+0000 7fd06e4a0700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd06806ad80 0x7fd068147c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:35.397 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:35.385+0000 7fd06e4a0700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0680b28b0 0x7fd0681f4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:35.397 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:35.385+0000 7fd06e4a0700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fd06815f130 0x7fd06815aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:35.397 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:35.385+0000 7fd06e4a0700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd06815b6d0 con 0x7fd0680b28b0 2023-10-14T10:37:35.398 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:35.385+0000 7fd06e4a0700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd06815b3f0 con 0x7fd06806ad80 2023-10-14T10:37:35.398 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:35.385+0000 7fd06e4a0700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0680c8720 con 0x7fd06815f130 2023-10-14T10:37:35.398 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:35.385+0000 7fd0677fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0680b28b0 0x7fd0681f4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:35.399 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:35.385+0000 7fd0677fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0680b28b0 0x7fd0681f4790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46302/0 (socket says 172.21.15.67:46302) 2023-10-14T10:37:35.399 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:35.385+0000 7fd0677fe700 1 -- 172.21.15.67:0/4041856689 learned_addr learned my addr 172.21.15.67:0/4041856689 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:37:35.399 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:35.385+0000 7fd067fff700 1 --2- 172.21.15.67:0/4041856689 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd06806ad80 0x7fd068147c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:35.399 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:35.385+0000 7fd06ca3d700 1 --2- 172.21.15.67:0/4041856689 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fd06815f130 0x7fd06815aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:35.400 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:35.385+0000 7fd0677fe700 1 -- 172.21.15.67:0/4041856689 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fd06815f130 msgr2=0x7fd06815aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:35.400 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:35.385+0000 7fd0677fe700 1 --2- 172.21.15.67:0/4041856689 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fd06815f130 0x7fd06815aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:35.400 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:35.385+0000 7fd0677fe700 1 -- 172.21.15.67:0/4041856689 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd06806ad80 msgr2=0x7fd068147c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:35.401 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:35.385+0000 7fd0677fe700 1 --2- 172.21.15.67:0/4041856689 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd06806ad80 0x7fd068147c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:35.401 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:35.385+0000 7fd0677fe700 1 -- 172.21.15.67:0/4041856689 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd058009550 con 0x7fd0680b28b0 2023-10-14T10:37:35.402 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:35.386+0000 7fd06ca3d700 1 --2- 172.21.15.67:0/4041856689 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fd06815f130 0x7fd06815aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-14T10:37:35.402 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:35.386+0000 7fd0677fe700 1 --2- 172.21.15.67:0/4041856689 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0680b28b0 0x7fd0681f4790 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fd05c006eb0 tx=0x7fd05c00eac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:35.403 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:35.386+0000 7fd064ff9700 1 -- 172.21.15.67:0/4041856689 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd05c005c20 con 0x7fd0680b28b0 2023-10-14T10:37:35.403 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:35.386+0000 7fd064ff9700 1 -- 172.21.15.67:0/4041856689 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd05c005d80 con 0x7fd0680b28b0 2023-10-14T10:37:35.403 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:35.386+0000 7fd06e4a0700 1 -- 172.21.15.67:0/4041856689 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0680c8a00 con 0x7fd0680b28b0 2023-10-14T10:37:35.403 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:35.386+0000 7fd064ff9700 1 -- 172.21.15.67:0/4041856689 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd05c006260 con 0x7fd0680b28b0 2023-10-14T10:37:35.404 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:35.386+0000 7fd06e4a0700 1 -- 172.21.15.67:0/4041856689 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0681f4cd0 con 0x7fd0680b28b0 2023-10-14T10:37:35.404 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:35.387+0000 7fd06e4a0700 1 -- 172.21.15.67:0/4041856689 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0681f4ec0 con 0x7fd0680b28b0 2023-10-14T10:37:35.404 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:35.387+0000 7fd064ff9700 1 -- 172.21.15.67:0/4041856689 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd05c0063c0 con 0x7fd0680b28b0 2023-10-14T10:37:35.405 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:35.388+0000 7fd064ff9700 1 --2- 172.21.15.67:0/4041856689 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fd05406af00 0x7fd05406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:35.405 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:35.388+0000 7fd064ff9700 1 -- 172.21.15.67:0/4041856689 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5729+0+0 (secure 0 0 0) 0x7fd05c088f50 con 0x7fd0680b28b0 2023-10-14T10:37:35.405 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:35.388+0000 7fd067fff700 1 --2- 172.21.15.67:0/4041856689 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fd05406af00 0x7fd05406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:35.405 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:35.389+0000 7fd067fff700 1 --2- 172.21.15.67:0/4041856689 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fd05406af00 0x7fd05406d3c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd058009130 tx=0x7fd058008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:35.406 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:35.392+0000 7fd064ff9700 1 -- 172.21.15.67:0/4041856689 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd05c053ff0 con 0x7fd0680b28b0 2023-10-14T10:37:35.590 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:35.588+0000 7fd06e4a0700 1 -- 172.21.15.67:0/4041856689 --> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd0680c8890 con 0x7fd05406af00 2023-10-14T10:37:35.598 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:35.596+0000 7fd064ff9700 1 -- 172.21.15.67:0/4041856689 <== mgr.4100 v2:172.21.15.72:6832/703488382 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+288844 (secure 0 0 0) 0x7fd0680c8890 con 0x7fd05406af00 2023-10-14T10:37:35.602 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-14T10:37:35.604 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:35.599+0000 7fd06e4a0700 1 -- 172.21.15.67:0/4041856689 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fd05406af00 msgr2=0x7fd05406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:35.605 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:35.599+0000 7fd06e4a0700 1 --2- 172.21.15.67:0/4041856689 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fd05406af00 0x7fd05406d3c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd058009130 tx=0x7fd058008030 comp rx=0 tx=0).stop 2023-10-14T10:37:35.605 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:35.599+0000 7fd06e4a0700 1 -- 172.21.15.67:0/4041856689 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0680b28b0 msgr2=0x7fd0681f4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:35.605 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:35.599+0000 7fd06e4a0700 1 --2- 172.21.15.67:0/4041856689 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0680b28b0 0x7fd0681f4790 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fd05c006eb0 tx=0x7fd05c00eac0 comp rx=0 tx=0).stop 2023-10-14T10:37:35.606 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:35.599+0000 7fd06e4a0700 1 -- 172.21.15.67:0/4041856689 shutdown_connections 2023-10-14T10:37:35.606 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:35.599+0000 7fd06e4a0700 1 --2- 172.21.15.67:0/4041856689 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fd05406af00 0x7fd05406d3c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:35.606 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:35.599+0000 7fd06e4a0700 1 --2- 172.21.15.67:0/4041856689 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fd06815f130 0x7fd06815aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:35.606 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:35.599+0000 7fd06e4a0700 1 --2- 172.21.15.67:0/4041856689 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd06806ad80 0x7fd068147c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:35.607 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:35.599+0000 7fd06e4a0700 1 --2- 172.21.15.67:0/4041856689 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0680b28b0 0x7fd0681f4790 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:35.607 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:35.599+0000 7fd06e4a0700 1 -- 172.21.15.67:0/4041856689 >> 172.21.15.67:0/4041856689 conn(0x7fd06806dae0 msgr2=0x7fd0680b4630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:37:35.607 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:35.600+0000 7fd06e4a0700 1 -- 172.21.15.67:0/4041856689 shutdown_connections 2023-10-14T10:37:35.608 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:35.600+0000 7fd06e4a0700 1 -- 172.21.15.67:0/4041856689 wait complete. 2023-10-14T10:37:35.608 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2023-10-14T10:37:35.614 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":75,"stamp":"2023-10-14T10:37:33.777325+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220232,"kb_used_data":4104,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754488,"statfs":{"total":773094113280,"available":772868595712,"internally_reserved":0,"allocated":4202496,"data_stored":2322644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12742,"internal_metadata":221236794},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002964"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:02.021993+0000","last_change":"2023-10-14T10:37:00.075102+0000","last_active":"2023-10-14T10:37:02.021993+0000","last_peered":"2023-10-14T10:37:02.021993+0000","last_clean":"2023-10-14T10:37:02.021993+0000","last_became_active":"2023-10-14T10:37:00.074851+0000","last_became_peered":"2023-10-14T10:37:00.074851+0000","last_unstale":"2023-10-14T10:37:02.021993+0000","last_undegraded":"2023-10-14T10:37:02.021993+0000","last_fullsized":"2023-10-14T10:37:02.021993+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.822679+0000","last_change":"2023-10-14T10:37:00.068871+0000","last_active":"2023-10-14T10:37:01.822679+0000","last_peered":"2023-10-14T10:37:01.822679+0000","last_clean":"2023-10-14T10:37:01.822679+0000","last_became_active":"2023-10-14T10:37:00.068586+0000","last_became_peered":"2023-10-14T10:37:00.068586+0000","last_unstale":"2023-10-14T10:37:01.822679+0000","last_undegraded":"2023-10-14T10:37:01.822679+0000","last_fullsized":"2023-10-14T10:37:01.822679+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.824253+0000","last_change":"2023-10-14T10:37:00.080988+0000","last_active":"2023-10-14T10:37:01.824253+0000","last_peered":"2023-10-14T10:37:01.824253+0000","last_clean":"2023-10-14T10:37:01.824253+0000","last_became_active":"2023-10-14T10:37:00.080476+0000","last_became_peered":"2023-10-14T10:37:00.080476+0000","last_unstale":"2023-10-14T10:37:01.824253+0000","last_undegraded":"2023-10-14T10:37:01.824253+0000","last_fullsized":"2023-10-14T10:37:01.824253+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.820430+0000","last_change":"2023-10-14T10:37:00.077598+0000","last_active":"2023-10-14T10:37:01.820430+0000","last_peered":"2023-10-14T10:37:01.820430+0000","last_clean":"2023-10-14T10:37:01.820430+0000","last_became_active":"2023-10-14T10:37:00.077322+0000","last_became_peered":"2023-10-14T10:37:00.077322+0000","last_unstale":"2023-10-14T10:37:01.820430+0000","last_undegraded":"2023-10-14T10:37:01.820430+0000","last_fullsized":"2023-10-14T10:37:01.820430+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:25.474346+0000","last_change":"2023-10-14T10:37:25.474436+0000","last_active":"2023-10-14T10:37:25.474346+0000","last_peered":"2023-10-14T10:37:25.474346+0000","last_clean":"2023-10-14T10:37:25.474346+0000","last_became_active":"2023-10-14T10:37:00.068055+0000","last_became_peered":"2023-10-14T10:37:00.068055+0000","last_unstale":"2023-10-14T10:37:25.474346+0000","last_undegraded":"2023-10-14T10:37:25.474346+0000","last_fullsized":"2023-10-14T10:37:25.474346+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:25.474208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:25.474208+0000","last_clean_scrub_stamp":"2023-10-14T10:37:25.474208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:57:27.388802+0000","scrub_duration":0.001660716,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.822562+0000","last_change":"2023-10-14T10:37:00.072629+0000","last_active":"2023-10-14T10:37:01.822562+0000","last_peered":"2023-10-14T10:37:01.822562+0000","last_clean":"2023-10-14T10:37:01.822562+0000","last_became_active":"2023-10-14T10:37:00.072345+0000","last_became_peered":"2023-10-14T10:37:00.072345+0000","last_unstale":"2023-10-14T10:37:01.822562+0000","last_undegraded":"2023-10-14T10:37:01.822562+0000","last_fullsized":"2023-10-14T10:37:01.822562+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:16.975562+0000","last_change":"2023-10-14T10:37:16.975628+0000","last_active":"2023-10-14T10:37:16.975562+0000","last_peered":"2023-10-14T10:37:16.975562+0000","last_clean":"2023-10-14T10:37:16.975562+0000","last_became_active":"2023-10-14T10:37:00.072368+0000","last_became_peered":"2023-10-14T10:37:00.072368+0000","last_unstale":"2023-10-14T10:37:16.975562+0000","last_undegraded":"2023-10-14T10:37:16.975562+0000","last_fullsized":"2023-10-14T10:37:16.975562+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:16.975436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:16.975436+0000","last_clean_scrub_stamp":"2023-10-14T10:37:16.975436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:31:42.878362+0000","scrub_duration":0.0016077960000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:29.970861+0000","last_change":"2023-10-14T10:37:29.970909+0000","last_active":"2023-10-14T10:37:29.970861+0000","last_peered":"2023-10-14T10:37:29.970861+0000","last_clean":"2023-10-14T10:37:29.970861+0000","last_became_active":"2023-10-14T10:37:00.073792+0000","last_became_peered":"2023-10-14T10:37:00.073792+0000","last_unstale":"2023-10-14T10:37:29.970861+0000","last_undegraded":"2023-10-14T10:37:29.970861+0000","last_fullsized":"2023-10-14T10:37:29.970861+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:29.970754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:29.970754+0000","last_clean_scrub_stamp":"2023-10-14T10:37:29.970754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T14:08:18.066248+0000","scrub_duration":0.001847558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.824332+0000","last_change":"2023-10-14T10:37:00.081745+0000","last_active":"2023-10-14T10:37:01.824332+0000","last_peered":"2023-10-14T10:37:01.824332+0000","last_clean":"2023-10-14T10:37:01.824332+0000","last_became_active":"2023-10-14T10:37:00.081451+0000","last_became_peered":"2023-10-14T10:37:00.081451+0000","last_unstale":"2023-10-14T10:37:01.824332+0000","last_undegraded":"2023-10-14T10:37:01.824332+0000","last_fullsized":"2023-10-14T10:37:01.824332+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.820490+0000","last_change":"2023-10-14T10:37:00.071138+0000","last_active":"2023-10-14T10:37:01.820490+0000","last_peered":"2023-10-14T10:37:01.820490+0000","last_clean":"2023-10-14T10:37:01.820490+0000","last_became_active":"2023-10-14T10:37:00.070667+0000","last_became_peered":"2023-10-14T10:37:00.070667+0000","last_unstale":"2023-10-14T10:37:01.820490+0000","last_undegraded":"2023-10-14T10:37:01.820490+0000","last_fullsized":"2023-10-14T10:37:01.820490+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:27.985170+0000","last_change":"2023-10-14T10:37:27.985252+0000","last_active":"2023-10-14T10:37:27.985170+0000","last_peered":"2023-10-14T10:37:27.985170+0000","last_clean":"2023-10-14T10:37:27.985170+0000","last_became_active":"2023-10-14T10:37:00.072639+0000","last_became_peered":"2023-10-14T10:37:00.072639+0000","last_unstale":"2023-10-14T10:37:27.985170+0000","last_undegraded":"2023-10-14T10:37:27.985170+0000","last_fullsized":"2023-10-14T10:37:27.985170+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:27.985038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:27.985038+0000","last_clean_scrub_stamp":"2023-10-14T10:37:27.985038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:26:54.065382+0000","scrub_duration":0.001768588,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.824214+0000","last_change":"2023-10-14T10:37:00.069681+0000","last_active":"2023-10-14T10:37:01.824214+0000","last_peered":"2023-10-14T10:37:01.824214+0000","last_clean":"2023-10-14T10:37:01.824214+0000","last_became_active":"2023-10-14T10:37:00.069373+0000","last_became_peered":"2023-10-14T10:37:00.069373+0000","last_unstale":"2023-10-14T10:37:01.824214+0000","last_undegraded":"2023-10-14T10:37:01.824214+0000","last_fullsized":"2023-10-14T10:37:01.824214+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:19.586632+0000","last_change":"2023-10-14T10:37:19.586687+0000","last_active":"2023-10-14T10:37:19.586632+0000","last_peered":"2023-10-14T10:37:19.586632+0000","last_clean":"2023-10-14T10:37:19.586632+0000","last_became_active":"2023-10-14T10:37:00.074473+0000","last_became_peered":"2023-10-14T10:37:00.074473+0000","last_unstale":"2023-10-14T10:37:19.586632+0000","last_undegraded":"2023-10-14T10:37:19.586632+0000","last_fullsized":"2023-10-14T10:37:19.586632+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:19.586500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:19.586500+0000","last_clean_scrub_stamp":"2023-10-14T10:37:19.586500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:57:11.433994+0000","scrub_duration":0.001596793,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.822571+0000","last_change":"2023-10-14T10:37:00.073986+0000","last_active":"2023-10-14T10:37:01.822571+0000","last_peered":"2023-10-14T10:37:01.822571+0000","last_clean":"2023-10-14T10:37:01.822571+0000","last_became_active":"2023-10-14T10:37:00.073651+0000","last_became_peered":"2023-10-14T10:37:00.073651+0000","last_unstale":"2023-10-14T10:37:01.822571+0000","last_undegraded":"2023-10-14T10:37:01.822571+0000","last_fullsized":"2023-10-14T10:37:01.822571+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.824169+0000","last_change":"2023-10-14T10:37:00.072277+0000","last_active":"2023-10-14T10:37:01.824169+0000","last_peered":"2023-10-14T10:37:01.824169+0000","last_clean":"2023-10-14T10:37:01.824169+0000","last_became_active":"2023-10-14T10:37:00.071999+0000","last_became_peered":"2023-10-14T10:37:00.071999+0000","last_unstale":"2023-10-14T10:37:01.824169+0000","last_undegraded":"2023-10-14T10:37:01.824169+0000","last_fullsized":"2023-10-14T10:37:01.824169+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:22.094919+0000","last_change":"2023-10-14T10:37:22.094971+0000","last_active":"2023-10-14T10:37:22.094919+0000","last_peered":"2023-10-14T10:37:22.094919+0000","last_clean":"2023-10-14T10:37:22.094919+0000","last_became_active":"2023-10-14T10:37:00.071509+0000","last_became_peered":"2023-10-14T10:37:00.071509+0000","last_unstale":"2023-10-14T10:37:22.094919+0000","last_undegraded":"2023-10-14T10:37:22.094919+0000","last_fullsized":"2023-10-14T10:37:22.094919+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:22.094817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:22.094817+0000","last_clean_scrub_stamp":"2023-10-14T10:37:22.094817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:57:03.789898+0000","scrub_duration":0.0017607390000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:21.067339+0000","last_change":"2023-10-14T10:37:21.067417+0000","last_active":"2023-10-14T10:37:21.067339+0000","last_peered":"2023-10-14T10:37:21.067339+0000","last_clean":"2023-10-14T10:37:21.067339+0000","last_became_active":"2023-10-14T10:37:00.071946+0000","last_became_peered":"2023-10-14T10:37:00.071946+0000","last_unstale":"2023-10-14T10:37:21.067339+0000","last_undegraded":"2023-10-14T10:37:21.067339+0000","last_fullsized":"2023-10-14T10:37:21.067339+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:21.067219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:21.067219+0000","last_clean_scrub_stamp":"2023-10-14T10:37:21.067219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T12:48:36.500809+0000","scrub_duration":0.0016934249999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.822669+0000","last_change":"2023-10-14T10:37:00.080581+0000","last_active":"2023-10-14T10:37:01.822669+0000","last_peered":"2023-10-14T10:37:01.822669+0000","last_clean":"2023-10-14T10:37:01.822669+0000","last_became_active":"2023-10-14T10:37:00.080288+0000","last_became_peered":"2023-10-14T10:37:00.080288+0000","last_unstale":"2023-10-14T10:37:01.822669+0000","last_undegraded":"2023-10-14T10:37:01.822669+0000","last_fullsized":"2023-10-14T10:37:01.822669+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:13.890326+0000","last_change":"2023-10-14T10:37:13.890365+0000","last_active":"2023-10-14T10:37:13.890326+0000","last_peered":"2023-10-14T10:37:13.890326+0000","last_clean":"2023-10-14T10:37:13.890326+0000","last_became_active":"2023-10-14T10:37:00.080186+0000","last_became_peered":"2023-10-14T10:37:00.080186+0000","last_unstale":"2023-10-14T10:37:13.890326+0000","last_undegraded":"2023-10-14T10:37:13.890326+0000","last_fullsized":"2023-10-14T10:37:13.890326+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:13.890234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:13.890234+0000","last_clean_scrub_stamp":"2023-10-14T10:37:13.890234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:48:50.907546+0000","scrub_duration":0.001362254,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:26.440259+0000","last_change":"2023-10-14T10:37:26.440336+0000","last_active":"2023-10-14T10:37:26.440259+0000","last_peered":"2023-10-14T10:37:26.440259+0000","last_clean":"2023-10-14T10:37:26.440259+0000","last_became_active":"2023-10-14T10:37:00.071796+0000","last_became_peered":"2023-10-14T10:37:00.071796+0000","last_unstale":"2023-10-14T10:37:26.440259+0000","last_undegraded":"2023-10-14T10:37:26.440259+0000","last_fullsized":"2023-10-14T10:37:26.440259+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:26.440115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:26.440115+0000","last_clean_scrub_stamp":"2023-10-14T10:37:26.440115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:54:37.394808+0000","scrub_duration":0.0021402970000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.824156+0000","last_change":"2023-10-14T10:37:00.080796+0000","last_active":"2023-10-14T10:37:01.824156+0000","last_peered":"2023-10-14T10:37:01.824156+0000","last_clean":"2023-10-14T10:37:01.824156+0000","last_became_active":"2023-10-14T10:37:00.080526+0000","last_became_peered":"2023-10-14T10:37:00.080526+0000","last_unstale":"2023-10-14T10:37:01.824156+0000","last_undegraded":"2023-10-14T10:37:01.824156+0000","last_fullsized":"2023-10-14T10:37:01.824156+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.824113+0000","last_change":"2023-10-14T10:37:00.080350+0000","last_active":"2023-10-14T10:37:01.824113+0000","last_peered":"2023-10-14T10:37:01.824113+0000","last_clean":"2023-10-14T10:37:01.824113+0000","last_became_active":"2023-10-14T10:37:00.080062+0000","last_became_peered":"2023-10-14T10:37:00.080062+0000","last_unstale":"2023-10-14T10:37:01.824113+0000","last_undegraded":"2023-10-14T10:37:01.824113+0000","last_fullsized":"2023-10-14T10:37:01.824113+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:24.101006+0000","last_change":"2023-10-14T10:37:24.101056+0000","last_active":"2023-10-14T10:37:24.101006+0000","last_peered":"2023-10-14T10:37:24.101006+0000","last_clean":"2023-10-14T10:37:24.101006+0000","last_became_active":"2023-10-14T10:37:00.070897+0000","last_became_peered":"2023-10-14T10:37:00.070897+0000","last_unstale":"2023-10-14T10:37:24.101006+0000","last_undegraded":"2023-10-14T10:37:24.101006+0000","last_fullsized":"2023-10-14T10:37:24.101006+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:24.100899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:24.100899+0000","last_clean_scrub_stamp":"2023-10-14T10:37:24.100899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:58:06.157813+0000","scrub_duration":0.001908496,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:26.820065+0000","last_change":"2023-10-14T10:37:26.820141+0000","last_active":"2023-10-14T10:37:26.820065+0000","last_peered":"2023-10-14T10:37:26.820065+0000","last_clean":"2023-10-14T10:37:26.820065+0000","last_became_active":"2023-10-14T10:37:00.069246+0000","last_became_peered":"2023-10-14T10:37:00.069246+0000","last_unstale":"2023-10-14T10:37:26.820065+0000","last_undegraded":"2023-10-14T10:37:26.820065+0000","last_fullsized":"2023-10-14T10:37:26.820065+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:26.819916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:26.819916+0000","last_clean_scrub_stamp":"2023-10-14T10:37:26.819916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T21:40:50.616911+0000","scrub_duration":0.0021912780000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.820501+0000","last_change":"2023-10-14T10:37:00.070144+0000","last_active":"2023-10-14T10:37:01.820501+0000","last_peered":"2023-10-14T10:37:01.820501+0000","last_clean":"2023-10-14T10:37:01.820501+0000","last_became_active":"2023-10-14T10:37:00.069871+0000","last_became_peered":"2023-10-14T10:37:00.069871+0000","last_unstale":"2023-10-14T10:37:01.820501+0000","last_undegraded":"2023-10-14T10:37:01.820501+0000","last_fullsized":"2023-10-14T10:37:01.820501+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.822659+0000","last_change":"2023-10-14T10:37:00.076474+0000","last_active":"2023-10-14T10:37:01.822659+0000","last_peered":"2023-10-14T10:37:01.822659+0000","last_clean":"2023-10-14T10:37:01.822659+0000","last_became_active":"2023-10-14T10:37:00.076169+0000","last_became_peered":"2023-10-14T10:37:00.076169+0000","last_unstale":"2023-10-14T10:37:01.822659+0000","last_undegraded":"2023-10-14T10:37:01.822659+0000","last_fullsized":"2023-10-14T10:37:01.822659+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.820465+0000","last_change":"2023-10-14T10:37:00.076568+0000","last_active":"2023-10-14T10:37:01.820465+0000","last_peered":"2023-10-14T10:37:01.820465+0000","last_clean":"2023-10-14T10:37:01.820465+0000","last_became_active":"2023-10-14T10:37:00.076263+0000","last_became_peered":"2023-10-14T10:37:00.076263+0000","last_unstale":"2023-10-14T10:37:01.820465+0000","last_undegraded":"2023-10-14T10:37:01.820465+0000","last_fullsized":"2023-10-14T10:37:01.820465+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:30.765830+0000","last_change":"2023-10-14T10:37:30.765882+0000","last_active":"2023-10-14T10:37:30.765830+0000","last_peered":"2023-10-14T10:37:30.765830+0000","last_clean":"2023-10-14T10:37:30.765830+0000","last_became_active":"2023-10-14T10:37:00.079178+0000","last_became_peered":"2023-10-14T10:37:00.079178+0000","last_unstale":"2023-10-14T10:37:30.765830+0000","last_undegraded":"2023-10-14T10:37:30.765830+0000","last_fullsized":"2023-10-14T10:37:30.765830+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:30.765720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:30.765720+0000","last_clean_scrub_stamp":"2023-10-14T10:37:30.765720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T12:48:46.199309+0000","scrub_duration":0.0022229910000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.824180+0000","last_change":"2023-10-14T10:37:00.080675+0000","last_active":"2023-10-14T10:37:01.824180+0000","last_peered":"2023-10-14T10:37:01.824180+0000","last_clean":"2023-10-14T10:37:01.824180+0000","last_became_active":"2023-10-14T10:37:00.080171+0000","last_became_peered":"2023-10-14T10:37:00.080171+0000","last_unstale":"2023-10-14T10:37:01.824180+0000","last_undegraded":"2023-10-14T10:37:01.824180+0000","last_fullsized":"2023-10-14T10:37:01.824180+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:30.956480+0000","last_change":"2023-10-14T10:37:30.956555+0000","last_active":"2023-10-14T10:37:30.956480+0000","last_peered":"2023-10-14T10:37:30.956480+0000","last_clean":"2023-10-14T10:37:30.956480+0000","last_became_active":"2023-10-14T10:37:00.071882+0000","last_became_peered":"2023-10-14T10:37:00.071882+0000","last_unstale":"2023-10-14T10:37:30.956480+0000","last_undegraded":"2023-10-14T10:37:30.956480+0000","last_fullsized":"2023-10-14T10:37:30.956480+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:30.956369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:30.956369+0000","last_clean_scrub_stamp":"2023-10-14T10:37:30.956369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:18:44.736053+0000","scrub_duration":0.001666013,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:02.021931+0000","last_change":"2023-10-14T10:37:00.081816+0000","last_active":"2023-10-14T10:37:02.021931+0000","last_peered":"2023-10-14T10:37:02.021931+0000","last_clean":"2023-10-14T10:37:02.021931+0000","last_became_active":"2023-10-14T10:37:00.081312+0000","last_became_peered":"2023-10-14T10:37:00.081312+0000","last_unstale":"2023-10-14T10:37:02.021931+0000","last_undegraded":"2023-10-14T10:37:02.021931+0000","last_fullsized":"2023-10-14T10:37:02.021931+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:15.957200+0000","last_change":"2023-10-14T10:37:15.957269+0000","last_active":"2023-10-14T10:37:15.957200+0000","last_peered":"2023-10-14T10:37:15.957200+0000","last_clean":"2023-10-14T10:37:15.957200+0000","last_became_active":"2023-10-14T10:37:00.078441+0000","last_became_peered":"2023-10-14T10:37:00.078441+0000","last_unstale":"2023-10-14T10:37:15.957200+0000","last_undegraded":"2023-10-14T10:37:15.957200+0000","last_fullsized":"2023-10-14T10:37:15.957200+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:15.957063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:15.957063+0000","last_clean_scrub_stamp":"2023-10-14T10:37:15.957063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:48:52.974375+0000","scrub_duration":0.001645093,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:25.103789+0000","last_change":"2023-10-14T10:37:25.103841+0000","last_active":"2023-10-14T10:37:25.103789+0000","last_peered":"2023-10-14T10:37:25.103789+0000","last_clean":"2023-10-14T10:37:25.103789+0000","last_became_active":"2023-10-14T10:37:00.070945+0000","last_became_peered":"2023-10-14T10:37:00.070945+0000","last_unstale":"2023-10-14T10:37:25.103789+0000","last_undegraded":"2023-10-14T10:37:25.103789+0000","last_fullsized":"2023-10-14T10:37:25.103789+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:25.103682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:25.103682+0000","last_clean_scrub_stamp":"2023-10-14T10:37:25.103682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:53:19.037516+0000","scrub_duration":0.0021103710000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":25,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:21.985058+0000","last_change":"2023-10-14T10:37:21.985135+0000","last_active":"2023-10-14T10:37:21.985058+0000","last_peered":"2023-10-14T10:37:21.985058+0000","last_clean":"2023-10-14T10:37:21.985058+0000","last_became_active":"2023-10-14T10:36:59.066488+0000","last_became_peered":"2023-10-14T10:36:59.066488+0000","last_unstale":"2023-10-14T10:37:21.985058+0000","last_undegraded":"2023-10-14T10:37:21.985058+0000","last_fullsized":"2023-10-14T10:37:21.985058+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'4","last_scrub_stamp":"2023-10-14T10:37:21.984953+0000","last_deep_scrub":"32'4","last_deep_scrub_stamp":"2023-10-14T10:37:21.984953+0000","last_clean_scrub_stamp":"2023-10-14T10:37:21.984953+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:31:39.071011+0000","scrub_duration":0.0021399549999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:31.160865+0000","last_change":"2023-10-14T10:37:31.160912+0000","last_active":"2023-10-14T10:37:31.160865+0000","last_peered":"2023-10-14T10:37:31.160865+0000","last_clean":"2023-10-14T10:37:31.160865+0000","last_became_active":"2023-10-14T10:37:00.077734+0000","last_became_peered":"2023-10-14T10:37:00.077734+0000","last_unstale":"2023-10-14T10:37:31.160865+0000","last_undegraded":"2023-10-14T10:37:31.160865+0000","last_fullsized":"2023-10-14T10:37:31.160865+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:31.160755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:31.160755+0000","last_clean_scrub_stamp":"2023-10-14T10:37:31.160755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T14:19:54.429928+0000","scrub_duration":0.0014218779999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:22.082855+0000","last_change":"2023-10-14T10:37:22.082911+0000","last_active":"2023-10-14T10:37:22.082855+0000","last_peered":"2023-10-14T10:37:22.082855+0000","last_clean":"2023-10-14T10:37:22.082855+0000","last_became_active":"2023-10-14T10:36:59.070463+0000","last_became_peered":"2023-10-14T10:36:59.070463+0000","last_unstale":"2023-10-14T10:37:22.082855+0000","last_undegraded":"2023-10-14T10:37:22.082855+0000","last_fullsized":"2023-10-14T10:37:22.082855+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1","last_scrub_stamp":"2023-10-14T10:37:22.082747+0000","last_deep_scrub":"32'1","last_deep_scrub_stamp":"2023-10-14T10:37:22.082747+0000","last_clean_scrub_stamp":"2023-10-14T10:37:22.082747+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:06:51.432100+0000","scrub_duration":0.0021442739999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.824214+0000","last_change":"2023-10-14T10:37:00.080768+0000","last_active":"2023-10-14T10:37:01.824214+0000","last_peered":"2023-10-14T10:37:01.824214+0000","last_clean":"2023-10-14T10:37:01.824214+0000","last_became_active":"2023-10-14T10:37:00.080160+0000","last_became_peered":"2023-10-14T10:37:00.080160+0000","last_unstale":"2023-10-14T10:37:01.824214+0000","last_undegraded":"2023-10-14T10:37:01.824214+0000","last_fullsized":"2023-10-14T10:37:01.824214+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":23,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:29.788305+0000","last_change":"2023-10-14T10:37:29.788362+0000","last_active":"2023-10-14T10:37:29.788305+0000","last_peered":"2023-10-14T10:37:29.788305+0000","last_clean":"2023-10-14T10:37:29.788305+0000","last_became_active":"2023-10-14T10:36:59.067616+0000","last_became_peered":"2023-10-14T10:36:59.067616+0000","last_unstale":"2023-10-14T10:37:29.788305+0000","last_undegraded":"2023-10-14T10:37:29.788305+0000","last_fullsized":"2023-10-14T10:37:29.788305+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2023-10-14T10:37:29.788187+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2023-10-14T10:37:29.788187+0000","last_clean_scrub_stamp":"2023-10-14T10:37:29.788187+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:26:55.868531+0000","scrub_duration":0.0020300269999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.820479+0000","last_change":"2023-10-14T10:37:00.079200+0000","last_active":"2023-10-14T10:37:01.820479+0000","last_peered":"2023-10-14T10:37:01.820479+0000","last_clean":"2023-10-14T10:37:01.820479+0000","last_became_active":"2023-10-14T10:37:00.078829+0000","last_became_peered":"2023-10-14T10:37:00.078829+0000","last_unstale":"2023-10-14T10:37:01.820479+0000","last_undegraded":"2023-10-14T10:37:01.820479+0000","last_fullsized":"2023-10-14T10:37:01.820479+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":23,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:12.636047+0000","last_change":"2023-10-14T10:37:12.636098+0000","last_active":"2023-10-14T10:37:12.636047+0000","last_peered":"2023-10-14T10:37:12.636047+0000","last_clean":"2023-10-14T10:37:12.636047+0000","last_became_active":"2023-10-14T10:36:59.065608+0000","last_became_peered":"2023-10-14T10:36:59.065608+0000","last_unstale":"2023-10-14T10:37:12.636047+0000","last_undegraded":"2023-10-14T10:37:12.636047+0000","last_fullsized":"2023-10-14T10:37:12.636047+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2023-10-14T10:37:12.635967+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2023-10-14T10:37:12.635967+0000","last_clean_scrub_stamp":"2023-10-14T10:37:12.635967+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:31:27.596001+0000","scrub_duration":0.0015686820000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.824439+0000","last_change":"2023-10-14T10:37:00.077115+0000","last_active":"2023-10-14T10:37:01.824439+0000","last_peered":"2023-10-14T10:37:01.824439+0000","last_clean":"2023-10-14T10:37:01.824439+0000","last_became_active":"2023-10-14T10:37:00.076840+0000","last_became_peered":"2023-10-14T10:37:00.076840+0000","last_unstale":"2023-10-14T10:37:01.824439+0000","last_undegraded":"2023-10-14T10:37:01.824439+0000","last_fullsized":"2023-10-14T10:37:01.824439+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:15.636426+0000","last_change":"2023-10-14T10:37:15.636479+0000","last_active":"2023-10-14T10:37:15.636426+0000","last_peered":"2023-10-14T10:37:15.636426+0000","last_clean":"2023-10-14T10:37:15.636426+0000","last_became_active":"2023-10-14T10:36:59.069192+0000","last_became_peered":"2023-10-14T10:36:59.069192+0000","last_unstale":"2023-10-14T10:37:15.636426+0000","last_undegraded":"2023-10-14T10:37:15.636426+0000","last_fullsized":"2023-10-14T10:37:15.636426+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1","last_scrub_stamp":"2023-10-14T10:37:15.636316+0000","last_deep_scrub":"32'1","last_deep_scrub_stamp":"2023-10-14T10:37:15.636316+0000","last_clean_scrub_stamp":"2023-10-14T10:37:15.636316+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:18:11.324774+0000","scrub_duration":0.0019544810000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.824189+0000","last_change":"2023-10-14T10:37:00.068786+0000","last_active":"2023-10-14T10:37:01.824189+0000","last_peered":"2023-10-14T10:37:01.824189+0000","last_clean":"2023-10-14T10:37:01.824189+0000","last_became_active":"2023-10-14T10:37:00.068517+0000","last_became_peered":"2023-10-14T10:37:00.068517+0000","last_unstale":"2023-10-14T10:37:01.824189+0000","last_undegraded":"2023-10-14T10:37:01.824189+0000","last_fullsized":"2023-10-14T10:37:01.824189+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.824136+0000","last_change":"2023-10-14T10:36:59.069445+0000","last_active":"2023-10-14T10:37:01.824136+0000","last_peered":"2023-10-14T10:37:01.824136+0000","last_clean":"2023-10-14T10:37:01.824136+0000","last_became_active":"2023-10-14T10:36:59.069151+0000","last_became_peered":"2023-10-14T10:36:59.069151+0000","last_unstale":"2023-10-14T10:37:01.824136+0000","last_undegraded":"2023-10-14T10:37:01.824136+0000","last_fullsized":"2023-10-14T10:37:01.824136+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.824008+0000","last_change":"2023-10-14T10:37:00.076984+0000","last_active":"2023-10-14T10:37:01.824008+0000","last_peered":"2023-10-14T10:37:01.824008+0000","last_clean":"2023-10-14T10:37:01.824008+0000","last_became_active":"2023-10-14T10:37:00.076706+0000","last_became_peered":"2023-10-14T10:37:00.076706+0000","last_unstale":"2023-10-14T10:37:01.824008+0000","last_undegraded":"2023-10-14T10:37:01.824008+0000","last_fullsized":"2023-10-14T10:37:01.824008+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:26.865191+0000","last_change":"2023-10-14T10:37:26.865278+0000","last_active":"2023-10-14T10:37:26.865191+0000","last_peered":"2023-10-14T10:37:26.865191+0000","last_clean":"2023-10-14T10:37:26.865191+0000","last_became_active":"2023-10-14T10:36:59.062075+0000","last_became_peered":"2023-10-14T10:36:59.062075+0000","last_unstale":"2023-10-14T10:37:26.865191+0000","last_undegraded":"2023-10-14T10:37:26.865191+0000","last_fullsized":"2023-10-14T10:37:26.865191+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:26.865054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:26.865054+0000","last_clean_scrub_stamp":"2023-10-14T10:37:26.865054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:38:54.815977+0000","scrub_duration":0.00159884,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.820360+0000","last_change":"2023-10-14T10:37:00.067486+0000","last_active":"2023-10-14T10:37:01.820360+0000","last_peered":"2023-10-14T10:37:01.820360+0000","last_clean":"2023-10-14T10:37:01.820360+0000","last_became_active":"2023-10-14T10:37:00.067187+0000","last_became_peered":"2023-10-14T10:37:00.067187+0000","last_unstale":"2023-10-14T10:37:01.820360+0000","last_undegraded":"2023-10-14T10:37:01.820360+0000","last_fullsized":"2023-10-14T10:37:01.820360+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:29.738486+0000","last_change":"2023-10-14T10:37:29.738579+0000","last_active":"2023-10-14T10:37:29.738486+0000","last_peered":"2023-10-14T10:37:29.738486+0000","last_clean":"2023-10-14T10:37:29.738486+0000","last_became_active":"2023-10-14T10:36:59.064990+0000","last_became_peered":"2023-10-14T10:36:59.064990+0000","last_unstale":"2023-10-14T10:37:29.738486+0000","last_undegraded":"2023-10-14T10:37:29.738486+0000","last_fullsized":"2023-10-14T10:37:29.738486+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:29.738368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:29.738368+0000","last_clean_scrub_stamp":"2023-10-14T10:37:29.738368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:27:09.238644+0000","scrub_duration":0.0016393549999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.824313+0000","last_change":"2023-10-14T10:37:00.077465+0000","last_active":"2023-10-14T10:37:01.824313+0000","last_peered":"2023-10-14T10:37:01.824313+0000","last_clean":"2023-10-14T10:37:01.824313+0000","last_became_active":"2023-10-14T10:37:00.077152+0000","last_became_peered":"2023-10-14T10:37:00.077152+0000","last_unstale":"2023-10-14T10:37:01.824313+0000","last_undegraded":"2023-10-14T10:37:01.824313+0000","last_fullsized":"2023-10-14T10:37:01.824313+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.822688+0000","last_change":"2023-10-14T10:36:59.061903+0000","last_active":"2023-10-14T10:37:01.822688+0000","last_peered":"2023-10-14T10:37:01.822688+0000","last_clean":"2023-10-14T10:37:01.822688+0000","last_became_active":"2023-10-14T10:36:59.061597+0000","last_became_peered":"2023-10-14T10:36:59.061597+0000","last_unstale":"2023-10-14T10:37:01.822688+0000","last_undegraded":"2023-10-14T10:37:01.822688+0000","last_fullsized":"2023-10-14T10:37:01.822688+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.822699+0000","last_change":"2023-10-14T10:37:00.076213+0000","last_active":"2023-10-14T10:37:01.822699+0000","last_peered":"2023-10-14T10:37:01.822699+0000","last_clean":"2023-10-14T10:37:01.822699+0000","last_became_active":"2023-10-14T10:37:00.075889+0000","last_became_peered":"2023-10-14T10:37:00.075889+0000","last_unstale":"2023-10-14T10:37:01.822699+0000","last_undegraded":"2023-10-14T10:37:01.822699+0000","last_fullsized":"2023-10-14T10:37:01.822699+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:20.017238+0000","last_change":"2023-10-14T10:37:20.017309+0000","last_active":"2023-10-14T10:37:20.017238+0000","last_peered":"2023-10-14T10:37:20.017238+0000","last_clean":"2023-10-14T10:37:20.017238+0000","last_became_active":"2023-10-14T10:36:59.069731+0000","last_became_peered":"2023-10-14T10:36:59.069731+0000","last_unstale":"2023-10-14T10:37:20.017238+0000","last_undegraded":"2023-10-14T10:37:20.017238+0000","last_fullsized":"2023-10-14T10:37:20.017238+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:20.017076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:20.017076+0000","last_clean_scrub_stamp":"2023-10-14T10:37:20.017076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T18:58:11.111987+0000","scrub_duration":0.001527189,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.822592+0000","last_change":"2023-10-14T10:37:00.072592+0000","last_active":"2023-10-14T10:37:01.822592+0000","last_peered":"2023-10-14T10:37:01.822592+0000","last_clean":"2023-10-14T10:37:01.822592+0000","last_became_active":"2023-10-14T10:37:00.072288+0000","last_became_peered":"2023-10-14T10:37:00.072288+0000","last_unstale":"2023-10-14T10:37:01.822592+0000","last_undegraded":"2023-10-14T10:37:01.822592+0000","last_fullsized":"2023-10-14T10:37:01.822592+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":24,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:24.791215+0000","last_change":"2023-10-14T10:37:24.791349+0000","last_active":"2023-10-14T10:37:24.791215+0000","last_peered":"2023-10-14T10:37:24.791215+0000","last_clean":"2023-10-14T10:37:24.791215+0000","last_became_active":"2023-10-14T10:36:59.061700+0000","last_became_peered":"2023-10-14T10:36:59.061700+0000","last_unstale":"2023-10-14T10:37:24.791215+0000","last_undegraded":"2023-10-14T10:37:24.791215+0000","last_fullsized":"2023-10-14T10:37:24.791215+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3","last_scrub_stamp":"2023-10-14T10:37:24.791071+0000","last_deep_scrub":"32'3","last_deep_scrub_stamp":"2023-10-14T10:37:24.791071+0000","last_clean_scrub_stamp":"2023-10-14T10:37:24.791071+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:49:01.808383+0000","scrub_duration":0.0023118180000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.824270+0000","last_change":"2023-10-14T10:37:00.074977+0000","last_active":"2023-10-14T10:37:01.824270+0000","last_peered":"2023-10-14T10:37:01.824270+0000","last_clean":"2023-10-14T10:37:01.824270+0000","last_became_active":"2023-10-14T10:37:00.074700+0000","last_became_peered":"2023-10-14T10:37:00.074700+0000","last_unstale":"2023-10-14T10:37:01.824270+0000","last_undegraded":"2023-10-14T10:37:01.824270+0000","last_fullsized":"2023-10-14T10:37:01.824270+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":19,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.822603+0000","last_change":"2023-10-14T10:36:59.069400+0000","last_active":"2023-10-14T10:37:01.822603+0000","last_peered":"2023-10-14T10:37:01.822603+0000","last_clean":"2023-10-14T10:37:01.822603+0000","last_became_active":"2023-10-14T10:36:59.069043+0000","last_became_peered":"2023-10-14T10:36:59.069043+0000","last_unstale":"2023-10-14T10:37:01.822603+0000","last_undegraded":"2023-10-14T10:37:01.822603+0000","last_fullsized":"2023-10-14T10:37:01.822603+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.820378+0000","last_change":"2023-10-14T10:37:00.068613+0000","last_active":"2023-10-14T10:37:01.820378+0000","last_peered":"2023-10-14T10:37:01.820378+0000","last_clean":"2023-10-14T10:37:01.820378+0000","last_became_active":"2023-10-14T10:37:00.065345+0000","last_became_peered":"2023-10-14T10:37:00.065345+0000","last_unstale":"2023-10-14T10:37:01.820378+0000","last_undegraded":"2023-10-14T10:37:01.820378+0000","last_fullsized":"2023-10-14T10:37:01.820378+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"33'4","reported_seq":25,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:30.752534+0000","last_change":"2023-10-14T10:37:30.752613+0000","last_active":"2023-10-14T10:37:30.752534+0000","last_peered":"2023-10-14T10:37:30.752534+0000","last_clean":"2023-10-14T10:37:30.752534+0000","last_became_active":"2023-10-14T10:36:59.065569+0000","last_became_peered":"2023-10-14T10:36:59.065569+0000","last_unstale":"2023-10-14T10:37:30.752534+0000","last_undegraded":"2023-10-14T10:37:30.752534+0000","last_fullsized":"2023-10-14T10:37:30.752534+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'4","last_scrub_stamp":"2023-10-14T10:37:30.752424+0000","last_deep_scrub":"33'4","last_deep_scrub_stamp":"2023-10-14T10:37:30.752424+0000","last_clean_scrub_stamp":"2023-10-14T10:37:30.752424+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T12:29:20.109316+0000","scrub_duration":0.0039809850000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.822583+0000","last_change":"2023-10-14T10:37:00.079137+0000","last_active":"2023-10-14T10:37:01.822583+0000","last_peered":"2023-10-14T10:37:01.822583+0000","last_clean":"2023-10-14T10:37:01.822583+0000","last_became_active":"2023-10-14T10:37:00.078844+0000","last_became_peered":"2023-10-14T10:37:00.078844+0000","last_unstale":"2023-10-14T10:37:01.822583+0000","last_undegraded":"2023-10-14T10:37:01.822583+0000","last_fullsized":"2023-10-14T10:37:01.822583+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:21.918946+0000","last_change":"2023-10-14T10:37:21.919039+0000","last_active":"2023-10-14T10:37:21.918946+0000","last_peered":"2023-10-14T10:37:21.918946+0000","last_clean":"2023-10-14T10:37:21.918946+0000","last_became_active":"2023-10-14T10:36:59.066952+0000","last_became_peered":"2023-10-14T10:36:59.066952+0000","last_unstale":"2023-10-14T10:37:21.918946+0000","last_undegraded":"2023-10-14T10:37:21.918946+0000","last_fullsized":"2023-10-14T10:37:21.918946+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:21.918813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:21.918813+0000","last_clean_scrub_stamp":"2023-10-14T10:37:21.918813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:53:15.852647+0000","scrub_duration":0.002202907,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.824399+0000","last_change":"2023-10-14T10:37:00.078646+0000","last_active":"2023-10-14T10:37:01.824399+0000","last_peered":"2023-10-14T10:37:01.824399+0000","last_clean":"2023-10-14T10:37:01.824399+0000","last_became_active":"2023-10-14T10:37:00.078353+0000","last_became_peered":"2023-10-14T10:37:00.078353+0000","last_unstale":"2023-10-14T10:37:01.824399+0000","last_undegraded":"2023-10-14T10:37:01.824399+0000","last_fullsized":"2023-10-14T10:37:01.824399+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:19.819168+0000","last_change":"2023-10-14T10:37:19.819220+0000","last_active":"2023-10-14T10:37:19.819168+0000","last_peered":"2023-10-14T10:37:19.819168+0000","last_clean":"2023-10-14T10:37:19.819168+0000","last_became_active":"2023-10-14T10:36:59.062380+0000","last_became_peered":"2023-10-14T10:36:59.062380+0000","last_unstale":"2023-10-14T10:37:19.819168+0000","last_undegraded":"2023-10-14T10:37:19.819168+0000","last_fullsized":"2023-10-14T10:37:19.819168+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:19.819047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:19.819047+0000","last_clean_scrub_stamp":"2023-10-14T10:37:19.819047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:16:49.960306+0000","scrub_duration":0.001802477,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:24.980015+0000","last_change":"2023-10-14T10:37:24.980059+0000","last_active":"2023-10-14T10:37:24.980015+0000","last_peered":"2023-10-14T10:37:24.980015+0000","last_clean":"2023-10-14T10:37:24.980015+0000","last_became_active":"2023-10-14T10:37:00.077806+0000","last_became_peered":"2023-10-14T10:37:00.077806+0000","last_unstale":"2023-10-14T10:37:24.980015+0000","last_undegraded":"2023-10-14T10:37:24.980015+0000","last_fullsized":"2023-10-14T10:37:24.980015+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:24.979906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:24.979906+0000","last_clean_scrub_stamp":"2023-10-14T10:37:24.979906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T17:28:16.344297+0000","scrub_duration":0.0015380680000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":29,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:17.814572+0000","last_change":"2023-10-14T10:37:17.814685+0000","last_active":"2023-10-14T10:37:17.814572+0000","last_peered":"2023-10-14T10:37:17.814572+0000","last_clean":"2023-10-14T10:37:17.814572+0000","last_became_active":"2023-10-14T10:36:59.067727+0000","last_became_peered":"2023-10-14T10:36:59.067727+0000","last_unstale":"2023-10-14T10:37:17.814572+0000","last_undegraded":"2023-10-14T10:37:17.814572+0000","last_fullsized":"2023-10-14T10:37:17.814572+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'8","last_scrub_stamp":"2023-10-14T10:37:17.814452+0000","last_deep_scrub":"32'8","last_deep_scrub_stamp":"2023-10-14T10:37:17.814452+0000","last_clean_scrub_stamp":"2023-10-14T10:37:17.814452+0000","objects_scrubbed":4,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:25:42.460457+0000","scrub_duration":0.002353843,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.824274+0000","last_change":"2023-10-14T10:37:00.076926+0000","last_active":"2023-10-14T10:37:01.824274+0000","last_peered":"2023-10-14T10:37:01.824274+0000","last_clean":"2023-10-14T10:37:01.824274+0000","last_became_active":"2023-10-14T10:37:00.076658+0000","last_became_peered":"2023-10-14T10:37:00.076658+0000","last_unstale":"2023-10-14T10:37:01.824274+0000","last_undegraded":"2023-10-14T10:37:01.824274+0000","last_fullsized":"2023-10-14T10:37:01.824274+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:14.880170+0000","last_change":"2023-10-14T10:37:14.880213+0000","last_active":"2023-10-14T10:37:14.880170+0000","last_peered":"2023-10-14T10:37:14.880170+0000","last_clean":"2023-10-14T10:37:14.880170+0000","last_became_active":"2023-10-14T10:36:59.071406+0000","last_became_peered":"2023-10-14T10:36:59.071406+0000","last_unstale":"2023-10-14T10:37:14.880170+0000","last_undegraded":"2023-10-14T10:37:14.880170+0000","last_fullsized":"2023-10-14T10:37:14.880170+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1","last_scrub_stamp":"2023-10-14T10:37:14.880064+0000","last_deep_scrub":"32'1","last_deep_scrub_stamp":"2023-10-14T10:37:14.880064+0000","last_clean_scrub_stamp":"2023-10-14T10:37:14.880064+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T17:09:37.404058+0000","scrub_duration":0.0017514850000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.824351+0000","last_change":"2023-10-14T10:37:00.080060+0000","last_active":"2023-10-14T10:37:01.824351+0000","last_peered":"2023-10-14T10:37:01.824351+0000","last_clean":"2023-10-14T10:37:01.824351+0000","last_became_active":"2023-10-14T10:37:00.079795+0000","last_became_peered":"2023-10-14T10:37:00.079795+0000","last_unstale":"2023-10-14T10:37:01.824351+0000","last_undegraded":"2023-10-14T10:37:01.824351+0000","last_fullsized":"2023-10-14T10:37:01.824351+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.824297+0000","last_change":"2023-10-14T10:36:59.065271+0000","last_active":"2023-10-14T10:37:01.824297+0000","last_peered":"2023-10-14T10:37:01.824297+0000","last_clean":"2023-10-14T10:37:01.824297+0000","last_became_active":"2023-10-14T10:36:59.064959+0000","last_became_peered":"2023-10-14T10:36:59.064959+0000","last_unstale":"2023-10-14T10:37:01.824297+0000","last_undegraded":"2023-10-14T10:37:01.824297+0000","last_fullsized":"2023-10-14T10:37:01.824297+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.820453+0000","last_change":"2023-10-14T10:37:00.077751+0000","last_active":"2023-10-14T10:37:01.820453+0000","last_peered":"2023-10-14T10:37:01.820453+0000","last_clean":"2023-10-14T10:37:01.820453+0000","last_became_active":"2023-10-14T10:37:00.077459+0000","last_became_peered":"2023-10-14T10:37:00.077459+0000","last_unstale":"2023-10-14T10:37:01.820453+0000","last_undegraded":"2023-10-14T10:37:01.820453+0000","last_fullsized":"2023-10-14T10:37:01.820453+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.824249+0000","last_change":"2023-10-14T10:36:59.058893+0000","last_active":"2023-10-14T10:37:01.824249+0000","last_peered":"2023-10-14T10:37:01.824249+0000","last_clean":"2023-10-14T10:37:01.824249+0000","last_became_active":"2023-10-14T10:36:59.058558+0000","last_became_peered":"2023-10-14T10:36:59.058558+0000","last_unstale":"2023-10-14T10:37:01.824249+0000","last_undegraded":"2023-10-14T10:37:01.824249+0000","last_fullsized":"2023-10-14T10:37:01.824249+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:31.349037+0000","last_change":"2023-10-14T10:37:31.349116+0000","last_active":"2023-10-14T10:37:31.349037+0000","last_peered":"2023-10-14T10:37:31.349037+0000","last_clean":"2023-10-14T10:37:31.349037+0000","last_became_active":"2023-10-14T10:37:00.071167+0000","last_became_peered":"2023-10-14T10:37:00.071167+0000","last_unstale":"2023-10-14T10:37:31.349037+0000","last_undegraded":"2023-10-14T10:37:31.349037+0000","last_fullsized":"2023-10-14T10:37:31.349037+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:31.348892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:31.348892+0000","last_clean_scrub_stamp":"2023-10-14T10:37:31.348892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T17:21:44.324153+0000","scrub_duration":0.001948319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:21.671764+0000","last_change":"2023-10-14T10:37:21.671823+0000","last_active":"2023-10-14T10:37:21.671764+0000","last_peered":"2023-10-14T10:37:21.671764+0000","last_clean":"2023-10-14T10:37:21.671764+0000","last_became_active":"2023-10-14T10:36:59.069110+0000","last_became_peered":"2023-10-14T10:36:59.069110+0000","last_unstale":"2023-10-14T10:37:21.671764+0000","last_undegraded":"2023-10-14T10:37:21.671764+0000","last_fullsized":"2023-10-14T10:37:21.671764+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:21.671648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:21.671648+0000","last_clean_scrub_stamp":"2023-10-14T10:37:21.671648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T14:19:44.940820+0000","scrub_duration":0.0022981799999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:29.416731+0000","last_change":"2023-10-14T10:37:29.416782+0000","last_active":"2023-10-14T10:37:29.416731+0000","last_peered":"2023-10-14T10:37:29.416731+0000","last_clean":"2023-10-14T10:37:29.416731+0000","last_became_active":"2023-10-14T10:37:00.077175+0000","last_became_peered":"2023-10-14T10:37:00.077175+0000","last_unstale":"2023-10-14T10:37:29.416731+0000","last_undegraded":"2023-10-14T10:37:29.416731+0000","last_fullsized":"2023-10-14T10:37:29.416731+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:29.416599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:29.416599+0000","last_clean_scrub_stamp":"2023-10-14T10:37:29.416599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:22:40.667629+0000","scrub_duration":0.00185724,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":23,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:18.035307+0000","last_change":"2023-10-14T10:37:18.035397+0000","last_active":"2023-10-14T10:37:18.035307+0000","last_peered":"2023-10-14T10:37:18.035307+0000","last_clean":"2023-10-14T10:37:18.035307+0000","last_became_active":"2023-10-14T10:36:59.067345+0000","last_became_peered":"2023-10-14T10:36:59.067345+0000","last_unstale":"2023-10-14T10:37:18.035307+0000","last_undegraded":"2023-10-14T10:37:18.035307+0000","last_fullsized":"2023-10-14T10:37:18.035307+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2023-10-14T10:37:18.035173+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2023-10-14T10:37:18.035173+0000","last_clean_scrub_stamp":"2023-10-14T10:37:18.035173+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:51:19.160977+0000","scrub_duration":0.0020979919999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:20.573039+0000","last_change":"2023-10-14T10:37:20.573138+0000","last_active":"2023-10-14T10:37:20.573039+0000","last_peered":"2023-10-14T10:37:20.573039+0000","last_clean":"2023-10-14T10:37:20.573039+0000","last_became_active":"2023-10-14T10:37:00.077465+0000","last_became_peered":"2023-10-14T10:37:00.077465+0000","last_unstale":"2023-10-14T10:37:20.573039+0000","last_undegraded":"2023-10-14T10:37:20.573039+0000","last_fullsized":"2023-10-14T10:37:20.573039+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:20.572921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:20.572921+0000","last_clean_scrub_stamp":"2023-10-14T10:37:20.572921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:27:44.149277+0000","scrub_duration":0.001569429,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:16.998159+0000","last_change":"2023-10-14T10:37:16.998214+0000","last_active":"2023-10-14T10:37:16.998159+0000","last_peered":"2023-10-14T10:37:16.998159+0000","last_clean":"2023-10-14T10:37:16.998159+0000","last_became_active":"2023-10-14T10:36:59.069269+0000","last_became_peered":"2023-10-14T10:36:59.069269+0000","last_unstale":"2023-10-14T10:37:16.998159+0000","last_undegraded":"2023-10-14T10:37:16.998159+0000","last_fullsized":"2023-10-14T10:37:16.998159+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:16.998041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:16.998041+0000","last_clean_scrub_stamp":"2023-10-14T10:37:16.998041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:52:55.851068+0000","scrub_duration":0.002349617,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:22.513537+0000","last_change":"2023-10-14T10:37:22.513590+0000","last_active":"2023-10-14T10:37:22.513537+0000","last_peered":"2023-10-14T10:37:22.513537+0000","last_clean":"2023-10-14T10:37:22.513537+0000","last_became_active":"2023-10-14T10:37:00.068620+0000","last_became_peered":"2023-10-14T10:37:00.068620+0000","last_unstale":"2023-10-14T10:37:22.513537+0000","last_undegraded":"2023-10-14T10:37:22.513537+0000","last_fullsized":"2023-10-14T10:37:22.513537+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:22.513405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:22.513405+0000","last_clean_scrub_stamp":"2023-10-14T10:37:22.513405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:28:45.771382+0000","scrub_duration":0.0016531670000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:17.591859+0000","last_change":"2023-10-14T10:37:17.591924+0000","last_active":"2023-10-14T10:37:17.591859+0000","last_peered":"2023-10-14T10:37:17.591859+0000","last_clean":"2023-10-14T10:37:17.591859+0000","last_became_active":"2023-10-14T10:36:59.061678+0000","last_became_peered":"2023-10-14T10:36:59.061678+0000","last_unstale":"2023-10-14T10:37:17.591859+0000","last_undegraded":"2023-10-14T10:37:17.591859+0000","last_fullsized":"2023-10-14T10:37:17.591859+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:17.591741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:17.591741+0000","last_clean_scrub_stamp":"2023-10-14T10:37:17.591741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T14:52:36.610528+0000","scrub_duration":0.0021650459999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:27.005340+0000","last_change":"2023-10-14T10:37:27.005388+0000","last_active":"2023-10-14T10:37:27.005340+0000","last_peered":"2023-10-14T10:37:27.005340+0000","last_clean":"2023-10-14T10:37:27.005340+0000","last_became_active":"2023-10-14T10:37:00.071692+0000","last_became_peered":"2023-10-14T10:37:00.071692+0000","last_unstale":"2023-10-14T10:37:27.005340+0000","last_undegraded":"2023-10-14T10:37:27.005340+0000","last_fullsized":"2023-10-14T10:37:27.005340+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:27.005226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:27.005226+0000","last_clean_scrub_stamp":"2023-10-14T10:37:27.005226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:38:54.956149+0000","scrub_duration":0.0015000110000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:13.801179+0000","last_change":"2023-10-14T10:37:13.801233+0000","last_active":"2023-10-14T10:37:13.801179+0000","last_peered":"2023-10-14T10:37:13.801179+0000","last_clean":"2023-10-14T10:37:13.801179+0000","last_became_active":"2023-10-14T10:36:59.063332+0000","last_became_peered":"2023-10-14T10:36:59.063332+0000","last_unstale":"2023-10-14T10:37:13.801179+0000","last_undegraded":"2023-10-14T10:37:13.801179+0000","last_fullsized":"2023-10-14T10:37:13.801179+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:13.801033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:13.801033+0000","last_clean_scrub_stamp":"2023-10-14T10:37:13.801033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T14:51:03.775865+0000","scrub_duration":0.0015480699999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:24.453404+0000","last_change":"2023-10-14T10:37:24.453463+0000","last_active":"2023-10-14T10:37:24.453404+0000","last_peered":"2023-10-14T10:37:24.453404+0000","last_clean":"2023-10-14T10:37:24.453404+0000","last_became_active":"2023-10-14T10:37:00.074859+0000","last_became_peered":"2023-10-14T10:37:00.074859+0000","last_unstale":"2023-10-14T10:37:24.453404+0000","last_undegraded":"2023-10-14T10:37:24.453404+0000","last_fullsized":"2023-10-14T10:37:24.453404+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:24.453276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:24.453276+0000","last_clean_scrub_stamp":"2023-10-14T10:37:24.453276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T21:33:20.669114+0000","scrub_duration":0.002218785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":23,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:23.963239+0000","last_change":"2023-10-14T10:37:23.963301+0000","last_active":"2023-10-14T10:37:23.963239+0000","last_peered":"2023-10-14T10:37:23.963239+0000","last_clean":"2023-10-14T10:37:23.963239+0000","last_became_active":"2023-10-14T10:36:59.069037+0000","last_became_peered":"2023-10-14T10:36:59.069037+0000","last_unstale":"2023-10-14T10:37:23.963239+0000","last_undegraded":"2023-10-14T10:37:23.963239+0000","last_fullsized":"2023-10-14T10:37:23.963239+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2023-10-14T10:37:23.963112+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2023-10-14T10:37:23.963112+0000","last_clean_scrub_stamp":"2023-10-14T10:37:23.963112+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:49:00.980424+0000","scrub_duration":0.0020882309999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.824330+0000","last_change":"2023-10-14T10:37:00.075731+0000","last_active":"2023-10-14T10:37:01.824330+0000","last_peered":"2023-10-14T10:37:01.824330+0000","last_clean":"2023-10-14T10:37:01.824330+0000","last_became_active":"2023-10-14T10:37:00.075491+0000","last_became_peered":"2023-10-14T10:37:00.075491+0000","last_unstale":"2023-10-14T10:37:01.824330+0000","last_undegraded":"2023-10-14T10:37:01.824330+0000","last_fullsized":"2023-10-14T10:37:01.824330+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"33'1","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:19.004875+0000","last_change":"2023-10-14T10:37:19.004932+0000","last_active":"2023-10-14T10:37:19.004875+0000","last_peered":"2023-10-14T10:37:19.004875+0000","last_clean":"2023-10-14T10:37:19.004875+0000","last_became_active":"2023-10-14T10:36:59.066146+0000","last_became_peered":"2023-10-14T10:36:59.066146+0000","last_unstale":"2023-10-14T10:37:19.004875+0000","last_undegraded":"2023-10-14T10:37:19.004875+0000","last_fullsized":"2023-10-14T10:37:19.004875+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'1","last_scrub_stamp":"2023-10-14T10:37:19.004745+0000","last_deep_scrub":"33'1","last_deep_scrub_stamp":"2023-10-14T10:37:19.004745+0000","last_clean_scrub_stamp":"2023-10-14T10:37:19.004745+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T10:48:23.006435+0000","scrub_duration":0.0020016869999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:26.115864+0000","last_change":"2023-10-14T10:37:26.115959+0000","last_active":"2023-10-14T10:37:26.115864+0000","last_peered":"2023-10-14T10:37:26.115864+0000","last_clean":"2023-10-14T10:37:26.115864+0000","last_became_active":"2023-10-14T10:37:00.082570+0000","last_became_peered":"2023-10-14T10:37:00.082570+0000","last_unstale":"2023-10-14T10:37:26.115864+0000","last_undegraded":"2023-10-14T10:37:26.115864+0000","last_fullsized":"2023-10-14T10:37:26.115864+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:26.115725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:26.115725+0000","last_clean_scrub_stamp":"2023-10-14T10:37:26.115725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:49:03.133037+0000","scrub_duration":0.0016919400000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.820442+0000","last_change":"2023-10-14T10:36:59.061936+0000","last_active":"2023-10-14T10:37:01.820442+0000","last_peered":"2023-10-14T10:37:01.820442+0000","last_clean":"2023-10-14T10:37:01.820442+0000","last_became_active":"2023-10-14T10:36:59.061642+0000","last_became_peered":"2023-10-14T10:36:59.061642+0000","last_unstale":"2023-10-14T10:37:01.820442+0000","last_undegraded":"2023-10-14T10:37:01.820442+0000","last_fullsized":"2023-10-14T10:37:01.820442+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:23.976059+0000","last_change":"2023-10-14T10:37:23.976112+0000","last_active":"2023-10-14T10:37:23.976059+0000","last_peered":"2023-10-14T10:37:23.976059+0000","last_clean":"2023-10-14T10:37:23.976059+0000","last_became_active":"2023-10-14T10:37:00.071244+0000","last_became_peered":"2023-10-14T10:37:00.071244+0000","last_unstale":"2023-10-14T10:37:23.976059+0000","last_undegraded":"2023-10-14T10:37:23.976059+0000","last_fullsized":"2023-10-14T10:37:23.976059+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:23.975945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:23.975945+0000","last_clean_scrub_stamp":"2023-10-14T10:37:23.975945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T10:48:27.977635+0000","scrub_duration":0.0019053919999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:12.061873+0000","last_change":"2023-10-14T10:37:12.061941+0000","last_active":"2023-10-14T10:37:12.061873+0000","last_peered":"2023-10-14T10:37:12.061873+0000","last_clean":"2023-10-14T10:37:12.061873+0000","last_became_active":"2023-10-14T10:36:59.066592+0000","last_became_peered":"2023-10-14T10:36:59.066592+0000","last_unstale":"2023-10-14T10:37:12.061873+0000","last_undegraded":"2023-10-14T10:37:12.061873+0000","last_fullsized":"2023-10-14T10:37:12.061873+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:12.061781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:12.061781+0000","last_clean_scrub_stamp":"2023-10-14T10:37:12.061781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:52:50.914807+0000","scrub_duration":0.0016784320000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:23.082899+0000","last_change":"2023-10-14T10:37:23.082942+0000","last_active":"2023-10-14T10:37:23.082899+0000","last_peered":"2023-10-14T10:37:23.082899+0000","last_clean":"2023-10-14T10:37:23.082899+0000","last_became_active":"2023-10-14T10:37:00.075794+0000","last_became_peered":"2023-10-14T10:37:00.075794+0000","last_unstale":"2023-10-14T10:37:23.082899+0000","last_undegraded":"2023-10-14T10:37:23.082899+0000","last_fullsized":"2023-10-14T10:37:23.082899+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:23.082797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:23.082797+0000","last_clean_scrub_stamp":"2023-10-14T10:37:23.082797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T17:09:45.606791+0000","scrub_duration":0.0017434429999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:12.796373+0000","last_change":"2023-10-14T10:37:12.796416+0000","last_active":"2023-10-14T10:37:12.796373+0000","last_peered":"2023-10-14T10:37:12.796373+0000","last_clean":"2023-10-14T10:37:12.796373+0000","last_became_active":"2023-10-14T10:36:59.065163+0000","last_became_peered":"2023-10-14T10:36:59.065163+0000","last_unstale":"2023-10-14T10:37:12.796373+0000","last_undegraded":"2023-10-14T10:37:12.796373+0000","last_fullsized":"2023-10-14T10:37:12.796373+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:12.796279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:12.796279+0000","last_clean_scrub_stamp":"2023-10-14T10:37:12.796279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:06:42.145631+0000","scrub_duration":0.001883366,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.822525+0000","last_change":"2023-10-14T10:37:00.078024+0000","last_active":"2023-10-14T10:37:01.822525+0000","last_peered":"2023-10-14T10:37:01.822525+0000","last_clean":"2023-10-14T10:37:01.822525+0000","last_became_active":"2023-10-14T10:37:00.077621+0000","last_became_peered":"2023-10-14T10:37:00.077621+0000","last_unstale":"2023-10-14T10:37:01.822525+0000","last_undegraded":"2023-10-14T10:37:01.822525+0000","last_fullsized":"2023-10-14T10:37:01.822525+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.824082+0000","last_change":"2023-10-14T10:36:59.064632+0000","last_active":"2023-10-14T10:37:01.824082+0000","last_peered":"2023-10-14T10:37:01.824082+0000","last_clean":"2023-10-14T10:37:01.824082+0000","last_became_active":"2023-10-14T10:36:59.064336+0000","last_became_peered":"2023-10-14T10:36:59.064336+0000","last_unstale":"2023-10-14T10:37:01.824082+0000","last_undegraded":"2023-10-14T10:37:01.824082+0000","last_fullsized":"2023-10-14T10:37:01.824082+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:58.035327+0000","last_clean_scrub_stamp":"2023-10-14T10:36:58.035327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.824042+0000","last_change":"2023-10-14T10:37:00.081042+0000","last_active":"2023-10-14T10:37:01.824042+0000","last_peered":"2023-10-14T10:37:01.824042+0000","last_clean":"2023-10-14T10:37:01.824042+0000","last_became_active":"2023-10-14T10:37:00.080340+0000","last_became_peered":"2023-10-14T10:37:00.080340+0000","last_unstale":"2023-10-14T10:37:01.824042+0000","last_undegraded":"2023-10-14T10:37:01.824042+0000","last_fullsized":"2023-10-14T10:37:01.824042+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:16.807142+0000","last_change":"2023-10-14T10:37:16.807232+0000","last_active":"2023-10-14T10:37:16.807142+0000","last_peered":"2023-10-14T10:37:16.807142+0000","last_clean":"2023-10-14T10:37:16.807142+0000","last_became_active":"2023-10-14T10:36:59.063130+0000","last_became_peered":"2023-10-14T10:36:59.063130+0000","last_unstale":"2023-10-14T10:37:16.807142+0000","last_undegraded":"2023-10-14T10:37:16.807142+0000","last_fullsized":"2023-10-14T10:37:16.807142+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:16.806982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:16.806982+0000","last_clean_scrub_stamp":"2023-10-14T10:37:16.806982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T21:40:40.603978+0000","scrub_duration":0.0021981460000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:28.948640+0000","last_change":"2023-10-14T10:37:28.948692+0000","last_active":"2023-10-14T10:37:28.948640+0000","last_peered":"2023-10-14T10:37:28.948640+0000","last_clean":"2023-10-14T10:37:28.948640+0000","last_became_active":"2023-10-14T10:37:00.068556+0000","last_became_peered":"2023-10-14T10:37:00.068556+0000","last_unstale":"2023-10-14T10:37:28.948640+0000","last_undegraded":"2023-10-14T10:37:28.948640+0000","last_fullsized":"2023-10-14T10:37:28.948640+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:28.948488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:28.948488+0000","last_clean_scrub_stamp":"2023-10-14T10:37:28.948488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:15:09.923968+0000","scrub_duration":0.0020176769999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:14.983447+0000","last_change":"2023-10-14T10:37:14.983500+0000","last_active":"2023-10-14T10:37:14.983447+0000","last_peered":"2023-10-14T10:37:14.983447+0000","last_clean":"2023-10-14T10:37:14.983447+0000","last_became_active":"2023-10-14T10:36:59.066424+0000","last_became_peered":"2023-10-14T10:36:59.066424+0000","last_unstale":"2023-10-14T10:37:14.983447+0000","last_undegraded":"2023-10-14T10:37:14.983447+0000","last_fullsized":"2023-10-14T10:37:14.983447+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:14.983339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:14.983339+0000","last_clean_scrub_stamp":"2023-10-14T10:37:14.983339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:28:20.478594+0000","scrub_duration":0.0016413619999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"33'119","reported_seq":187,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:14.949519+0000","last_change":"2023-10-14T10:37:14.949621+0000","last_active":"2023-10-14T10:37:14.949519+0000","last_peered":"2023-10-14T10:37:14.949519+0000","last_clean":"2023-10-14T10:37:14.949519+0000","last_became_active":"2023-10-14T10:35:59.769199+0000","last_became_peered":"2023-10-14T10:35:59.769199+0000","last_unstale":"2023-10-14T10:37:14.949519+0000","last_undegraded":"2023-10-14T10:37:14.949519+0000","last_fullsized":"2023-10-14T10:37:14.949519+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'119","last_scrub_stamp":"2023-10-14T10:37:14.949400+0000","last_deep_scrub":"33'119","last_deep_scrub_stamp":"2023-10-14T10:37:14.949400+0000","last_clean_scrub_stamp":"2023-10-14T10:37:14.949400+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:53:08.883234+0000","scrub_duration":0.0050812649999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542167,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27960,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343880,"statfs":{"total":96636764160,"available":96608133120,"internally_reserved":0,"allocated":966656,"data_stored":732594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 10:36:59 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":2.4649999999999999,"5min":2.4649999999999999,"15min":2.4649999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.50700000000000001}]},{"osd":1,"last update":"Sat Oct 14 10:36:59 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":2.4289999999999998,"5min":2.4289999999999998,"15min":2.4289999999999998},"last":0.58999999999999997}]},{"osd":2,"last update":"Sat Oct 14 10:36:59 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.57299999999999995}]},{"osd":3,"last update":"Sat Oct 14 10:36:59 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.405,"5min":1.405,"15min":1.405},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.47899999999999998}]},{"osd":4,"last update":"Sat Oct 14 10:36:59 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.73699999999999999}]},{"osd":5,"last update":"Sat Oct 14 10:36:59 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.64900000000000002}]},{"osd":6,"last update":"Sat Oct 14 10:36:59 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.68700000000000006}]}]},{"osd":6,"up_from":14,"seq":60129542168,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":376832,"data_stored":142752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 10:36:57 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.752},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.61299999999999999}]},{"osd":1,"last update":"Sat Oct 14 10:36:57 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.389,"5min":1.389,"15min":1.389},"last":0.68400000000000005}]},{"osd":2,"last update":"Sat Oct 14 10:36:57 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.626}]},{"osd":3,"last update":"Sat Oct 14 10:36:57 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.72599999999999998}]},{"osd":4,"last update":"Sat Oct 14 10:36:57 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.53500000000000003}]},{"osd":5,"last update":"Sat Oct 14 10:36:57 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.55700000000000005}]},{"osd":7,"last update":"Sat Oct 14 10:36:57 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.495},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.51300000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542167,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":376,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":385024,"data_stored":143770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.254,"15min":1.254},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":16.855,"5min":16.855,"15min":16.855},"last":0.35299999999999998},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":19.353000000000002,"5min":19.353000000000002,"15min":19.353000000000002},"last":0.33300000000000002}]},{"osd":2,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":3.0670000000000002,"5min":3.0670000000000002,"15min":3.0670000000000002},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.46899999999999997}]},{"osd":3,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":10.005000000000001,"5min":10.005000000000001,"15min":10.005000000000001},"last":0.53700000000000003}]},{"osd":4,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":2.9500000000000002,"5min":2.9500000000000002,"15min":2.9500000000000002},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":5.7270000000000003,"5min":5.7270000000000003,"15min":5.7270000000000003},"last":0.67500000000000004}]},{"osd":5,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.129,"15min":1.129},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":9.8800000000000008,"5min":9.8800000000000008,"15min":9.8800000000000008},"last":0.62},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":3.0350000000000001,"5min":3.0350000000000001,"15min":3.0350000000000001},"last":0.63600000000000001}]},{"osd":6,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":5.681,"5min":5.681,"15min":5.681},"last":0.73099999999999998},{"interface":"front","average":{"1min":1.173,"5min":1.173,"15min":1.173},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":9.9380000000000006,"5min":9.9380000000000006,"15min":9.9380000000000006},"last":0.58099999999999996}]},{"osd":7,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":9.9730000000000008,"5min":9.9730000000000008,"15min":9.9730000000000008},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":3.0009999999999999,"5min":3.0009999999999999,"15min":3.0009999999999999},"last":0.69799999999999995}]}]},{"osd":0,"up_from":14,"seq":60129542166,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343884,"statfs":{"total":96636764160,"available":96608137216,"internally_reserved":0,"allocated":962560,"data_stored":732504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.62,"5min":0.62,"15min":0.62},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.495,"5min":0.495,"15min":0.495},"last":0.50900000000000001}]},{"osd":2,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"last":0.35299999999999998}]},{"osd":3,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"last":0.48999999999999999}]},{"osd":4,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.55500000000000005}]},{"osd":5,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.53900000000000003}]},{"osd":6,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.622},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.63400000000000001}]},{"osd":7,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.60199999999999998}]}]},{"osd":2,"up_from":14,"seq":60129542167,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":372736,"data_stored":142206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":1.429,"5min":1.429,"15min":1.429},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.38300000000000001}]},{"osd":1,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":1.5269999999999999,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.46400000000000002}]},{"osd":3,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":4.0629999999999997,"5min":4.0629999999999997,"15min":4.0629999999999997},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.381,"5min":1.381,"15min":1.381},"last":0.5}]},{"osd":4,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.54300000000000004}]},{"osd":5,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":3.8599999999999999,"5min":3.8599999999999999,"15min":3.8599999999999999},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.464,"5min":1.464,"15min":1.464},"last":0.63300000000000001}]},{"osd":6,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":3.9820000000000002,"5min":3.9820000000000002,"15min":3.9820000000000002},"last":0.53100000000000003}]},{"osd":7,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":3.9049999999999998,"5min":3.9049999999999998,"15min":3.9049999999999998},"last":0.68200000000000005}]}]},{"osd":3,"up_from":14,"seq":60129542167,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":376,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":385024,"data_stored":143770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"last":0.39000000000000001}]},{"osd":1,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"last":0.36399999999999999}]},{"osd":2,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.437,"5min":0.437,"15min":0.437},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.44800000000000001}]},{"osd":4,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":6.0350000000000001,"5min":6.0350000000000001,"15min":6.0350000000000001},"last":0.56399999999999995}]},{"osd":5,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":6.0540000000000003,"5min":6.0540000000000003,"15min":6.0540000000000003},"last":0.68799999999999994}]},{"osd":6,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.80200000000000005}]},{"osd":7,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":6.0819999999999999,"5min":6.0819999999999999,"15min":6.0819999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.745}]}]},{"osd":4,"up_from":14,"seq":60129542166,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":376832,"data_stored":142240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 10:36:56 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.628},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.59099999999999997}]},{"osd":1,"last update":"Sat Oct 14 10:36:56 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.61699999999999999}]},{"osd":2,"last update":"Sat Oct 14 10:36:56 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.68200000000000005}]},{"osd":3,"last update":"Sat Oct 14 10:36:56 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.70399999999999996}]},{"osd":5,"last update":"Sat Oct 14 10:36:56 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.442}]},{"osd":6,"last update":"Sat Oct 14 10:36:56 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.55900000000000005}]},{"osd":7,"last update":"Sat Oct 14 10:36:56 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.52800000000000002}]}]},{"osd":5,"up_from":14,"seq":60129542167,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":376832,"data_stored":142808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.60099999999999998}]},{"osd":1,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.78600000000000003}]},{"osd":2,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.58399999999999996}]},{"osd":3,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.629},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.76300000000000001}]},{"osd":4,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.49099999999999999}]},{"osd":6,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.46899999999999997}]},{"osd":7,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.45300000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-14T10:37:35.621 INFO:tasks.ceph:pgid 5.3f last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:35.621 INFO:tasks.ceph:pgid 5.3e last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:35.622 INFO:tasks.ceph:pgid 5.3d last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:35.622 INFO:tasks.ceph:pgid 5.3c last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:35.622 INFO:tasks.ceph:pgid 5.3a last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:35.623 INFO:tasks.ceph:pgid 5.37 last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:35.623 INFO:tasks.ceph:pgid 5.36 last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:35.624 INFO:tasks.ceph:pgid 5.34 last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:35.624 INFO:tasks.ceph:pgid 5.32 last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:35.624 INFO:tasks.ceph:pgid 5.31 last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:35.625 INFO:tasks.ceph:pgid 5.2e last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:35.625 INFO:tasks.ceph:pgid 5.2b last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:35.625 INFO:tasks.ceph:pgid 5.2a last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:35.626 INFO:tasks.ceph:pgid 5.27 last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:35.626 INFO:tasks.ceph:pgid 5.26 last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:35.627 INFO:tasks.ceph:pgid 5.25 last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:35.627 INFO:tasks.ceph:pgid 5.23 last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:35.627 INFO:tasks.ceph:pgid 5.21 last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:35.628 INFO:tasks.ceph:pgid 5.1c last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:35.628 INFO:tasks.ceph:pgid 5.1d last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:35.629 INFO:tasks.ceph:pgid 5.1a last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:35.629 INFO:tasks.ceph:pgid 5.1b last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:35.629 INFO:tasks.ceph:pgid 4.1a last_scrub_stamp 2023-10-14T10:36:58.035327+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=58, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:35.630 INFO:tasks.ceph:pgid 5.18 last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:35.630 INFO:tasks.ceph:pgid 5.19 last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:35.631 INFO:tasks.ceph:pgid 5.16 last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:35.631 INFO:tasks.ceph:pgid 4.17 last_scrub_stamp 2023-10-14T10:36:58.035327+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=58, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:35.631 INFO:tasks.ceph:pgid 5.17 last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:35.632 INFO:tasks.ceph:pgid 5.14 last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:35.632 INFO:tasks.ceph:pgid 5.15 last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:35.632 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2023-10-14T10:36:58.035327+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=58, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:35.633 INFO:tasks.ceph:pgid 5.12 last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:35.633 INFO:tasks.ceph:pgid 5.13 last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:35.634 INFO:tasks.ceph:pgid 5.10 last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:35.634 INFO:tasks.ceph:pgid 5.e last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:35.634 INFO:tasks.ceph:pgid 5.f last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:35.635 INFO:tasks.ceph:pgid 4.e last_scrub_stamp 2023-10-14T10:36:58.035327+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=58, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:35.635 INFO:tasks.ceph:pgid 5.c last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:35.635 INFO:tasks.ceph:pgid 4.d last_scrub_stamp 2023-10-14T10:36:58.035327+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=58, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:35.636 INFO:tasks.ceph:pgid 5.7 last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:35.636 INFO:tasks.ceph:pgid 4.4 last_scrub_stamp 2023-10-14T10:36:58.035327+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=58, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:35.637 INFO:tasks.ceph:pgid 5.0 last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:35.637 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2023-10-14T10:36:58.035327+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=58, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:35.637 INFO:tasks.ceph:pgid 5.1 last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:35.638 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-14T10:37:39.525 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-14T10:37:39.528 DEBUG:teuthology.orchestra.run.smithi072:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-14T10:37:55.640 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T10:37:55.780 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:55.778+0000 7f41efc61700 1 Processor -- start 2023-10-14T10:37:55.780 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:55.778+0000 7f41efc61700 1 -- start start 2023-10-14T10:37:55.783 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:55.779+0000 7f41efc61700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f41e80b2880 0x7f41e806c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:55.783 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:55.779+0000 7f41efc61700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f41e80b2c60 0x7f41e80cde60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:55.783 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:55.779+0000 7f41efc61700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41e80d0d00 con 0x7f41e80ce3a0 2023-10-14T10:37:55.784 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:55.779+0000 7f41efc61700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41e80ce810 con 0x7f41e80b2880 2023-10-14T10:37:55.784 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:55.779+0000 7f41efc61700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41e80d2450 con 0x7f41e80b2c60 2023-10-14T10:37:55.784 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:55.779+0000 7f41ed9fd700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f41e80b2880 0x7f41e806c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:55.785 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:55.779+0000 7f41ed9fd700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f41e80b2880 0x7f41e806c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46308/0 (socket says 172.21.15.67:46308) 2023-10-14T10:37:55.785 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:55.779+0000 7f41ed9fd700 1 -- 172.21.15.67:0/2801375010 learned_addr learned my addr 172.21.15.67:0/2801375010 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:37:55.785 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:55.779+0000 7f41ed1fc700 1 --2- 172.21.15.67:0/2801375010 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f41e80b2c60 0x7f41e80cde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:55.785 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:55.779+0000 7f41ed9fd700 1 -- 172.21.15.67:0/2801375010 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f41e80b2c60 msgr2=0x7f41e80cde60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:37:55.786 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:55.779+0000 7f41ed9fd700 1 --2- 172.21.15.67:0/2801375010 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f41e80b2c60 0x7f41e80cde60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:55.786 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:55.779+0000 7f41ed9fd700 1 -- 172.21.15.67:0/2801375010 >> v1:172.21.15.72:6789/0 conn(0x7f41e80ce3a0 legacy=0x7f41e81617a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:55.786 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:55.779+0000 7f41ed9fd700 1 -- 172.21.15.67:0/2801375010 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41e80cfc60 con 0x7f41e80b2880 2023-10-14T10:37:55.787 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:55.780+0000 7f41ed9fd700 1 --2- 172.21.15.67:0/2801375010 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41e80b2880 0x7f41e806c660 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f41e4006fd0 tx=0x7f41e4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5a80cd7e128b3e7 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:55.787 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:55.780+0000 7f41ec9fb700 1 -- 172.21.15.67:0/2801375010 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e4005e30 con 0x7f41e80b2880 2023-10-14T10:37:55.787 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:55.780+0000 7f41ec9fb700 1 -- 172.21.15.67:0/2801375010 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41e4004030 con 0x7f41e80b2880 2023-10-14T10:37:55.787 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:55.780+0000 7f41ec9fb700 1 -- 172.21.15.67:0/2801375010 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e40043e0 con 0x7f41e80b2880 2023-10-14T10:37:55.788 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:55.780+0000 7f41efc61700 1 -- 172.21.15.67:0/2801375010 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41e80b2880 msgr2=0x7f41e806c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:55.788 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:55.780+0000 7f41efc61700 1 --2- 172.21.15.67:0/2801375010 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41e80b2880 0x7f41e806c660 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f41e4006fd0 tx=0x7f41e4009bd0 comp rx=0 tx=0).stop 2023-10-14T10:37:55.788 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:55.780+0000 7f41efc61700 1 -- 172.21.15.67:0/2801375010 shutdown_connections 2023-10-14T10:37:55.789 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:55.780+0000 7f41efc61700 1 --2- 172.21.15.67:0/2801375010 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41e80b2880 0x7f41e806c660 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:55.789 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:55.780+0000 7f41efc61700 1 --2- 172.21.15.67:0/2801375010 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f41e80b2c60 0x7f41e80cde60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:55.789 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:55.780+0000 7f41efc61700 1 -- 172.21.15.67:0/2801375010 >> 172.21.15.67:0/2801375010 conn(0x7f41e80d15e0 msgr2=0x7f41e80d19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:37:55.789 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:55.781+0000 7f41efc61700 1 -- 172.21.15.67:0/2801375010 shutdown_connections 2023-10-14T10:37:55.790 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:55.781+0000 7f41efc61700 1 -- 172.21.15.67:0/2801375010 wait complete. 2023-10-14T10:37:55.793 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:55.782+0000 7f41efc61700 1 Processor -- start 2023-10-14T10:37:55.793 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:55.782+0000 7f41efc61700 1 -- start start 2023-10-14T10:37:55.793 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:55.783+0000 7f41efc61700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f41e80b2880 0x7f41e815cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:55.794 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:55.783+0000 7f41efc61700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41e80b2c60 0x7f41e81f3e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:55.794 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:55.783+0000 7f41efc61700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f41e80ce3a0 0x7f41e81f6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:55.794 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:55.783+0000 7f41efc61700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41e815d4e0 con 0x7f41e80b2c60 2023-10-14T10:37:55.794 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:55.783+0000 7f41efc61700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41e815d7c0 con 0x7f41e80ce3a0 2023-10-14T10:37:55.795 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:55.783+0000 7f41efc61700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41e815daa0 con 0x7f41e80b2880 2023-10-14T10:37:55.795 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:55.783+0000 7f41ed1fc700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41e80b2c60 0x7f41e81f3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:55.795 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:55.783+0000 7f41ed1fc700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41e80b2c60 0x7f41e81f3e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46316/0 (socket says 172.21.15.67:46316) 2023-10-14T10:37:55.796 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:55.783+0000 7f41ed1fc700 1 -- 172.21.15.67:0/3620839113 learned_addr learned my addr 172.21.15.67:0/3620839113 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:37:55.796 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:55.783+0000 7f41ed9fd700 1 --2- 172.21.15.67:0/3620839113 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f41e80b2880 0x7f41e815cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:55.796 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:55.783+0000 7f41ee1fe700 1 --2- 172.21.15.67:0/3620839113 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f41e80ce3a0 0x7f41e81f6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:55.796 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:55.783+0000 7f41ed1fc700 1 -- 172.21.15.67:0/3620839113 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f41e80b2880 msgr2=0x7f41e815cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:55.797 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:55.783+0000 7f41ed1fc700 1 --2- 172.21.15.67:0/3620839113 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f41e80b2880 0x7f41e815cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:55.797 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:55.783+0000 7f41ed1fc700 1 -- 172.21.15.67:0/3620839113 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f41e80ce3a0 msgr2=0x7f41e81f6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:55.797 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:55.783+0000 7f41ed1fc700 1 --2- 172.21.15.67:0/3620839113 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f41e80ce3a0 0x7f41e81f6bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:55.797 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:55.783+0000 7f41ed1fc700 1 -- 172.21.15.67:0/3620839113 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41e4009500 con 0x7f41e80b2c60 2023-10-14T10:37:55.798 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:55.783+0000 7f41ed1fc700 1 --2- 172.21.15.67:0/3620839113 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41e80b2c60 0x7f41e81f3e50 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f41d800e810 tx=0x7f41d800eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:55.798 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:55.783+0000 7f41de7fc700 1 -- 172.21.15.67:0/3620839113 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d8004230 con 0x7f41e80b2c60 2023-10-14T10:37:55.798 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:55.783+0000 7f41de7fc700 1 -- 172.21.15.67:0/3620839113 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41d8004390 con 0x7f41e80b2c60 2023-10-14T10:37:55.798 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:55.783+0000 7f41efc61700 1 -- 172.21.15.67:0/3620839113 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41e815dd80 con 0x7f41e80b2c60 2023-10-14T10:37:55.799 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:55.783+0000 7f41de7fc700 1 -- 172.21.15.67:0/3620839113 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d80036a0 con 0x7f41e80b2c60 2023-10-14T10:37:55.799 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:55.784+0000 7f41efc61700 1 -- 172.21.15.67:0/3620839113 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41e815d090 con 0x7f41e80b2c60 2023-10-14T10:37:55.799 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:55.784+0000 7f41efc61700 1 -- 172.21.15.67:0/3620839113 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41e815e2a0 con 0x7f41e80b2c60 2023-10-14T10:37:55.800 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:55.785+0000 7f41de7fc700 1 -- 172.21.15.67:0/3620839113 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f41d8003800 con 0x7f41e80b2c60 2023-10-14T10:37:55.800 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:55.785+0000 7f41de7fc700 1 --2- 172.21.15.67:0/3620839113 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f41cc06af00 0x7f41cc06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:37:55.800 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:55.785+0000 7f41de7fc700 1 -- 172.21.15.67:0/3620839113 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5729+0+0 (secure 0 0 0) 0x7f41d8089040 con 0x7f41e80b2c60 2023-10-14T10:37:55.800 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:55.786+0000 7f41ed9fd700 1 --2- 172.21.15.67:0/3620839113 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f41cc06af00 0x7f41cc06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:37:55.801 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:55.786+0000 7f41ed9fd700 1 --2- 172.21.15.67:0/3620839113 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f41cc06af00 0x7f41cc06d3c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f41e4000c00 tx=0x7f41e4008070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:37:55.801 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:55.790+0000 7f41de7fc700 1 -- 172.21.15.67:0/3620839113 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f41e815e2a0 con 0x7f41e80b2c60 2023-10-14T10:37:55.993 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:55.991+0000 7f41efc61700 1 -- 172.21.15.67:0/3620839113 --> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f41e815d930 con 0x7f41cc06af00 2023-10-14T10:37:56.002 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:56.000+0000 7f41de7fc700 1 -- 172.21.15.67:0/3620839113 <== mgr.4100 v2:172.21.15.72:6832/703488382 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+290358 (secure 0 0 0) 0x7f41e815d930 con 0x7f41cc06af00 2023-10-14T10:37:56.007 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-14T10:37:56.009 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:56.004+0000 7f41efc61700 1 -- 172.21.15.67:0/3620839113 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f41cc06af00 msgr2=0x7f41cc06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:56.009 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:56.004+0000 7f41efc61700 1 --2- 172.21.15.67:0/3620839113 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f41cc06af00 0x7f41cc06d3c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f41e4000c00 tx=0x7f41e4008070 comp rx=0 tx=0).stop 2023-10-14T10:37:56.010 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:56.004+0000 7f41efc61700 1 -- 172.21.15.67:0/3620839113 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41e80b2c60 msgr2=0x7f41e81f3e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:37:56.010 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:56.004+0000 7f41efc61700 1 --2- 172.21.15.67:0/3620839113 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41e80b2c60 0x7f41e81f3e50 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f41d800e810 tx=0x7f41d800eb20 comp rx=0 tx=0).stop 2023-10-14T10:37:56.010 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:56.004+0000 7f41efc61700 1 -- 172.21.15.67:0/3620839113 shutdown_connections 2023-10-14T10:37:56.011 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:56.004+0000 7f41efc61700 1 --2- 172.21.15.67:0/3620839113 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f41cc06af00 0x7f41cc06d3c0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:56.011 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:56.004+0000 7f41efc61700 1 --2- 172.21.15.67:0/3620839113 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f41e80ce3a0 0x7f41e81f6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:56.011 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:56.004+0000 7f41efc61700 1 --2- 172.21.15.67:0/3620839113 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f41e80b2880 0x7f41e815cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:56.012 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:56.004+0000 7f41efc61700 1 --2- 172.21.15.67:0/3620839113 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41e80b2c60 0x7f41e81f3e50 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:37:56.012 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:56.004+0000 7f41efc61700 1 -- 172.21.15.67:0/3620839113 >> 172.21.15.67:0/3620839113 conn(0x7f41e80d15e0 msgr2=0x7f41e806b950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:37:56.012 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:56.004+0000 7f41efc61700 1 -- 172.21.15.67:0/3620839113 shutdown_connections 2023-10-14T10:37:56.013 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:37:56.004+0000 7f41efc61700 1 -- 172.21.15.67:0/3620839113 wait complete. 2023-10-14T10:37:56.013 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2023-10-14T10:37:56.018 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":86,"stamp":"2023-10-14T10:37:55.783067+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":166,"ondisk_log_size":166,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220232,"kb_used_data":4104,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754488,"statfs":{"total":773094113280,"available":772868595712,"internally_reserved":0,"allocated":4202496,"data_stored":2322644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12742,"internal_metadata":221236794},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003139"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:36.161423+0000","last_change":"2023-10-14T10:37:36.161469+0000","last_active":"2023-10-14T10:37:36.161423+0000","last_peered":"2023-10-14T10:37:36.161423+0000","last_clean":"2023-10-14T10:37:36.161423+0000","last_became_active":"2023-10-14T10:37:00.074851+0000","last_became_peered":"2023-10-14T10:37:00.074851+0000","last_unstale":"2023-10-14T10:37:36.161423+0000","last_undegraded":"2023-10-14T10:37:36.161423+0000","last_fullsized":"2023-10-14T10:37:36.161423+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:36.161309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:36.161309+0000","last_clean_scrub_stamp":"2023-10-14T10:37:36.161309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T12:29:25.518201+0000","scrub_duration":0.0015859890000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.822679+0000","last_change":"2023-10-14T10:37:00.068871+0000","last_active":"2023-10-14T10:37:01.822679+0000","last_peered":"2023-10-14T10:37:01.822679+0000","last_clean":"2023-10-14T10:37:01.822679+0000","last_became_active":"2023-10-14T10:37:00.068586+0000","last_became_peered":"2023-10-14T10:37:00.068586+0000","last_unstale":"2023-10-14T10:37:01.822679+0000","last_undegraded":"2023-10-14T10:37:01.822679+0000","last_fullsized":"2023-10-14T10:37:01.822679+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:47.896163+0000","last_change":"2023-10-14T10:37:47.896230+0000","last_active":"2023-10-14T10:37:47.896163+0000","last_peered":"2023-10-14T10:37:47.896163+0000","last_clean":"2023-10-14T10:37:47.896163+0000","last_became_active":"2023-10-14T10:37:00.080476+0000","last_became_peered":"2023-10-14T10:37:00.080476+0000","last_unstale":"2023-10-14T10:37:47.896163+0000","last_undegraded":"2023-10-14T10:37:47.896163+0000","last_fullsized":"2023-10-14T10:37:47.896163+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:47.896025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:47.896025+0000","last_clean_scrub_stamp":"2023-10-14T10:37:47.896025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T14:08:35.991519+0000","scrub_duration":0.0020600129999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:36.043044+0000","last_change":"2023-10-14T10:37:36.043116+0000","last_active":"2023-10-14T10:37:36.043044+0000","last_peered":"2023-10-14T10:37:36.043044+0000","last_clean":"2023-10-14T10:37:36.043044+0000","last_became_active":"2023-10-14T10:37:00.077322+0000","last_became_peered":"2023-10-14T10:37:00.077322+0000","last_unstale":"2023-10-14T10:37:36.043044+0000","last_undegraded":"2023-10-14T10:37:36.043044+0000","last_fullsized":"2023-10-14T10:37:36.043044+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:36.042949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:36.042949+0000","last_clean_scrub_stamp":"2023-10-14T10:37:36.042949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T18:48:48.217020+0000","scrub_duration":0.001538579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:25.474346+0000","last_change":"2023-10-14T10:37:25.474436+0000","last_active":"2023-10-14T10:37:25.474346+0000","last_peered":"2023-10-14T10:37:25.474346+0000","last_clean":"2023-10-14T10:37:25.474346+0000","last_became_active":"2023-10-14T10:37:00.068055+0000","last_became_peered":"2023-10-14T10:37:00.068055+0000","last_unstale":"2023-10-14T10:37:25.474346+0000","last_undegraded":"2023-10-14T10:37:25.474346+0000","last_fullsized":"2023-10-14T10:37:25.474346+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:25.474208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:25.474208+0000","last_clean_scrub_stamp":"2023-10-14T10:37:25.474208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:57:27.388802+0000","scrub_duration":0.001660716,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:47.854873+0000","last_change":"2023-10-14T10:37:47.854929+0000","last_active":"2023-10-14T10:37:47.854873+0000","last_peered":"2023-10-14T10:37:47.854873+0000","last_clean":"2023-10-14T10:37:47.854873+0000","last_became_active":"2023-10-14T10:37:00.072345+0000","last_became_peered":"2023-10-14T10:37:00.072345+0000","last_unstale":"2023-10-14T10:37:47.854873+0000","last_undegraded":"2023-10-14T10:37:47.854873+0000","last_fullsized":"2023-10-14T10:37:47.854873+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:47.854750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:47.854750+0000","last_clean_scrub_stamp":"2023-10-14T10:37:47.854750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:08:22.220361+0000","scrub_duration":0.0020211510000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:16.975562+0000","last_change":"2023-10-14T10:37:16.975628+0000","last_active":"2023-10-14T10:37:16.975562+0000","last_peered":"2023-10-14T10:37:16.975562+0000","last_clean":"2023-10-14T10:37:16.975562+0000","last_became_active":"2023-10-14T10:37:00.072368+0000","last_became_peered":"2023-10-14T10:37:00.072368+0000","last_unstale":"2023-10-14T10:37:16.975562+0000","last_undegraded":"2023-10-14T10:37:16.975562+0000","last_fullsized":"2023-10-14T10:37:16.975562+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:16.975436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:16.975436+0000","last_clean_scrub_stamp":"2023-10-14T10:37:16.975436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:31:42.878362+0000","scrub_duration":0.0016077960000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:29.970861+0000","last_change":"2023-10-14T10:37:29.970909+0000","last_active":"2023-10-14T10:37:29.970861+0000","last_peered":"2023-10-14T10:37:29.970861+0000","last_clean":"2023-10-14T10:37:29.970861+0000","last_became_active":"2023-10-14T10:37:00.073792+0000","last_became_peered":"2023-10-14T10:37:00.073792+0000","last_unstale":"2023-10-14T10:37:29.970861+0000","last_undegraded":"2023-10-14T10:37:29.970861+0000","last_fullsized":"2023-10-14T10:37:29.970861+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:29.970754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:29.970754+0000","last_clean_scrub_stamp":"2023-10-14T10:37:29.970754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T14:08:18.066248+0000","scrub_duration":0.001847558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.824332+0000","last_change":"2023-10-14T10:37:00.081745+0000","last_active":"2023-10-14T10:37:01.824332+0000","last_peered":"2023-10-14T10:37:01.824332+0000","last_clean":"2023-10-14T10:37:01.824332+0000","last_became_active":"2023-10-14T10:37:00.081451+0000","last_became_peered":"2023-10-14T10:37:00.081451+0000","last_unstale":"2023-10-14T10:37:01.824332+0000","last_undegraded":"2023-10-14T10:37:01.824332+0000","last_fullsized":"2023-10-14T10:37:01.824332+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:49.039728+0000","last_change":"2023-10-14T10:37:49.039775+0000","last_active":"2023-10-14T10:37:49.039728+0000","last_peered":"2023-10-14T10:37:49.039728+0000","last_clean":"2023-10-14T10:37:49.039728+0000","last_became_active":"2023-10-14T10:37:00.070667+0000","last_became_peered":"2023-10-14T10:37:00.070667+0000","last_unstale":"2023-10-14T10:37:49.039728+0000","last_undegraded":"2023-10-14T10:37:49.039728+0000","last_fullsized":"2023-10-14T10:37:49.039728+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:49.039605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:49.039605+0000","last_clean_scrub_stamp":"2023-10-14T10:37:49.039605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:57:52.753077+0000","scrub_duration":0.001714559,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:27.985170+0000","last_change":"2023-10-14T10:37:27.985252+0000","last_active":"2023-10-14T10:37:27.985170+0000","last_peered":"2023-10-14T10:37:27.985170+0000","last_clean":"2023-10-14T10:37:27.985170+0000","last_became_active":"2023-10-14T10:37:00.072639+0000","last_became_peered":"2023-10-14T10:37:00.072639+0000","last_unstale":"2023-10-14T10:37:27.985170+0000","last_undegraded":"2023-10-14T10:37:27.985170+0000","last_fullsized":"2023-10-14T10:37:27.985170+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:27.985038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:27.985038+0000","last_clean_scrub_stamp":"2023-10-14T10:37:27.985038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:26:54.065382+0000","scrub_duration":0.001768588,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:42.762513+0000","last_change":"2023-10-14T10:37:42.762560+0000","last_active":"2023-10-14T10:37:42.762513+0000","last_peered":"2023-10-14T10:37:42.762513+0000","last_clean":"2023-10-14T10:37:42.762513+0000","last_became_active":"2023-10-14T10:37:00.069373+0000","last_became_peered":"2023-10-14T10:37:00.069373+0000","last_unstale":"2023-10-14T10:37:42.762513+0000","last_undegraded":"2023-10-14T10:37:42.762513+0000","last_fullsized":"2023-10-14T10:37:42.762513+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:42.762373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:42.762373+0000","last_clean_scrub_stamp":"2023-10-14T10:37:42.762373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:27:08.842717+0000","scrub_duration":0.002121197,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:19.586632+0000","last_change":"2023-10-14T10:37:19.586687+0000","last_active":"2023-10-14T10:37:19.586632+0000","last_peered":"2023-10-14T10:37:19.586632+0000","last_clean":"2023-10-14T10:37:19.586632+0000","last_became_active":"2023-10-14T10:37:00.074473+0000","last_became_peered":"2023-10-14T10:37:00.074473+0000","last_unstale":"2023-10-14T10:37:19.586632+0000","last_undegraded":"2023-10-14T10:37:19.586632+0000","last_fullsized":"2023-10-14T10:37:19.586632+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:19.586500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:19.586500+0000","last_clean_scrub_stamp":"2023-10-14T10:37:19.586500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:57:11.433994+0000","scrub_duration":0.001596793,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:42.660367+0000","last_change":"2023-10-14T10:37:42.660417+0000","last_active":"2023-10-14T10:37:42.660367+0000","last_peered":"2023-10-14T10:37:42.660367+0000","last_clean":"2023-10-14T10:37:42.660367+0000","last_became_active":"2023-10-14T10:37:00.073651+0000","last_became_peered":"2023-10-14T10:37:00.073651+0000","last_unstale":"2023-10-14T10:37:42.660367+0000","last_undegraded":"2023-10-14T10:37:42.660367+0000","last_fullsized":"2023-10-14T10:37:42.660367+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:42.660244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:42.660244+0000","last_clean_scrub_stamp":"2023-10-14T10:37:42.660244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T21:22:40.618175+0000","scrub_duration":0.0016045129999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.824169+0000","last_change":"2023-10-14T10:37:00.072277+0000","last_active":"2023-10-14T10:37:01.824169+0000","last_peered":"2023-10-14T10:37:01.824169+0000","last_clean":"2023-10-14T10:37:01.824169+0000","last_became_active":"2023-10-14T10:37:00.071999+0000","last_became_peered":"2023-10-14T10:37:00.071999+0000","last_unstale":"2023-10-14T10:37:01.824169+0000","last_undegraded":"2023-10-14T10:37:01.824169+0000","last_fullsized":"2023-10-14T10:37:01.824169+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:22.094919+0000","last_change":"2023-10-14T10:37:22.094971+0000","last_active":"2023-10-14T10:37:22.094919+0000","last_peered":"2023-10-14T10:37:22.094919+0000","last_clean":"2023-10-14T10:37:22.094919+0000","last_became_active":"2023-10-14T10:37:00.071509+0000","last_became_peered":"2023-10-14T10:37:00.071509+0000","last_unstale":"2023-10-14T10:37:22.094919+0000","last_undegraded":"2023-10-14T10:37:22.094919+0000","last_fullsized":"2023-10-14T10:37:22.094919+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:22.094817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:22.094817+0000","last_clean_scrub_stamp":"2023-10-14T10:37:22.094817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:57:03.789898+0000","scrub_duration":0.0017607390000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:21.067339+0000","last_change":"2023-10-14T10:37:21.067417+0000","last_active":"2023-10-14T10:37:21.067339+0000","last_peered":"2023-10-14T10:37:21.067339+0000","last_clean":"2023-10-14T10:37:21.067339+0000","last_became_active":"2023-10-14T10:37:00.071946+0000","last_became_peered":"2023-10-14T10:37:00.071946+0000","last_unstale":"2023-10-14T10:37:21.067339+0000","last_undegraded":"2023-10-14T10:37:21.067339+0000","last_fullsized":"2023-10-14T10:37:21.067339+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:21.067219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:21.067219+0000","last_clean_scrub_stamp":"2023-10-14T10:37:21.067219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T12:48:36.500809+0000","scrub_duration":0.0016934249999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.822669+0000","last_change":"2023-10-14T10:37:00.080581+0000","last_active":"2023-10-14T10:37:01.822669+0000","last_peered":"2023-10-14T10:37:01.822669+0000","last_clean":"2023-10-14T10:37:01.822669+0000","last_became_active":"2023-10-14T10:37:00.080288+0000","last_became_peered":"2023-10-14T10:37:00.080288+0000","last_unstale":"2023-10-14T10:37:01.822669+0000","last_undegraded":"2023-10-14T10:37:01.822669+0000","last_fullsized":"2023-10-14T10:37:01.822669+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:13.890326+0000","last_change":"2023-10-14T10:37:13.890365+0000","last_active":"2023-10-14T10:37:13.890326+0000","last_peered":"2023-10-14T10:37:13.890326+0000","last_clean":"2023-10-14T10:37:13.890326+0000","last_became_active":"2023-10-14T10:37:00.080186+0000","last_became_peered":"2023-10-14T10:37:00.080186+0000","last_unstale":"2023-10-14T10:37:13.890326+0000","last_undegraded":"2023-10-14T10:37:13.890326+0000","last_fullsized":"2023-10-14T10:37:13.890326+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:13.890234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:13.890234+0000","last_clean_scrub_stamp":"2023-10-14T10:37:13.890234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:48:50.907546+0000","scrub_duration":0.001362254,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:26.440259+0000","last_change":"2023-10-14T10:37:26.440336+0000","last_active":"2023-10-14T10:37:26.440259+0000","last_peered":"2023-10-14T10:37:26.440259+0000","last_clean":"2023-10-14T10:37:26.440259+0000","last_became_active":"2023-10-14T10:37:00.071796+0000","last_became_peered":"2023-10-14T10:37:00.071796+0000","last_unstale":"2023-10-14T10:37:26.440259+0000","last_undegraded":"2023-10-14T10:37:26.440259+0000","last_fullsized":"2023-10-14T10:37:26.440259+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:26.440115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:26.440115+0000","last_clean_scrub_stamp":"2023-10-14T10:37:26.440115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:54:37.394808+0000","scrub_duration":0.0021402970000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.824156+0000","last_change":"2023-10-14T10:37:00.080796+0000","last_active":"2023-10-14T10:37:01.824156+0000","last_peered":"2023-10-14T10:37:01.824156+0000","last_clean":"2023-10-14T10:37:01.824156+0000","last_became_active":"2023-10-14T10:37:00.080526+0000","last_became_peered":"2023-10-14T10:37:00.080526+0000","last_unstale":"2023-10-14T10:37:01.824156+0000","last_undegraded":"2023-10-14T10:37:01.824156+0000","last_fullsized":"2023-10-14T10:37:01.824156+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.824113+0000","last_change":"2023-10-14T10:37:00.080350+0000","last_active":"2023-10-14T10:37:01.824113+0000","last_peered":"2023-10-14T10:37:01.824113+0000","last_clean":"2023-10-14T10:37:01.824113+0000","last_became_active":"2023-10-14T10:37:00.080062+0000","last_became_peered":"2023-10-14T10:37:00.080062+0000","last_unstale":"2023-10-14T10:37:01.824113+0000","last_undegraded":"2023-10-14T10:37:01.824113+0000","last_fullsized":"2023-10-14T10:37:01.824113+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:24.101006+0000","last_change":"2023-10-14T10:37:24.101056+0000","last_active":"2023-10-14T10:37:24.101006+0000","last_peered":"2023-10-14T10:37:24.101006+0000","last_clean":"2023-10-14T10:37:24.101006+0000","last_became_active":"2023-10-14T10:37:00.070897+0000","last_became_peered":"2023-10-14T10:37:00.070897+0000","last_unstale":"2023-10-14T10:37:24.101006+0000","last_undegraded":"2023-10-14T10:37:24.101006+0000","last_fullsized":"2023-10-14T10:37:24.101006+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:24.100899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:24.100899+0000","last_clean_scrub_stamp":"2023-10-14T10:37:24.100899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:58:06.157813+0000","scrub_duration":0.001908496,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:26.820065+0000","last_change":"2023-10-14T10:37:26.820141+0000","last_active":"2023-10-14T10:37:26.820065+0000","last_peered":"2023-10-14T10:37:26.820065+0000","last_clean":"2023-10-14T10:37:26.820065+0000","last_became_active":"2023-10-14T10:37:00.069246+0000","last_became_peered":"2023-10-14T10:37:00.069246+0000","last_unstale":"2023-10-14T10:37:26.820065+0000","last_undegraded":"2023-10-14T10:37:26.820065+0000","last_fullsized":"2023-10-14T10:37:26.820065+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:26.819916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:26.819916+0000","last_clean_scrub_stamp":"2023-10-14T10:37:26.819916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T21:40:50.616911+0000","scrub_duration":0.0021912780000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:47.037229+0000","last_change":"2023-10-14T10:37:47.037305+0000","last_active":"2023-10-14T10:37:47.037229+0000","last_peered":"2023-10-14T10:37:47.037229+0000","last_clean":"2023-10-14T10:37:47.037229+0000","last_became_active":"2023-10-14T10:37:00.069871+0000","last_became_peered":"2023-10-14T10:37:00.069871+0000","last_unstale":"2023-10-14T10:37:47.037229+0000","last_undegraded":"2023-10-14T10:37:47.037229+0000","last_fullsized":"2023-10-14T10:37:47.037229+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:47.037074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:47.037074+0000","last_clean_scrub_stamp":"2023-10-14T10:37:47.037074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T20:01:24.085952+0000","scrub_duration":0.001819242,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.822659+0000","last_change":"2023-10-14T10:37:00.076474+0000","last_active":"2023-10-14T10:37:01.822659+0000","last_peered":"2023-10-14T10:37:01.822659+0000","last_clean":"2023-10-14T10:37:01.822659+0000","last_became_active":"2023-10-14T10:37:00.076169+0000","last_became_peered":"2023-10-14T10:37:00.076169+0000","last_unstale":"2023-10-14T10:37:01.822659+0000","last_undegraded":"2023-10-14T10:37:01.822659+0000","last_fullsized":"2023-10-14T10:37:01.822659+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:42.041674+0000","last_change":"2023-10-14T10:37:42.041756+0000","last_active":"2023-10-14T10:37:42.041674+0000","last_peered":"2023-10-14T10:37:42.041674+0000","last_clean":"2023-10-14T10:37:42.041674+0000","last_became_active":"2023-10-14T10:37:00.076263+0000","last_became_peered":"2023-10-14T10:37:00.076263+0000","last_unstale":"2023-10-14T10:37:42.041674+0000","last_undegraded":"2023-10-14T10:37:42.041674+0000","last_fullsized":"2023-10-14T10:37:42.041674+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:42.041529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:42.041529+0000","last_clean_scrub_stamp":"2023-10-14T10:37:42.041529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T14:08:30.137022+0000","scrub_duration":0.0018411619999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:30.765830+0000","last_change":"2023-10-14T10:37:30.765882+0000","last_active":"2023-10-14T10:37:30.765830+0000","last_peered":"2023-10-14T10:37:30.765830+0000","last_clean":"2023-10-14T10:37:30.765830+0000","last_became_active":"2023-10-14T10:37:00.079178+0000","last_became_peered":"2023-10-14T10:37:00.079178+0000","last_unstale":"2023-10-14T10:37:30.765830+0000","last_undegraded":"2023-10-14T10:37:30.765830+0000","last_fullsized":"2023-10-14T10:37:30.765830+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:30.765720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:30.765720+0000","last_clean_scrub_stamp":"2023-10-14T10:37:30.765720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T12:48:46.199309+0000","scrub_duration":0.0022229910000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:42.718659+0000","last_change":"2023-10-14T10:37:42.718740+0000","last_active":"2023-10-14T10:37:42.718659+0000","last_peered":"2023-10-14T10:37:42.718659+0000","last_clean":"2023-10-14T10:37:42.718659+0000","last_became_active":"2023-10-14T10:37:00.080171+0000","last_became_peered":"2023-10-14T10:37:00.080171+0000","last_unstale":"2023-10-14T10:37:42.718659+0000","last_undegraded":"2023-10-14T10:37:42.718659+0000","last_fullsized":"2023-10-14T10:37:42.718659+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:42.718545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:42.718545+0000","last_clean_scrub_stamp":"2023-10-14T10:37:42.718545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T18:34:28.005587+0000","scrub_duration":0.001795675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:30.956480+0000","last_change":"2023-10-14T10:37:30.956555+0000","last_active":"2023-10-14T10:37:30.956480+0000","last_peered":"2023-10-14T10:37:30.956480+0000","last_clean":"2023-10-14T10:37:30.956480+0000","last_became_active":"2023-10-14T10:37:00.071882+0000","last_became_peered":"2023-10-14T10:37:00.071882+0000","last_unstale":"2023-10-14T10:37:30.956480+0000","last_undegraded":"2023-10-14T10:37:30.956480+0000","last_fullsized":"2023-10-14T10:37:30.956480+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:30.956369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:30.956369+0000","last_clean_scrub_stamp":"2023-10-14T10:37:30.956369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:18:44.736053+0000","scrub_duration":0.001666013,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:35.120416+0000","last_change":"2023-10-14T10:37:35.120460+0000","last_active":"2023-10-14T10:37:35.120416+0000","last_peered":"2023-10-14T10:37:35.120416+0000","last_clean":"2023-10-14T10:37:35.120416+0000","last_became_active":"2023-10-14T10:37:00.081312+0000","last_became_peered":"2023-10-14T10:37:00.081312+0000","last_unstale":"2023-10-14T10:37:35.120416+0000","last_undegraded":"2023-10-14T10:37:35.120416+0000","last_fullsized":"2023-10-14T10:37:35.120416+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:35.120311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:35.120311+0000","last_clean_scrub_stamp":"2023-10-14T10:37:35.120311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:58:17.177225+0000","scrub_duration":0.001906561,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:15.957200+0000","last_change":"2023-10-14T10:37:15.957269+0000","last_active":"2023-10-14T10:37:15.957200+0000","last_peered":"2023-10-14T10:37:15.957200+0000","last_clean":"2023-10-14T10:37:15.957200+0000","last_became_active":"2023-10-14T10:37:00.078441+0000","last_became_peered":"2023-10-14T10:37:00.078441+0000","last_unstale":"2023-10-14T10:37:15.957200+0000","last_undegraded":"2023-10-14T10:37:15.957200+0000","last_fullsized":"2023-10-14T10:37:15.957200+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:15.957063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:15.957063+0000","last_clean_scrub_stamp":"2023-10-14T10:37:15.957063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:48:52.974375+0000","scrub_duration":0.001645093,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:25.103789+0000","last_change":"2023-10-14T10:37:25.103841+0000","last_active":"2023-10-14T10:37:25.103789+0000","last_peered":"2023-10-14T10:37:25.103789+0000","last_clean":"2023-10-14T10:37:25.103789+0000","last_became_active":"2023-10-14T10:37:00.070945+0000","last_became_peered":"2023-10-14T10:37:00.070945+0000","last_unstale":"2023-10-14T10:37:25.103789+0000","last_undegraded":"2023-10-14T10:37:25.103789+0000","last_fullsized":"2023-10-14T10:37:25.103789+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:25.103682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:25.103682+0000","last_clean_scrub_stamp":"2023-10-14T10:37:25.103682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:53:19.037516+0000","scrub_duration":0.0021103710000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":25,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:21.985058+0000","last_change":"2023-10-14T10:37:21.985135+0000","last_active":"2023-10-14T10:37:21.985058+0000","last_peered":"2023-10-14T10:37:21.985058+0000","last_clean":"2023-10-14T10:37:21.985058+0000","last_became_active":"2023-10-14T10:36:59.066488+0000","last_became_peered":"2023-10-14T10:36:59.066488+0000","last_unstale":"2023-10-14T10:37:21.985058+0000","last_undegraded":"2023-10-14T10:37:21.985058+0000","last_fullsized":"2023-10-14T10:37:21.985058+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'4","last_scrub_stamp":"2023-10-14T10:37:21.984953+0000","last_deep_scrub":"32'4","last_deep_scrub_stamp":"2023-10-14T10:37:21.984953+0000","last_clean_scrub_stamp":"2023-10-14T10:37:21.984953+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:31:39.071011+0000","scrub_duration":0.0021399549999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:31.160865+0000","last_change":"2023-10-14T10:37:31.160912+0000","last_active":"2023-10-14T10:37:31.160865+0000","last_peered":"2023-10-14T10:37:31.160865+0000","last_clean":"2023-10-14T10:37:31.160865+0000","last_became_active":"2023-10-14T10:37:00.077734+0000","last_became_peered":"2023-10-14T10:37:00.077734+0000","last_unstale":"2023-10-14T10:37:31.160865+0000","last_undegraded":"2023-10-14T10:37:31.160865+0000","last_fullsized":"2023-10-14T10:37:31.160865+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:31.160755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:31.160755+0000","last_clean_scrub_stamp":"2023-10-14T10:37:31.160755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T14:19:54.429928+0000","scrub_duration":0.0014218779999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:22.082855+0000","last_change":"2023-10-14T10:37:22.082911+0000","last_active":"2023-10-14T10:37:22.082855+0000","last_peered":"2023-10-14T10:37:22.082855+0000","last_clean":"2023-10-14T10:37:22.082855+0000","last_became_active":"2023-10-14T10:36:59.070463+0000","last_became_peered":"2023-10-14T10:36:59.070463+0000","last_unstale":"2023-10-14T10:37:22.082855+0000","last_undegraded":"2023-10-14T10:37:22.082855+0000","last_fullsized":"2023-10-14T10:37:22.082855+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1","last_scrub_stamp":"2023-10-14T10:37:22.082747+0000","last_deep_scrub":"32'1","last_deep_scrub_stamp":"2023-10-14T10:37:22.082747+0000","last_clean_scrub_stamp":"2023-10-14T10:37:22.082747+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:06:51.432100+0000","scrub_duration":0.0021442739999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:50.635047+0000","last_change":"2023-10-14T10:37:50.635125+0000","last_active":"2023-10-14T10:37:50.635047+0000","last_peered":"2023-10-14T10:37:50.635047+0000","last_clean":"2023-10-14T10:37:50.635047+0000","last_became_active":"2023-10-14T10:37:00.080160+0000","last_became_peered":"2023-10-14T10:37:00.080160+0000","last_unstale":"2023-10-14T10:37:50.635047+0000","last_undegraded":"2023-10-14T10:37:50.635047+0000","last_fullsized":"2023-10-14T10:37:50.635047+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:50.634876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:50.634876+0000","last_clean_scrub_stamp":"2023-10-14T10:37:50.634876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T18:35:52.324078+0000","scrub_duration":0.0023350100000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":23,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:29.788305+0000","last_change":"2023-10-14T10:37:29.788362+0000","last_active":"2023-10-14T10:37:29.788305+0000","last_peered":"2023-10-14T10:37:29.788305+0000","last_clean":"2023-10-14T10:37:29.788305+0000","last_became_active":"2023-10-14T10:36:59.067616+0000","last_became_peered":"2023-10-14T10:36:59.067616+0000","last_unstale":"2023-10-14T10:37:29.788305+0000","last_undegraded":"2023-10-14T10:37:29.788305+0000","last_fullsized":"2023-10-14T10:37:29.788305+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2023-10-14T10:37:29.788187+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2023-10-14T10:37:29.788187+0000","last_clean_scrub_stamp":"2023-10-14T10:37:29.788187+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:26:55.868531+0000","scrub_duration":0.0020300269999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:50.083851+0000","last_change":"2023-10-14T10:37:50.083896+0000","last_active":"2023-10-14T10:37:50.083851+0000","last_peered":"2023-10-14T10:37:50.083851+0000","last_clean":"2023-10-14T10:37:50.083851+0000","last_became_active":"2023-10-14T10:37:00.078829+0000","last_became_peered":"2023-10-14T10:37:00.078829+0000","last_unstale":"2023-10-14T10:37:50.083851+0000","last_undegraded":"2023-10-14T10:37:50.083851+0000","last_fullsized":"2023-10-14T10:37:50.083851+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:50.083751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:50.083751+0000","last_clean_scrub_stamp":"2023-10-14T10:37:50.083751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T20:09:30.814536+0000","scrub_duration":0.0019669029999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":23,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:12.636047+0000","last_change":"2023-10-14T10:37:12.636098+0000","last_active":"2023-10-14T10:37:12.636047+0000","last_peered":"2023-10-14T10:37:12.636047+0000","last_clean":"2023-10-14T10:37:12.636047+0000","last_became_active":"2023-10-14T10:36:59.065608+0000","last_became_peered":"2023-10-14T10:36:59.065608+0000","last_unstale":"2023-10-14T10:37:12.636047+0000","last_undegraded":"2023-10-14T10:37:12.636047+0000","last_fullsized":"2023-10-14T10:37:12.636047+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2023-10-14T10:37:12.635967+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2023-10-14T10:37:12.635967+0000","last_clean_scrub_stamp":"2023-10-14T10:37:12.635967+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:31:27.596001+0000","scrub_duration":0.0015686820000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:40.965920+0000","last_change":"2023-10-14T10:37:40.965968+0000","last_active":"2023-10-14T10:37:40.965920+0000","last_peered":"2023-10-14T10:37:40.965920+0000","last_clean":"2023-10-14T10:37:40.965920+0000","last_became_active":"2023-10-14T10:37:00.076840+0000","last_became_peered":"2023-10-14T10:37:00.076840+0000","last_unstale":"2023-10-14T10:37:40.965920+0000","last_undegraded":"2023-10-14T10:37:40.965920+0000","last_fullsized":"2023-10-14T10:37:40.965920+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:40.965808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:40.965808+0000","last_clean_scrub_stamp":"2023-10-14T10:37:40.965808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T21:22:38.923740+0000","scrub_duration":0.001657047,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:15.636426+0000","last_change":"2023-10-14T10:37:15.636479+0000","last_active":"2023-10-14T10:37:15.636426+0000","last_peered":"2023-10-14T10:37:15.636426+0000","last_clean":"2023-10-14T10:37:15.636426+0000","last_became_active":"2023-10-14T10:36:59.069192+0000","last_became_peered":"2023-10-14T10:36:59.069192+0000","last_unstale":"2023-10-14T10:37:15.636426+0000","last_undegraded":"2023-10-14T10:37:15.636426+0000","last_fullsized":"2023-10-14T10:37:15.636426+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1","last_scrub_stamp":"2023-10-14T10:37:15.636316+0000","last_deep_scrub":"32'1","last_deep_scrub_stamp":"2023-10-14T10:37:15.636316+0000","last_clean_scrub_stamp":"2023-10-14T10:37:15.636316+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:18:11.324774+0000","scrub_duration":0.0019544810000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.824189+0000","last_change":"2023-10-14T10:37:00.068786+0000","last_active":"2023-10-14T10:37:01.824189+0000","last_peered":"2023-10-14T10:37:01.824189+0000","last_clean":"2023-10-14T10:37:01.824189+0000","last_became_active":"2023-10-14T10:37:00.068517+0000","last_became_peered":"2023-10-14T10:37:00.068517+0000","last_unstale":"2023-10-14T10:37:01.824189+0000","last_undegraded":"2023-10-14T10:37:01.824189+0000","last_fullsized":"2023-10-14T10:37:01.824189+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:41.694588+0000","last_change":"2023-10-14T10:37:41.694642+0000","last_active":"2023-10-14T10:37:41.694588+0000","last_peered":"2023-10-14T10:37:41.694588+0000","last_clean":"2023-10-14T10:37:41.694588+0000","last_became_active":"2023-10-14T10:36:59.069151+0000","last_became_peered":"2023-10-14T10:36:59.069151+0000","last_unstale":"2023-10-14T10:37:41.694588+0000","last_undegraded":"2023-10-14T10:37:41.694588+0000","last_fullsized":"2023-10-14T10:37:41.694588+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1","last_scrub_stamp":"2023-10-14T10:37:41.694476+0000","last_deep_scrub":"32'1","last_deep_scrub_stamp":"2023-10-14T10:37:41.694476+0000","last_clean_scrub_stamp":"2023-10-14T10:37:41.694476+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T18:48:53.868548+0000","scrub_duration":0.00202717,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.824008+0000","last_change":"2023-10-14T10:37:00.076984+0000","last_active":"2023-10-14T10:37:01.824008+0000","last_peered":"2023-10-14T10:37:01.824008+0000","last_clean":"2023-10-14T10:37:01.824008+0000","last_became_active":"2023-10-14T10:37:00.076706+0000","last_became_peered":"2023-10-14T10:37:00.076706+0000","last_unstale":"2023-10-14T10:37:01.824008+0000","last_undegraded":"2023-10-14T10:37:01.824008+0000","last_fullsized":"2023-10-14T10:37:01.824008+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:26.865191+0000","last_change":"2023-10-14T10:37:26.865278+0000","last_active":"2023-10-14T10:37:26.865191+0000","last_peered":"2023-10-14T10:37:26.865191+0000","last_clean":"2023-10-14T10:37:26.865191+0000","last_became_active":"2023-10-14T10:36:59.062075+0000","last_became_peered":"2023-10-14T10:36:59.062075+0000","last_unstale":"2023-10-14T10:37:26.865191+0000","last_undegraded":"2023-10-14T10:37:26.865191+0000","last_fullsized":"2023-10-14T10:37:26.865191+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:26.865054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:26.865054+0000","last_clean_scrub_stamp":"2023-10-14T10:37:26.865054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:38:54.815977+0000","scrub_duration":0.00159884,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:35.070204+0000","last_change":"2023-10-14T10:37:35.070257+0000","last_active":"2023-10-14T10:37:35.070204+0000","last_peered":"2023-10-14T10:37:35.070204+0000","last_clean":"2023-10-14T10:37:35.070204+0000","last_became_active":"2023-10-14T10:37:00.067187+0000","last_became_peered":"2023-10-14T10:37:00.067187+0000","last_unstale":"2023-10-14T10:37:35.070204+0000","last_undegraded":"2023-10-14T10:37:35.070204+0000","last_fullsized":"2023-10-14T10:37:35.070204+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:35.070094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:35.070094+0000","last_clean_scrub_stamp":"2023-10-14T10:37:35.070094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:17:05.211354+0000","scrub_duration":0.00188517,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:29.738486+0000","last_change":"2023-10-14T10:37:29.738579+0000","last_active":"2023-10-14T10:37:29.738486+0000","last_peered":"2023-10-14T10:37:29.738486+0000","last_clean":"2023-10-14T10:37:29.738486+0000","last_became_active":"2023-10-14T10:36:59.064990+0000","last_became_peered":"2023-10-14T10:36:59.064990+0000","last_unstale":"2023-10-14T10:37:29.738486+0000","last_undegraded":"2023-10-14T10:37:29.738486+0000","last_fullsized":"2023-10-14T10:37:29.738486+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:29.738368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:29.738368+0000","last_clean_scrub_stamp":"2023-10-14T10:37:29.738368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:27:09.238644+0000","scrub_duration":0.0016393549999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.824313+0000","last_change":"2023-10-14T10:37:00.077465+0000","last_active":"2023-10-14T10:37:01.824313+0000","last_peered":"2023-10-14T10:37:01.824313+0000","last_clean":"2023-10-14T10:37:01.824313+0000","last_became_active":"2023-10-14T10:37:00.077152+0000","last_became_peered":"2023-10-14T10:37:00.077152+0000","last_unstale":"2023-10-14T10:37:01.824313+0000","last_undegraded":"2023-10-14T10:37:01.824313+0000","last_fullsized":"2023-10-14T10:37:01.824313+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:33.751083+0000","last_change":"2023-10-14T10:37:33.751175+0000","last_active":"2023-10-14T10:37:33.751083+0000","last_peered":"2023-10-14T10:37:33.751083+0000","last_clean":"2023-10-14T10:37:33.751083+0000","last_became_active":"2023-10-14T10:36:59.061597+0000","last_became_peered":"2023-10-14T10:36:59.061597+0000","last_unstale":"2023-10-14T10:37:33.751083+0000","last_undegraded":"2023-10-14T10:37:33.751083+0000","last_fullsized":"2023-10-14T10:37:33.751083+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:33.750809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:33.750809+0000","last_clean_scrub_stamp":"2023-10-14T10:37:33.750809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T14:08:21.846303+0000","scrub_duration":0.0020450709999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.822699+0000","last_change":"2023-10-14T10:37:00.076213+0000","last_active":"2023-10-14T10:37:01.822699+0000","last_peered":"2023-10-14T10:37:01.822699+0000","last_clean":"2023-10-14T10:37:01.822699+0000","last_became_active":"2023-10-14T10:37:00.075889+0000","last_became_peered":"2023-10-14T10:37:00.075889+0000","last_unstale":"2023-10-14T10:37:01.822699+0000","last_undegraded":"2023-10-14T10:37:01.822699+0000","last_fullsized":"2023-10-14T10:37:01.822699+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:20.017238+0000","last_change":"2023-10-14T10:37:20.017309+0000","last_active":"2023-10-14T10:37:20.017238+0000","last_peered":"2023-10-14T10:37:20.017238+0000","last_clean":"2023-10-14T10:37:20.017238+0000","last_became_active":"2023-10-14T10:36:59.069731+0000","last_became_peered":"2023-10-14T10:36:59.069731+0000","last_unstale":"2023-10-14T10:37:20.017238+0000","last_undegraded":"2023-10-14T10:37:20.017238+0000","last_fullsized":"2023-10-14T10:37:20.017238+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:20.017076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:20.017076+0000","last_clean_scrub_stamp":"2023-10-14T10:37:20.017076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T18:58:11.111987+0000","scrub_duration":0.001527189,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:41.672519+0000","last_change":"2023-10-14T10:37:41.672603+0000","last_active":"2023-10-14T10:37:41.672519+0000","last_peered":"2023-10-14T10:37:41.672519+0000","last_clean":"2023-10-14T10:37:41.672519+0000","last_became_active":"2023-10-14T10:37:00.072288+0000","last_became_peered":"2023-10-14T10:37:00.072288+0000","last_unstale":"2023-10-14T10:37:41.672519+0000","last_undegraded":"2023-10-14T10:37:41.672519+0000","last_fullsized":"2023-10-14T10:37:41.672519+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:41.672393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:41.672393+0000","last_clean_scrub_stamp":"2023-10-14T10:37:41.672393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:22:00.778165+0000","scrub_duration":0.0019683190000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":24,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:24.791215+0000","last_change":"2023-10-14T10:37:24.791349+0000","last_active":"2023-10-14T10:37:24.791215+0000","last_peered":"2023-10-14T10:37:24.791215+0000","last_clean":"2023-10-14T10:37:24.791215+0000","last_became_active":"2023-10-14T10:36:59.061700+0000","last_became_peered":"2023-10-14T10:36:59.061700+0000","last_unstale":"2023-10-14T10:37:24.791215+0000","last_undegraded":"2023-10-14T10:37:24.791215+0000","last_fullsized":"2023-10-14T10:37:24.791215+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3","last_scrub_stamp":"2023-10-14T10:37:24.791071+0000","last_deep_scrub":"32'3","last_deep_scrub_stamp":"2023-10-14T10:37:24.791071+0000","last_clean_scrub_stamp":"2023-10-14T10:37:24.791071+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:49:01.808383+0000","scrub_duration":0.0023118180000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:32.745124+0000","last_change":"2023-10-14T10:37:32.745181+0000","last_active":"2023-10-14T10:37:32.745124+0000","last_peered":"2023-10-14T10:37:32.745124+0000","last_clean":"2023-10-14T10:37:32.745124+0000","last_became_active":"2023-10-14T10:37:00.074700+0000","last_became_peered":"2023-10-14T10:37:00.074700+0000","last_unstale":"2023-10-14T10:37:32.745124+0000","last_undegraded":"2023-10-14T10:37:32.745124+0000","last_fullsized":"2023-10-14T10:37:32.745124+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:32.745006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:32.745006+0000","last_clean_scrub_stamp":"2023-10-14T10:37:32.745006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:57:14.440088+0000","scrub_duration":0.0019991470000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"33'10","reported_seq":31,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:36.684773+0000","last_change":"2023-10-14T10:37:36.684835+0000","last_active":"2023-10-14T10:37:36.684773+0000","last_peered":"2023-10-14T10:37:36.684773+0000","last_clean":"2023-10-14T10:37:36.684773+0000","last_became_active":"2023-10-14T10:36:59.069043+0000","last_became_peered":"2023-10-14T10:36:59.069043+0000","last_unstale":"2023-10-14T10:37:36.684773+0000","last_undegraded":"2023-10-14T10:37:36.684773+0000","last_fullsized":"2023-10-14T10:37:36.684773+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'10","last_scrub_stamp":"2023-10-14T10:37:36.684645+0000","last_deep_scrub":"33'10","last_deep_scrub_stamp":"2023-10-14T10:37:36.684645+0000","last_clean_scrub_stamp":"2023-10-14T10:37:36.684645+0000","objects_scrubbed":4,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:18:50.464330+0000","scrub_duration":0.005487306,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:38.036544+0000","last_change":"2023-10-14T10:37:38.036631+0000","last_active":"2023-10-14T10:37:38.036544+0000","last_peered":"2023-10-14T10:37:38.036544+0000","last_clean":"2023-10-14T10:37:38.036544+0000","last_became_active":"2023-10-14T10:37:00.065345+0000","last_became_peered":"2023-10-14T10:37:00.065345+0000","last_unstale":"2023-10-14T10:37:38.036544+0000","last_undegraded":"2023-10-14T10:37:38.036544+0000","last_fullsized":"2023-10-14T10:37:38.036544+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:38.036400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:38.036400+0000","last_clean_scrub_stamp":"2023-10-14T10:37:38.036400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:39:05.987323+0000","scrub_duration":0.002126911,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"33'4","reported_seq":25,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:30.752534+0000","last_change":"2023-10-14T10:37:30.752613+0000","last_active":"2023-10-14T10:37:30.752534+0000","last_peered":"2023-10-14T10:37:30.752534+0000","last_clean":"2023-10-14T10:37:30.752534+0000","last_became_active":"2023-10-14T10:36:59.065569+0000","last_became_peered":"2023-10-14T10:36:59.065569+0000","last_unstale":"2023-10-14T10:37:30.752534+0000","last_undegraded":"2023-10-14T10:37:30.752534+0000","last_fullsized":"2023-10-14T10:37:30.752534+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'4","last_scrub_stamp":"2023-10-14T10:37:30.752424+0000","last_deep_scrub":"33'4","last_deep_scrub_stamp":"2023-10-14T10:37:30.752424+0000","last_clean_scrub_stamp":"2023-10-14T10:37:30.752424+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T12:29:20.109316+0000","scrub_duration":0.0039809850000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:49.868165+0000","last_change":"2023-10-14T10:37:49.868216+0000","last_active":"2023-10-14T10:37:49.868165+0000","last_peered":"2023-10-14T10:37:49.868165+0000","last_clean":"2023-10-14T10:37:49.868165+0000","last_became_active":"2023-10-14T10:37:00.078844+0000","last_became_peered":"2023-10-14T10:37:00.078844+0000","last_unstale":"2023-10-14T10:37:49.868165+0000","last_undegraded":"2023-10-14T10:37:49.868165+0000","last_fullsized":"2023-10-14T10:37:49.868165+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:49.868025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:49.868025+0000","last_clean_scrub_stamp":"2023-10-14T10:37:49.868025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:34:29.629255+0000","scrub_duration":0.001667027,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:21.918946+0000","last_change":"2023-10-14T10:37:21.919039+0000","last_active":"2023-10-14T10:37:21.918946+0000","last_peered":"2023-10-14T10:37:21.918946+0000","last_clean":"2023-10-14T10:37:21.918946+0000","last_became_active":"2023-10-14T10:36:59.066952+0000","last_became_peered":"2023-10-14T10:36:59.066952+0000","last_unstale":"2023-10-14T10:37:21.918946+0000","last_undegraded":"2023-10-14T10:37:21.918946+0000","last_fullsized":"2023-10-14T10:37:21.918946+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:21.918813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:21.918813+0000","last_clean_scrub_stamp":"2023-10-14T10:37:21.918813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:53:15.852647+0000","scrub_duration":0.002202907,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:33.943228+0000","last_change":"2023-10-14T10:37:33.943309+0000","last_active":"2023-10-14T10:37:33.943228+0000","last_peered":"2023-10-14T10:37:33.943228+0000","last_clean":"2023-10-14T10:37:33.943228+0000","last_became_active":"2023-10-14T10:37:00.078353+0000","last_became_peered":"2023-10-14T10:37:00.078353+0000","last_unstale":"2023-10-14T10:37:33.943228+0000","last_undegraded":"2023-10-14T10:37:33.943228+0000","last_fullsized":"2023-10-14T10:37:33.943228+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:33.943115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:33.943115+0000","last_clean_scrub_stamp":"2023-10-14T10:37:33.943115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T20:01:10.991993+0000","scrub_duration":0.001510136,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:19.819168+0000","last_change":"2023-10-14T10:37:19.819220+0000","last_active":"2023-10-14T10:37:19.819168+0000","last_peered":"2023-10-14T10:37:19.819168+0000","last_clean":"2023-10-14T10:37:19.819168+0000","last_became_active":"2023-10-14T10:36:59.062380+0000","last_became_peered":"2023-10-14T10:36:59.062380+0000","last_unstale":"2023-10-14T10:37:19.819168+0000","last_undegraded":"2023-10-14T10:37:19.819168+0000","last_fullsized":"2023-10-14T10:37:19.819168+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:19.819047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:19.819047+0000","last_clean_scrub_stamp":"2023-10-14T10:37:19.819047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:16:49.960306+0000","scrub_duration":0.001802477,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:24.980015+0000","last_change":"2023-10-14T10:37:24.980059+0000","last_active":"2023-10-14T10:37:24.980015+0000","last_peered":"2023-10-14T10:37:24.980015+0000","last_clean":"2023-10-14T10:37:24.980015+0000","last_became_active":"2023-10-14T10:37:00.077806+0000","last_became_peered":"2023-10-14T10:37:00.077806+0000","last_unstale":"2023-10-14T10:37:24.980015+0000","last_undegraded":"2023-10-14T10:37:24.980015+0000","last_fullsized":"2023-10-14T10:37:24.980015+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:24.979906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:24.979906+0000","last_clean_scrub_stamp":"2023-10-14T10:37:24.979906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T17:28:16.344297+0000","scrub_duration":0.0015380680000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":29,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:17.814572+0000","last_change":"2023-10-14T10:37:17.814685+0000","last_active":"2023-10-14T10:37:17.814572+0000","last_peered":"2023-10-14T10:37:17.814572+0000","last_clean":"2023-10-14T10:37:17.814572+0000","last_became_active":"2023-10-14T10:36:59.067727+0000","last_became_peered":"2023-10-14T10:36:59.067727+0000","last_unstale":"2023-10-14T10:37:17.814572+0000","last_undegraded":"2023-10-14T10:37:17.814572+0000","last_fullsized":"2023-10-14T10:37:17.814572+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'8","last_scrub_stamp":"2023-10-14T10:37:17.814452+0000","last_deep_scrub":"32'8","last_deep_scrub_stamp":"2023-10-14T10:37:17.814452+0000","last_clean_scrub_stamp":"2023-10-14T10:37:17.814452+0000","objects_scrubbed":4,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:25:42.460457+0000","scrub_duration":0.002353843,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.824274+0000","last_change":"2023-10-14T10:37:00.076926+0000","last_active":"2023-10-14T10:37:01.824274+0000","last_peered":"2023-10-14T10:37:01.824274+0000","last_clean":"2023-10-14T10:37:01.824274+0000","last_became_active":"2023-10-14T10:37:00.076658+0000","last_became_peered":"2023-10-14T10:37:00.076658+0000","last_unstale":"2023-10-14T10:37:01.824274+0000","last_undegraded":"2023-10-14T10:37:01.824274+0000","last_fullsized":"2023-10-14T10:37:01.824274+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:14.880170+0000","last_change":"2023-10-14T10:37:14.880213+0000","last_active":"2023-10-14T10:37:14.880170+0000","last_peered":"2023-10-14T10:37:14.880170+0000","last_clean":"2023-10-14T10:37:14.880170+0000","last_became_active":"2023-10-14T10:36:59.071406+0000","last_became_peered":"2023-10-14T10:36:59.071406+0000","last_unstale":"2023-10-14T10:37:14.880170+0000","last_undegraded":"2023-10-14T10:37:14.880170+0000","last_fullsized":"2023-10-14T10:37:14.880170+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1","last_scrub_stamp":"2023-10-14T10:37:14.880064+0000","last_deep_scrub":"32'1","last_deep_scrub_stamp":"2023-10-14T10:37:14.880064+0000","last_clean_scrub_stamp":"2023-10-14T10:37:14.880064+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T17:09:37.404058+0000","scrub_duration":0.0017514850000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.824351+0000","last_change":"2023-10-14T10:37:00.080060+0000","last_active":"2023-10-14T10:37:01.824351+0000","last_peered":"2023-10-14T10:37:01.824351+0000","last_clean":"2023-10-14T10:37:01.824351+0000","last_became_active":"2023-10-14T10:37:00.079795+0000","last_became_peered":"2023-10-14T10:37:00.079795+0000","last_unstale":"2023-10-14T10:37:01.824351+0000","last_undegraded":"2023-10-14T10:37:01.824351+0000","last_fullsized":"2023-10-14T10:37:01.824351+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"33'3","reported_seq":24,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:35.680521+0000","last_change":"2023-10-14T10:37:35.680617+0000","last_active":"2023-10-14T10:37:35.680521+0000","last_peered":"2023-10-14T10:37:35.680521+0000","last_clean":"2023-10-14T10:37:35.680521+0000","last_became_active":"2023-10-14T10:36:59.064959+0000","last_became_peered":"2023-10-14T10:36:59.064959+0000","last_unstale":"2023-10-14T10:37:35.680521+0000","last_undegraded":"2023-10-14T10:37:35.680521+0000","last_fullsized":"2023-10-14T10:37:35.680521+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'3","last_scrub_stamp":"2023-10-14T10:37:35.680397+0000","last_deep_scrub":"33'3","last_deep_scrub_stamp":"2023-10-14T10:37:35.680397+0000","last_clean_scrub_stamp":"2023-10-14T10:37:35.680397+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T21:40:59.477393+0000","scrub_duration":0.0031079279999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:44.984654+0000","last_change":"2023-10-14T10:37:44.984702+0000","last_active":"2023-10-14T10:37:44.984654+0000","last_peered":"2023-10-14T10:37:44.984654+0000","last_clean":"2023-10-14T10:37:44.984654+0000","last_became_active":"2023-10-14T10:37:00.077459+0000","last_became_peered":"2023-10-14T10:37:00.077459+0000","last_unstale":"2023-10-14T10:37:44.984654+0000","last_undegraded":"2023-10-14T10:37:44.984654+0000","last_fullsized":"2023-10-14T10:37:44.984654+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:44.984535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:44.984535+0000","last_clean_scrub_stamp":"2023-10-14T10:37:44.984535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:18:58.764220+0000","scrub_duration":0.0020937130000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"33'2","reported_seq":23,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:39.719846+0000","last_change":"2023-10-14T10:37:39.719936+0000","last_active":"2023-10-14T10:37:39.719846+0000","last_peered":"2023-10-14T10:37:39.719846+0000","last_clean":"2023-10-14T10:37:39.719846+0000","last_became_active":"2023-10-14T10:36:59.058558+0000","last_became_peered":"2023-10-14T10:36:59.058558+0000","last_unstale":"2023-10-14T10:37:39.719846+0000","last_undegraded":"2023-10-14T10:37:39.719846+0000","last_fullsized":"2023-10-14T10:37:39.719846+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2023-10-14T10:37:39.719723+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2023-10-14T10:37:39.719723+0000","last_clean_scrub_stamp":"2023-10-14T10:37:39.719723+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:17:09.860982+0000","scrub_duration":0.0042980579999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:31.349037+0000","last_change":"2023-10-14T10:37:31.349116+0000","last_active":"2023-10-14T10:37:31.349037+0000","last_peered":"2023-10-14T10:37:31.349037+0000","last_clean":"2023-10-14T10:37:31.349037+0000","last_became_active":"2023-10-14T10:37:00.071167+0000","last_became_peered":"2023-10-14T10:37:00.071167+0000","last_unstale":"2023-10-14T10:37:31.349037+0000","last_undegraded":"2023-10-14T10:37:31.349037+0000","last_fullsized":"2023-10-14T10:37:31.349037+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:31.348892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:31.348892+0000","last_clean_scrub_stamp":"2023-10-14T10:37:31.348892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T17:21:44.324153+0000","scrub_duration":0.001948319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:21.671764+0000","last_change":"2023-10-14T10:37:21.671823+0000","last_active":"2023-10-14T10:37:21.671764+0000","last_peered":"2023-10-14T10:37:21.671764+0000","last_clean":"2023-10-14T10:37:21.671764+0000","last_became_active":"2023-10-14T10:36:59.069110+0000","last_became_peered":"2023-10-14T10:36:59.069110+0000","last_unstale":"2023-10-14T10:37:21.671764+0000","last_undegraded":"2023-10-14T10:37:21.671764+0000","last_fullsized":"2023-10-14T10:37:21.671764+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:21.671648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:21.671648+0000","last_clean_scrub_stamp":"2023-10-14T10:37:21.671648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T14:19:44.940820+0000","scrub_duration":0.0022981799999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:29.416731+0000","last_change":"2023-10-14T10:37:29.416782+0000","last_active":"2023-10-14T10:37:29.416731+0000","last_peered":"2023-10-14T10:37:29.416731+0000","last_clean":"2023-10-14T10:37:29.416731+0000","last_became_active":"2023-10-14T10:37:00.077175+0000","last_became_peered":"2023-10-14T10:37:00.077175+0000","last_unstale":"2023-10-14T10:37:29.416731+0000","last_undegraded":"2023-10-14T10:37:29.416731+0000","last_fullsized":"2023-10-14T10:37:29.416731+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:29.416599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:29.416599+0000","last_clean_scrub_stamp":"2023-10-14T10:37:29.416599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:22:40.667629+0000","scrub_duration":0.00185724,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":23,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:18.035307+0000","last_change":"2023-10-14T10:37:18.035397+0000","last_active":"2023-10-14T10:37:18.035307+0000","last_peered":"2023-10-14T10:37:18.035307+0000","last_clean":"2023-10-14T10:37:18.035307+0000","last_became_active":"2023-10-14T10:36:59.067345+0000","last_became_peered":"2023-10-14T10:36:59.067345+0000","last_unstale":"2023-10-14T10:37:18.035307+0000","last_undegraded":"2023-10-14T10:37:18.035307+0000","last_fullsized":"2023-10-14T10:37:18.035307+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2023-10-14T10:37:18.035173+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2023-10-14T10:37:18.035173+0000","last_clean_scrub_stamp":"2023-10-14T10:37:18.035173+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:51:19.160977+0000","scrub_duration":0.0020979919999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:20.573039+0000","last_change":"2023-10-14T10:37:20.573138+0000","last_active":"2023-10-14T10:37:20.573039+0000","last_peered":"2023-10-14T10:37:20.573039+0000","last_clean":"2023-10-14T10:37:20.573039+0000","last_became_active":"2023-10-14T10:37:00.077465+0000","last_became_peered":"2023-10-14T10:37:00.077465+0000","last_unstale":"2023-10-14T10:37:20.573039+0000","last_undegraded":"2023-10-14T10:37:20.573039+0000","last_fullsized":"2023-10-14T10:37:20.573039+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:20.572921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:20.572921+0000","last_clean_scrub_stamp":"2023-10-14T10:37:20.572921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:27:44.149277+0000","scrub_duration":0.001569429,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:16.998159+0000","last_change":"2023-10-14T10:37:16.998214+0000","last_active":"2023-10-14T10:37:16.998159+0000","last_peered":"2023-10-14T10:37:16.998159+0000","last_clean":"2023-10-14T10:37:16.998159+0000","last_became_active":"2023-10-14T10:36:59.069269+0000","last_became_peered":"2023-10-14T10:36:59.069269+0000","last_unstale":"2023-10-14T10:37:16.998159+0000","last_undegraded":"2023-10-14T10:37:16.998159+0000","last_fullsized":"2023-10-14T10:37:16.998159+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:16.998041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:16.998041+0000","last_clean_scrub_stamp":"2023-10-14T10:37:16.998041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:52:55.851068+0000","scrub_duration":0.002349617,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:22.513537+0000","last_change":"2023-10-14T10:37:22.513590+0000","last_active":"2023-10-14T10:37:22.513537+0000","last_peered":"2023-10-14T10:37:22.513537+0000","last_clean":"2023-10-14T10:37:22.513537+0000","last_became_active":"2023-10-14T10:37:00.068620+0000","last_became_peered":"2023-10-14T10:37:00.068620+0000","last_unstale":"2023-10-14T10:37:22.513537+0000","last_undegraded":"2023-10-14T10:37:22.513537+0000","last_fullsized":"2023-10-14T10:37:22.513537+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:22.513405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:22.513405+0000","last_clean_scrub_stamp":"2023-10-14T10:37:22.513405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:28:45.771382+0000","scrub_duration":0.0016531670000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:17.591859+0000","last_change":"2023-10-14T10:37:17.591924+0000","last_active":"2023-10-14T10:37:17.591859+0000","last_peered":"2023-10-14T10:37:17.591859+0000","last_clean":"2023-10-14T10:37:17.591859+0000","last_became_active":"2023-10-14T10:36:59.061678+0000","last_became_peered":"2023-10-14T10:36:59.061678+0000","last_unstale":"2023-10-14T10:37:17.591859+0000","last_undegraded":"2023-10-14T10:37:17.591859+0000","last_fullsized":"2023-10-14T10:37:17.591859+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:17.591741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:17.591741+0000","last_clean_scrub_stamp":"2023-10-14T10:37:17.591741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T14:52:36.610528+0000","scrub_duration":0.0021650459999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:27.005340+0000","last_change":"2023-10-14T10:37:27.005388+0000","last_active":"2023-10-14T10:37:27.005340+0000","last_peered":"2023-10-14T10:37:27.005340+0000","last_clean":"2023-10-14T10:37:27.005340+0000","last_became_active":"2023-10-14T10:37:00.071692+0000","last_became_peered":"2023-10-14T10:37:00.071692+0000","last_unstale":"2023-10-14T10:37:27.005340+0000","last_undegraded":"2023-10-14T10:37:27.005340+0000","last_fullsized":"2023-10-14T10:37:27.005340+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:27.005226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:27.005226+0000","last_clean_scrub_stamp":"2023-10-14T10:37:27.005226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:38:54.956149+0000","scrub_duration":0.0015000110000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:13.801179+0000","last_change":"2023-10-14T10:37:13.801233+0000","last_active":"2023-10-14T10:37:13.801179+0000","last_peered":"2023-10-14T10:37:13.801179+0000","last_clean":"2023-10-14T10:37:13.801179+0000","last_became_active":"2023-10-14T10:36:59.063332+0000","last_became_peered":"2023-10-14T10:36:59.063332+0000","last_unstale":"2023-10-14T10:37:13.801179+0000","last_undegraded":"2023-10-14T10:37:13.801179+0000","last_fullsized":"2023-10-14T10:37:13.801179+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:13.801033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:13.801033+0000","last_clean_scrub_stamp":"2023-10-14T10:37:13.801033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T14:51:03.775865+0000","scrub_duration":0.0015480699999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:24.453404+0000","last_change":"2023-10-14T10:37:24.453463+0000","last_active":"2023-10-14T10:37:24.453404+0000","last_peered":"2023-10-14T10:37:24.453404+0000","last_clean":"2023-10-14T10:37:24.453404+0000","last_became_active":"2023-10-14T10:37:00.074859+0000","last_became_peered":"2023-10-14T10:37:00.074859+0000","last_unstale":"2023-10-14T10:37:24.453404+0000","last_undegraded":"2023-10-14T10:37:24.453404+0000","last_fullsized":"2023-10-14T10:37:24.453404+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:24.453276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:24.453276+0000","last_clean_scrub_stamp":"2023-10-14T10:37:24.453276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T21:33:20.669114+0000","scrub_duration":0.002218785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":23,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:23.963239+0000","last_change":"2023-10-14T10:37:23.963301+0000","last_active":"2023-10-14T10:37:23.963239+0000","last_peered":"2023-10-14T10:37:23.963239+0000","last_clean":"2023-10-14T10:37:23.963239+0000","last_became_active":"2023-10-14T10:36:59.069037+0000","last_became_peered":"2023-10-14T10:36:59.069037+0000","last_unstale":"2023-10-14T10:37:23.963239+0000","last_undegraded":"2023-10-14T10:37:23.963239+0000","last_fullsized":"2023-10-14T10:37:23.963239+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2023-10-14T10:37:23.963112+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2023-10-14T10:37:23.963112+0000","last_clean_scrub_stamp":"2023-10-14T10:37:23.963112+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:49:00.980424+0000","scrub_duration":0.0020882309999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:01.824330+0000","last_change":"2023-10-14T10:37:00.075731+0000","last_active":"2023-10-14T10:37:01.824330+0000","last_peered":"2023-10-14T10:37:01.824330+0000","last_clean":"2023-10-14T10:37:01.824330+0000","last_became_active":"2023-10-14T10:37:00.075491+0000","last_became_peered":"2023-10-14T10:37:00.075491+0000","last_unstale":"2023-10-14T10:37:01.824330+0000","last_undegraded":"2023-10-14T10:37:01.824330+0000","last_fullsized":"2023-10-14T10:37:01.824330+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:36:59.041694+0000","last_clean_scrub_stamp":"2023-10-14T10:36:59.041694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"33'1","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:19.004875+0000","last_change":"2023-10-14T10:37:19.004932+0000","last_active":"2023-10-14T10:37:19.004875+0000","last_peered":"2023-10-14T10:37:19.004875+0000","last_clean":"2023-10-14T10:37:19.004875+0000","last_became_active":"2023-10-14T10:36:59.066146+0000","last_became_peered":"2023-10-14T10:36:59.066146+0000","last_unstale":"2023-10-14T10:37:19.004875+0000","last_undegraded":"2023-10-14T10:37:19.004875+0000","last_fullsized":"2023-10-14T10:37:19.004875+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'1","last_scrub_stamp":"2023-10-14T10:37:19.004745+0000","last_deep_scrub":"33'1","last_deep_scrub_stamp":"2023-10-14T10:37:19.004745+0000","last_clean_scrub_stamp":"2023-10-14T10:37:19.004745+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T10:48:23.006435+0000","scrub_duration":0.0020016869999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:26.115864+0000","last_change":"2023-10-14T10:37:26.115959+0000","last_active":"2023-10-14T10:37:26.115864+0000","last_peered":"2023-10-14T10:37:26.115864+0000","last_clean":"2023-10-14T10:37:26.115864+0000","last_became_active":"2023-10-14T10:37:00.082570+0000","last_became_peered":"2023-10-14T10:37:00.082570+0000","last_unstale":"2023-10-14T10:37:26.115864+0000","last_undegraded":"2023-10-14T10:37:26.115864+0000","last_fullsized":"2023-10-14T10:37:26.115864+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:26.115725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:26.115725+0000","last_clean_scrub_stamp":"2023-10-14T10:37:26.115725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:49:03.133037+0000","scrub_duration":0.0016919400000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:33.077799+0000","last_change":"2023-10-14T10:37:33.077851+0000","last_active":"2023-10-14T10:37:33.077799+0000","last_peered":"2023-10-14T10:37:33.077799+0000","last_clean":"2023-10-14T10:37:33.077799+0000","last_became_active":"2023-10-14T10:36:59.061642+0000","last_became_peered":"2023-10-14T10:36:59.061642+0000","last_unstale":"2023-10-14T10:37:33.077799+0000","last_undegraded":"2023-10-14T10:37:33.077799+0000","last_fullsized":"2023-10-14T10:37:33.077799+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:33.077684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:33.077684+0000","last_clean_scrub_stamp":"2023-10-14T10:37:33.077684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:25:57.723689+0000","scrub_duration":0.0016094810000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:23.976059+0000","last_change":"2023-10-14T10:37:23.976112+0000","last_active":"2023-10-14T10:37:23.976059+0000","last_peered":"2023-10-14T10:37:23.976059+0000","last_clean":"2023-10-14T10:37:23.976059+0000","last_became_active":"2023-10-14T10:37:00.071244+0000","last_became_peered":"2023-10-14T10:37:00.071244+0000","last_unstale":"2023-10-14T10:37:23.976059+0000","last_undegraded":"2023-10-14T10:37:23.976059+0000","last_fullsized":"2023-10-14T10:37:23.976059+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:23.975945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:23.975945+0000","last_clean_scrub_stamp":"2023-10-14T10:37:23.975945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T10:48:27.977635+0000","scrub_duration":0.0019053919999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:12.061873+0000","last_change":"2023-10-14T10:37:12.061941+0000","last_active":"2023-10-14T10:37:12.061873+0000","last_peered":"2023-10-14T10:37:12.061873+0000","last_clean":"2023-10-14T10:37:12.061873+0000","last_became_active":"2023-10-14T10:36:59.066592+0000","last_became_peered":"2023-10-14T10:36:59.066592+0000","last_unstale":"2023-10-14T10:37:12.061873+0000","last_undegraded":"2023-10-14T10:37:12.061873+0000","last_fullsized":"2023-10-14T10:37:12.061873+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:12.061781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:12.061781+0000","last_clean_scrub_stamp":"2023-10-14T10:37:12.061781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:52:50.914807+0000","scrub_duration":0.0016784320000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:23.082899+0000","last_change":"2023-10-14T10:37:23.082942+0000","last_active":"2023-10-14T10:37:23.082899+0000","last_peered":"2023-10-14T10:37:23.082899+0000","last_clean":"2023-10-14T10:37:23.082899+0000","last_became_active":"2023-10-14T10:37:00.075794+0000","last_became_peered":"2023-10-14T10:37:00.075794+0000","last_unstale":"2023-10-14T10:37:23.082899+0000","last_undegraded":"2023-10-14T10:37:23.082899+0000","last_fullsized":"2023-10-14T10:37:23.082899+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:23.082797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:23.082797+0000","last_clean_scrub_stamp":"2023-10-14T10:37:23.082797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T17:09:45.606791+0000","scrub_duration":0.0017434429999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:12.796373+0000","last_change":"2023-10-14T10:37:12.796416+0000","last_active":"2023-10-14T10:37:12.796373+0000","last_peered":"2023-10-14T10:37:12.796373+0000","last_clean":"2023-10-14T10:37:12.796373+0000","last_became_active":"2023-10-14T10:36:59.065163+0000","last_became_peered":"2023-10-14T10:36:59.065163+0000","last_unstale":"2023-10-14T10:37:12.796373+0000","last_undegraded":"2023-10-14T10:37:12.796373+0000","last_fullsized":"2023-10-14T10:37:12.796373+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:12.796279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:12.796279+0000","last_clean_scrub_stamp":"2023-10-14T10:37:12.796279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:06:42.145631+0000","scrub_duration":0.001883366,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:39.741488+0000","last_change":"2023-10-14T10:37:39.741533+0000","last_active":"2023-10-14T10:37:39.741488+0000","last_peered":"2023-10-14T10:37:39.741488+0000","last_clean":"2023-10-14T10:37:39.741488+0000","last_became_active":"2023-10-14T10:37:00.077621+0000","last_became_peered":"2023-10-14T10:37:00.077621+0000","last_unstale":"2023-10-14T10:37:39.741488+0000","last_undegraded":"2023-10-14T10:37:39.741488+0000","last_fullsized":"2023-10-14T10:37:39.741488+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:39.741387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:39.741387+0000","last_clean_scrub_stamp":"2023-10-14T10:37:39.741387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:08:03.954584+0000","scrub_duration":0.0017780370000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:36.714140+0000","last_change":"2023-10-14T10:37:36.714192+0000","last_active":"2023-10-14T10:37:36.714140+0000","last_peered":"2023-10-14T10:37:36.714140+0000","last_clean":"2023-10-14T10:37:36.714140+0000","last_became_active":"2023-10-14T10:36:59.064336+0000","last_became_peered":"2023-10-14T10:36:59.064336+0000","last_unstale":"2023-10-14T10:37:36.714140+0000","last_undegraded":"2023-10-14T10:37:36.714140+0000","last_fullsized":"2023-10-14T10:37:36.714140+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:36.714012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:36.714012+0000","last_clean_scrub_stamp":"2023-10-14T10:37:36.714012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:26:01.360017+0000","scrub_duration":0.0015309290000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:45.701835+0000","last_change":"2023-10-14T10:37:45.701887+0000","last_active":"2023-10-14T10:37:45.701835+0000","last_peered":"2023-10-14T10:37:45.701835+0000","last_clean":"2023-10-14T10:37:45.701835+0000","last_became_active":"2023-10-14T10:37:00.080340+0000","last_became_peered":"2023-10-14T10:37:00.080340+0000","last_unstale":"2023-10-14T10:37:45.701835+0000","last_undegraded":"2023-10-14T10:37:45.701835+0000","last_fullsized":"2023-10-14T10:37:45.701835+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:45.701714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:45.701714+0000","last_clean_scrub_stamp":"2023-10-14T10:37:45.701714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T14:08:33.797207+0000","scrub_duration":0.002227639,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:16.807142+0000","last_change":"2023-10-14T10:37:16.807232+0000","last_active":"2023-10-14T10:37:16.807142+0000","last_peered":"2023-10-14T10:37:16.807142+0000","last_clean":"2023-10-14T10:37:16.807142+0000","last_became_active":"2023-10-14T10:36:59.063130+0000","last_became_peered":"2023-10-14T10:36:59.063130+0000","last_unstale":"2023-10-14T10:37:16.807142+0000","last_undegraded":"2023-10-14T10:37:16.807142+0000","last_fullsized":"2023-10-14T10:37:16.807142+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:16.806982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:16.806982+0000","last_clean_scrub_stamp":"2023-10-14T10:37:16.806982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T21:40:40.603978+0000","scrub_duration":0.0021981460000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:28.948640+0000","last_change":"2023-10-14T10:37:28.948692+0000","last_active":"2023-10-14T10:37:28.948640+0000","last_peered":"2023-10-14T10:37:28.948640+0000","last_clean":"2023-10-14T10:37:28.948640+0000","last_became_active":"2023-10-14T10:37:00.068556+0000","last_became_peered":"2023-10-14T10:37:00.068556+0000","last_unstale":"2023-10-14T10:37:28.948640+0000","last_undegraded":"2023-10-14T10:37:28.948640+0000","last_fullsized":"2023-10-14T10:37:28.948640+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:28.948488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:28.948488+0000","last_clean_scrub_stamp":"2023-10-14T10:37:28.948488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:15:09.923968+0000","scrub_duration":0.0020176769999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:14.983447+0000","last_change":"2023-10-14T10:37:14.983500+0000","last_active":"2023-10-14T10:37:14.983447+0000","last_peered":"2023-10-14T10:37:14.983447+0000","last_clean":"2023-10-14T10:37:14.983447+0000","last_became_active":"2023-10-14T10:36:59.066424+0000","last_became_peered":"2023-10-14T10:36:59.066424+0000","last_unstale":"2023-10-14T10:37:14.983447+0000","last_undegraded":"2023-10-14T10:37:14.983447+0000","last_fullsized":"2023-10-14T10:37:14.983447+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:14.983339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:14.983339+0000","last_clean_scrub_stamp":"2023-10-14T10:37:14.983339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:28:20.478594+0000","scrub_duration":0.0016413619999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"33'119","reported_seq":187,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:14.949519+0000","last_change":"2023-10-14T10:37:14.949621+0000","last_active":"2023-10-14T10:37:14.949519+0000","last_peered":"2023-10-14T10:37:14.949519+0000","last_clean":"2023-10-14T10:37:14.949519+0000","last_became_active":"2023-10-14T10:35:59.769199+0000","last_became_peered":"2023-10-14T10:35:59.769199+0000","last_unstale":"2023-10-14T10:37:14.949519+0000","last_undegraded":"2023-10-14T10:37:14.949519+0000","last_fullsized":"2023-10-14T10:37:14.949519+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'119","last_scrub_stamp":"2023-10-14T10:37:14.949400+0000","last_deep_scrub":"33'119","last_deep_scrub_stamp":"2023-10-14T10:37:14.949400+0000","last_clean_scrub_stamp":"2023-10-14T10:37:14.949400+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:53:08.883234+0000","scrub_duration":0.0050812649999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":47,"ondisk_log_size":47,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542171,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27960,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343880,"statfs":{"total":96636764160,"available":96608133120,"internally_reserved":0,"allocated":966656,"data_stored":732594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 10:36:59 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":2.4649999999999999,"5min":2.4649999999999999,"15min":2.4649999999999999},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.61299999999999999}]},{"osd":1,"last update":"Sat Oct 14 10:36:59 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.626},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":2.4289999999999998,"5min":2.4289999999999998,"15min":2.4289999999999998},"last":0.59999999999999998}]},{"osd":2,"last update":"Sat Oct 14 10:36:59 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.55100000000000005}]},{"osd":3,"last update":"Sat Oct 14 10:36:59 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.405,"5min":1.405,"15min":1.405},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.56399999999999995}]},{"osd":4,"last update":"Sat Oct 14 10:36:59 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.53100000000000003}]},{"osd":5,"last update":"Sat Oct 14 10:36:59 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.37},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.41999999999999998}]},{"osd":6,"last update":"Sat Oct 14 10:36:59 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.44600000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542172,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":376832,"data_stored":142752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 10:36:57 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.46800000000000003}]},{"osd":1,"last update":"Sat Oct 14 10:36:57 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.389,"5min":1.389,"15min":1.389},"last":0.63200000000000001}]},{"osd":2,"last update":"Sat Oct 14 10:36:57 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.44800000000000001}]},{"osd":3,"last update":"Sat Oct 14 10:36:57 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.36899999999999999}]},{"osd":4,"last update":"Sat Oct 14 10:36:57 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.52700000000000002}]},{"osd":5,"last update":"Sat Oct 14 10:36:57 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.495},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.50900000000000001}]},{"osd":7,"last update":"Sat Oct 14 10:36:57 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.42199999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542171,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":376,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":385024,"data_stored":143770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.254,"15min":1.254},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":16.855,"5min":16.855,"15min":16.855},"last":0.44500000000000001},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":19.353000000000002,"5min":19.353000000000002,"15min":19.353000000000002},"last":0.42299999999999999}]},{"osd":2,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":3.0670000000000002,"5min":3.0670000000000002,"15min":3.0670000000000002},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.58699999999999997}]},{"osd":3,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":10.005000000000001,"5min":10.005000000000001,"15min":10.005000000000001},"last":0.48099999999999998}]},{"osd":4,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":2.9500000000000002,"5min":2.9500000000000002,"15min":2.9500000000000002},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":5.7270000000000003,"5min":5.7270000000000003,"15min":5.7270000000000003},"last":0.69499999999999995}]},{"osd":5,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.129,"15min":1.129},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":9.8800000000000008,"5min":9.8800000000000008,"15min":9.8800000000000008},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":3.0350000000000001,"5min":3.0350000000000001,"15min":3.0350000000000001},"last":0.71099999999999997}]},{"osd":6,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":5.681,"5min":5.681,"15min":5.681},"last":0.66200000000000003},{"interface":"front","average":{"1min":1.173,"5min":1.173,"15min":1.173},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":9.9380000000000006,"5min":9.9380000000000006,"15min":9.9380000000000006},"last":0.67500000000000004}]},{"osd":7,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":9.9730000000000008,"5min":9.9730000000000008,"15min":9.9730000000000008},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":3.0009999999999999,"5min":3.0009999999999999,"15min":3.0009999999999999},"last":0.62}]}]},{"osd":0,"up_from":14,"seq":60129542170,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343884,"statfs":{"total":96636764160,"available":96608137216,"internally_reserved":0,"allocated":962560,"data_stored":732504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.62,"5min":0.62,"15min":0.62},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.495,"5min":0.495,"15min":0.495},"last":0.49199999999999999}]},{"osd":2,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"last":0.42599999999999999}]},{"osd":3,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"last":0.51900000000000002}]},{"osd":4,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.58699999999999997}]},{"osd":5,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.70299999999999996}]},{"osd":6,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.64400000000000002}]},{"osd":7,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.747}]}]},{"osd":2,"up_from":14,"seq":60129542171,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":372736,"data_stored":142206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":1.429,"5min":1.429,"15min":1.429},"last":0.443},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.48899999999999999}]},{"osd":1,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":1.5269999999999999,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.56100000000000005}]},{"osd":3,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":4.0629999999999997,"5min":4.0629999999999997,"15min":4.0629999999999997},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.381,"5min":1.381,"15min":1.381},"last":0.57799999999999996}]},{"osd":4,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.63400000000000001}]},{"osd":5,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":3.8599999999999999,"5min":3.8599999999999999,"15min":3.8599999999999999},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.464,"5min":1.464,"15min":1.464},"last":0.77500000000000002}]},{"osd":6,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":3.9820000000000002,"5min":3.9820000000000002,"15min":3.9820000000000002},"last":0.74399999999999999}]},{"osd":7,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":3.9049999999999998,"5min":3.9049999999999998,"15min":3.9049999999999998},"last":0.64700000000000002}]}]},{"osd":3,"up_from":14,"seq":60129542171,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":376,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":385024,"data_stored":143770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.307},{"interface":"front","average":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"last":0.39100000000000001}]},{"osd":1,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"last":0.36499999999999999}]},{"osd":2,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.437,"5min":0.437,"15min":0.437},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.442}]},{"osd":4,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":6.0350000000000001,"5min":6.0350000000000001,"15min":6.0350000000000001},"last":0.52300000000000002}]},{"osd":5,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":6.0540000000000003,"5min":6.0540000000000003,"15min":6.0540000000000003},"last":0.54000000000000004}]},{"osd":6,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.627}]},{"osd":7,"last update":"Sat Oct 14 10:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":6.0819999999999999,"5min":6.0819999999999999,"15min":6.0819999999999999},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.55700000000000005}]}]},{"osd":4,"up_from":14,"seq":60129542170,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":376832,"data_stored":142240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 10:36:56 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.57899999999999996}]},{"osd":1,"last update":"Sat Oct 14 10:36:56 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.53000000000000003}]},{"osd":2,"last update":"Sat Oct 14 10:36:56 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.626}]},{"osd":3,"last update":"Sat Oct 14 10:36:56 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.59299999999999997}]},{"osd":5,"last update":"Sat Oct 14 10:36:56 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.40200000000000002}]},{"osd":6,"last update":"Sat Oct 14 10:36:56 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.48299999999999998}]},{"osd":7,"last update":"Sat Oct 14 10:36:56 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.503}]}]},{"osd":5,"up_from":14,"seq":60129542171,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":376832,"data_stored":142808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.60099999999999998}]},{"osd":1,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.57299999999999995}]},{"osd":2,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.53400000000000003}]},{"osd":3,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.66100000000000003}]},{"osd":4,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.442},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.55200000000000005}]},{"osd":6,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.61499999999999999}]},{"osd":7,"last update":"Sat Oct 14 10:37:00 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.63400000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-14T10:37:56.025 INFO:tasks.ceph:pgid 5.3e last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:56.026 INFO:tasks.ceph:pgid 5.37 last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:56.026 INFO:tasks.ceph:pgid 5.31 last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:56.027 INFO:tasks.ceph:pgid 5.2e last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:56.027 INFO:tasks.ceph:pgid 5.2b last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:56.027 INFO:tasks.ceph:pgid 5.2a last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:56.028 INFO:tasks.ceph:pgid 5.26 last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:56.028 INFO:tasks.ceph:pgid 5.1b last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:56.029 INFO:tasks.ceph:pgid 5.18 last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:56.029 INFO:tasks.ceph:pgid 5.16 last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:56.029 INFO:tasks.ceph:pgid 5.17 last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:56.030 INFO:tasks.ceph:pgid 5.e last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:56.030 INFO:tasks.ceph:pgid 5.f last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:56.031 INFO:tasks.ceph:pgid 5.7 last_scrub_stamp 2023-10-14T10:36:59.041694+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=36, tm_sec=59, tm_wday=5, tm_yday=287, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=10, tm_min=37, tm_sec=9, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T10:37:56.031 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-14T10:38:09.592 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-14T10:38:09.595 DEBUG:teuthology.orchestra.run.smithi072:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-14T10:38:16.032 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T10:38:16.172 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.170+0000 7fb3c66a9700 1 Processor -- start 2023-10-14T10:38:16.172 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.170+0000 7fb3c66a9700 1 -- start start 2023-10-14T10:38:16.175 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.170+0000 7fb3c66a9700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fb3c015eab0 0x7fb3c00cf190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:38:16.176 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.171+0000 7fb3c66a9700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb3c00d1d90 0x7fb3c00cbc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:38:16.176 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.171+0000 7fb3c66a9700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3c00d2170 con 0x7fb3c00cf6d0 2023-10-14T10:38:16.176 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.171+0000 7fb3c66a9700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3c00cfb60 con 0x7fb3c00d1d90 2023-10-14T10:38:16.176 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.171+0000 7fb3c66a9700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3c015ee90 con 0x7fb3c015eab0 2023-10-14T10:38:16.177 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.171+0000 7fb3c4c46700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb3c00d1d90 0x7fb3c00cbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:38:16.177 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.171+0000 7fb3bf7fe700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fb3c00cf6d0 0x7fb3c00d1680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:54090/0 (socket says 172.21.15.67:54090) 2023-10-14T10:38:16.177 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.171+0000 7fb3c4c46700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb3c00d1d90 0x7fb3c00cbc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46326/0 (socket says 172.21.15.67:46326) 2023-10-14T10:38:16.177 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.171+0000 7fb3bf7fe700 1 -- 172.21.15.67:0/3312288121 learned_addr learned my addr 172.21.15.67:0/3312288121 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:38:16.178 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.171+0000 7fb3bffff700 1 --2- 172.21.15.67:0/3312288121 >> v2:172.21.15.72:3300/0 conn(0x7fb3c015eab0 0x7fb3c00cf190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:38:16.178 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.171+0000 7fb3c4c46700 1 -- 172.21.15.67:0/3312288121 >> v2:172.21.15.72:3300/0 conn(0x7fb3c015eab0 msgr2=0x7fb3c00cf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:38:16.178 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.171+0000 7fb3c4c46700 1 --2- 172.21.15.67:0/3312288121 >> v2:172.21.15.72:3300/0 conn(0x7fb3c015eab0 0x7fb3c00cf190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:38:16.179 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.171+0000 7fb3c4c46700 1 -- 172.21.15.67:0/3312288121 >> v1:172.21.15.67:6789/0 conn(0x7fb3c00cf6d0 legacy=0x7fb3c00d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:38:16.179 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.171+0000 7fb3c4c46700 1 -- 172.21.15.67:0/3312288121 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3c0158370 con 0x7fb3c00d1d90 2023-10-14T10:38:16.179 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.171+0000 7fb3beffd700 1 -- 172.21.15.67:0/3312288121 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4284892418 0 0) 0x7fb3b4001110 con 0x7fb3c00cf6d0 2023-10-14T10:38:16.179 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.171+0000 7fb3c4c46700 1 --2- 172.21.15.67:0/3312288121 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3c00d1d90 0x7fb3c00cbc10 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fb3c0067550 tx=0x7fb3b800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cf6d20cecd62097 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:38:16.180 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.171+0000 7fb3beffd700 1 -- 172.21.15.67:0/3312288121 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b800e070 con 0x7fb3c00d1d90 2023-10-14T10:38:16.180 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.171+0000 7fb3beffd700 1 -- 172.21.15.67:0/3312288121 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3b800bb20 con 0x7fb3c00d1d90 2023-10-14T10:38:16.180 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.171+0000 7fb3beffd700 1 -- 172.21.15.67:0/3312288121 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b8012430 con 0x7fb3c00d1d90 2023-10-14T10:38:16.180 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.172+0000 7fb3c66a9700 1 -- 172.21.15.67:0/3312288121 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3c00d1d90 msgr2=0x7fb3c00cbc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:38:16.181 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.172+0000 7fb3c66a9700 1 --2- 172.21.15.67:0/3312288121 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3c00d1d90 0x7fb3c00cbc10 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fb3c0067550 tx=0x7fb3b800b330 comp rx=0 tx=0).stop 2023-10-14T10:38:16.181 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.172+0000 7fb3c66a9700 1 -- 172.21.15.67:0/3312288121 shutdown_connections 2023-10-14T10:38:16.181 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.172+0000 7fb3c66a9700 1 --2- 172.21.15.67:0/3312288121 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3c00d1d90 0x7fb3c00cbc10 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:38:16.182 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.172+0000 7fb3c66a9700 1 --2- 172.21.15.67:0/3312288121 >> v2:172.21.15.72:3300/0 conn(0x7fb3c015eab0 0x7fb3c00cf190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:38:16.182 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.172+0000 7fb3c66a9700 1 -- 172.21.15.67:0/3312288121 >> 172.21.15.67:0/3312288121 conn(0x7fb3c00665a0 msgr2=0x7fb3c0066980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:38:16.182 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.172+0000 7fb3c66a9700 1 -- 172.21.15.67:0/3312288121 shutdown_connections 2023-10-14T10:38:16.182 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.172+0000 7fb3c66a9700 1 -- 172.21.15.67:0/3312288121 wait complete. 2023-10-14T10:38:16.183 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.173+0000 7fb3c66a9700 1 Processor -- start 2023-10-14T10:38:16.183 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.173+0000 7fb3c66a9700 1 -- start start 2023-10-14T10:38:16.186 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.174+0000 7fb3c66a9700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fb3c00cf6d0 0x7fb3c015f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:38:16.186 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.174+0000 7fb3c66a9700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3c00d1d90 0x7fb3c0155050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:38:16.186 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.174+0000 7fb3c66a9700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb3c015eab0 0x7fb3c00b61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:38:16.187 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.174+0000 7fb3c66a9700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c00b5850 con 0x7fb3c00d1d90 2023-10-14T10:38:16.187 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.174+0000 7fb3c66a9700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c00b4bd0 con 0x7fb3c015eab0 2023-10-14T10:38:16.187 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.174+0000 7fb3c66a9700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c00b4eb0 con 0x7fb3c00cf6d0 2023-10-14T10:38:16.187 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.174+0000 7fb3bf7fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3c00d1d90 0x7fb3c0155050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:38:16.188 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.174+0000 7fb3bf7fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3c00d1d90 0x7fb3c0155050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46330/0 (socket says 172.21.15.67:46330) 2023-10-14T10:38:16.188 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.174+0000 7fb3bf7fe700 1 -- 172.21.15.67:0/2871946429 learned_addr learned my addr 172.21.15.67:0/2871946429 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:38:16.188 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.175+0000 7fb3bffff700 1 --2- 172.21.15.67:0/2871946429 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fb3c00cf6d0 0x7fb3c015f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:38:16.189 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.175+0000 7fb3c4c46700 1 --2- 172.21.15.67:0/2871946429 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb3c015eab0 0x7fb3c00b61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:38:16.189 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.175+0000 7fb3bf7fe700 1 -- 172.21.15.67:0/2871946429 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fb3c00cf6d0 msgr2=0x7fb3c015f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:38:16.189 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.175+0000 7fb3bf7fe700 1 --2- 172.21.15.67:0/2871946429 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fb3c00cf6d0 0x7fb3c015f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:38:16.189 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.175+0000 7fb3bf7fe700 1 -- 172.21.15.67:0/2871946429 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb3c015eab0 msgr2=0x7fb3c00b61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:38:16.190 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.175+0000 7fb3bf7fe700 1 --2- 172.21.15.67:0/2871946429 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb3c015eab0 0x7fb3c00b61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:38:16.190 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.175+0000 7fb3bf7fe700 1 -- 172.21.15.67:0/2871946429 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3b8009a70 con 0x7fb3c00d1d90 2023-10-14T10:38:16.190 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.175+0000 7fb3bf7fe700 1 --2- 172.21.15.67:0/2871946429 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3c00d1d90 0x7fb3c0155050 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fb3b400c550 tx=0x7fb3b400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:38:16.191 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.175+0000 7fb3bcff9700 1 -- 172.21.15.67:0/2871946429 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b4004c80 con 0x7fb3c00d1d90 2023-10-14T10:38:16.191 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.175+0000 7fb3bcff9700 1 -- 172.21.15.67:0/2871946429 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3b4004e00 con 0x7fb3c00d1d90 2023-10-14T10:38:16.191 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.175+0000 7fb3c66a9700 1 -- 172.21.15.67:0/2871946429 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3c00b5190 con 0x7fb3c00d1d90 2023-10-14T10:38:16.191 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.175+0000 7fb3bcff9700 1 -- 172.21.15.67:0/2871946429 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b4007310 con 0x7fb3c00d1d90 2023-10-14T10:38:16.192 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.175+0000 7fb3c66a9700 1 -- 172.21.15.67:0/2871946429 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3c006c4b0 con 0x7fb3c00d1d90 2023-10-14T10:38:16.192 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.176+0000 7fb3c66a9700 1 -- 172.21.15.67:0/2871946429 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3c00b5990 con 0x7fb3c00d1d90 2023-10-14T10:38:16.192 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.177+0000 7fb3bcff9700 1 -- 172.21.15.67:0/2871946429 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fb3b4007470 con 0x7fb3c00d1d90 2023-10-14T10:38:16.192 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.177+0000 7fb3bcff9700 1 --2- 172.21.15.67:0/2871946429 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fb3ac06af00 0x7fb3ac06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:38:16.193 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.177+0000 7fb3bcff9700 1 -- 172.21.15.67:0/2871946429 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5729+0+0 (secure 0 0 0) 0x7fb3b4089170 con 0x7fb3c00d1d90 2023-10-14T10:38:16.193 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.177+0000 7fb3bffff700 1 --2- 172.21.15.67:0/2871946429 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fb3ac06af00 0x7fb3ac06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:38:16.193 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.178+0000 7fb3bffff700 1 --2- 172.21.15.67:0/2871946429 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fb3ac06af00 0x7fb3ac06d3c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb3b0009b70 tx=0x7fb3b0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:38:16.193 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.181+0000 7fb3bcff9700 1 -- 172.21.15.67:0/2871946429 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb3c00b5990 con 0x7fb3c00d1d90 2023-10-14T10:38:16.377 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.375+0000 7fb3c66a9700 1 -- 172.21.15.67:0/2871946429 --> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb3c00cda20 con 0x7fb3ac06af00 2023-10-14T10:38:16.386 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.384+0000 7fb3bcff9700 1 -- 172.21.15.67:0/2871946429 <== mgr.4100 v2:172.21.15.72:6832/703488382 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+291130 (secure 0 0 0) 0x7fb3b00117b0 con 0x7fb3ac06af00 2023-10-14T10:38:16.390 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-14T10:38:16.392 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.387+0000 7fb3c66a9700 1 -- 172.21.15.67:0/2871946429 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fb3ac06af00 msgr2=0x7fb3ac06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:38:16.393 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.387+0000 7fb3c66a9700 1 --2- 172.21.15.67:0/2871946429 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fb3ac06af00 0x7fb3ac06d3c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb3b0009b70 tx=0x7fb3b0008040 comp rx=0 tx=0).stop 2023-10-14T10:38:16.393 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.387+0000 7fb3c66a9700 1 -- 172.21.15.67:0/2871946429 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3c00d1d90 msgr2=0x7fb3c0155050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:38:16.393 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.387+0000 7fb3c66a9700 1 --2- 172.21.15.67:0/2871946429 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3c00d1d90 0x7fb3c0155050 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fb3b400c550 tx=0x7fb3b400c910 comp rx=0 tx=0).stop 2023-10-14T10:38:16.394 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.387+0000 7fb3c66a9700 1 -- 172.21.15.67:0/2871946429 shutdown_connections 2023-10-14T10:38:16.394 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.387+0000 7fb3c66a9700 1 --2- 172.21.15.67:0/2871946429 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7fb3ac06af00 0x7fb3ac06d3c0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:38:16.394 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.387+0000 7fb3c66a9700 1 --2- 172.21.15.67:0/2871946429 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb3c015eab0 0x7fb3c00b61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:38:16.394 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.387+0000 7fb3c66a9700 1 --2- 172.21.15.67:0/2871946429 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7fb3c00cf6d0 0x7fb3c015f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:38:16.395 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.387+0000 7fb3c66a9700 1 --2- 172.21.15.67:0/2871946429 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3c00d1d90 0x7fb3c0155050 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:38:16.395 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.387+0000 7fb3c66a9700 1 -- 172.21.15.67:0/2871946429 >> 172.21.15.67:0/2871946429 conn(0x7fb3c00665a0 msgr2=0x7fb3c00cc540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:38:16.395 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.387+0000 7fb3c66a9700 1 -- 172.21.15.67:0/2871946429 shutdown_connections 2023-10-14T10:38:16.395 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.387+0000 7fb3c66a9700 1 -- 172.21.15.67:0/2871946429 wait complete. 2023-10-14T10:38:16.396 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2023-10-14T10:38:16.402 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":96,"stamp":"2023-10-14T10:38:15.788463+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":166,"ondisk_log_size":166,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220232,"kb_used_data":4104,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754488,"statfs":{"total":773094113280,"available":772868595712,"internally_reserved":0,"allocated":4202496,"data_stored":2322644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12742,"internal_metadata":221236794},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003466"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:36.161423+0000","last_change":"2023-10-14T10:37:36.161469+0000","last_active":"2023-10-14T10:37:36.161423+0000","last_peered":"2023-10-14T10:37:36.161423+0000","last_clean":"2023-10-14T10:37:36.161423+0000","last_became_active":"2023-10-14T10:37:00.074851+0000","last_became_peered":"2023-10-14T10:37:00.074851+0000","last_unstale":"2023-10-14T10:37:36.161423+0000","last_undegraded":"2023-10-14T10:37:36.161423+0000","last_fullsized":"2023-10-14T10:37:36.161423+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:36.161309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:36.161309+0000","last_clean_scrub_stamp":"2023-10-14T10:37:36.161309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T12:29:25.518201+0000","scrub_duration":0.0015859890000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:38:01.799327+0000","last_change":"2023-10-14T10:38:01.799379+0000","last_active":"2023-10-14T10:38:01.799327+0000","last_peered":"2023-10-14T10:38:01.799327+0000","last_clean":"2023-10-14T10:38:01.799327+0000","last_became_active":"2023-10-14T10:37:00.068586+0000","last_became_peered":"2023-10-14T10:37:00.068586+0000","last_unstale":"2023-10-14T10:38:01.799327+0000","last_undegraded":"2023-10-14T10:38:01.799327+0000","last_fullsized":"2023-10-14T10:38:01.799327+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:38:01.799195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:38:01.799195+0000","last_clean_scrub_stamp":"2023-10-14T10:38:01.799195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T14:44:03.455625+0000","scrub_duration":0.001553394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:47.896163+0000","last_change":"2023-10-14T10:37:47.896230+0000","last_active":"2023-10-14T10:37:47.896163+0000","last_peered":"2023-10-14T10:37:47.896163+0000","last_clean":"2023-10-14T10:37:47.896163+0000","last_became_active":"2023-10-14T10:37:00.080476+0000","last_became_peered":"2023-10-14T10:37:00.080476+0000","last_unstale":"2023-10-14T10:37:47.896163+0000","last_undegraded":"2023-10-14T10:37:47.896163+0000","last_fullsized":"2023-10-14T10:37:47.896163+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:47.896025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:47.896025+0000","last_clean_scrub_stamp":"2023-10-14T10:37:47.896025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T14:08:35.991519+0000","scrub_duration":0.0020600129999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:36.043044+0000","last_change":"2023-10-14T10:37:36.043116+0000","last_active":"2023-10-14T10:37:36.043044+0000","last_peered":"2023-10-14T10:37:36.043044+0000","last_clean":"2023-10-14T10:37:36.043044+0000","last_became_active":"2023-10-14T10:37:00.077322+0000","last_became_peered":"2023-10-14T10:37:00.077322+0000","last_unstale":"2023-10-14T10:37:36.043044+0000","last_undegraded":"2023-10-14T10:37:36.043044+0000","last_fullsized":"2023-10-14T10:37:36.043044+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:36.042949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:36.042949+0000","last_clean_scrub_stamp":"2023-10-14T10:37:36.042949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T18:48:48.217020+0000","scrub_duration":0.001538579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:25.474346+0000","last_change":"2023-10-14T10:37:25.474436+0000","last_active":"2023-10-14T10:37:25.474346+0000","last_peered":"2023-10-14T10:37:25.474346+0000","last_clean":"2023-10-14T10:37:25.474346+0000","last_became_active":"2023-10-14T10:37:00.068055+0000","last_became_peered":"2023-10-14T10:37:00.068055+0000","last_unstale":"2023-10-14T10:37:25.474346+0000","last_undegraded":"2023-10-14T10:37:25.474346+0000","last_fullsized":"2023-10-14T10:37:25.474346+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:25.474208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:25.474208+0000","last_clean_scrub_stamp":"2023-10-14T10:37:25.474208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:57:27.388802+0000","scrub_duration":0.001660716,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:47.854873+0000","last_change":"2023-10-14T10:37:47.854929+0000","last_active":"2023-10-14T10:37:47.854873+0000","last_peered":"2023-10-14T10:37:47.854873+0000","last_clean":"2023-10-14T10:37:47.854873+0000","last_became_active":"2023-10-14T10:37:00.072345+0000","last_became_peered":"2023-10-14T10:37:00.072345+0000","last_unstale":"2023-10-14T10:37:47.854873+0000","last_undegraded":"2023-10-14T10:37:47.854873+0000","last_fullsized":"2023-10-14T10:37:47.854873+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:47.854750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:47.854750+0000","last_clean_scrub_stamp":"2023-10-14T10:37:47.854750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:08:22.220361+0000","scrub_duration":0.0020211510000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:16.975562+0000","last_change":"2023-10-14T10:37:16.975628+0000","last_active":"2023-10-14T10:37:16.975562+0000","last_peered":"2023-10-14T10:37:16.975562+0000","last_clean":"2023-10-14T10:37:16.975562+0000","last_became_active":"2023-10-14T10:37:00.072368+0000","last_became_peered":"2023-10-14T10:37:00.072368+0000","last_unstale":"2023-10-14T10:37:16.975562+0000","last_undegraded":"2023-10-14T10:37:16.975562+0000","last_fullsized":"2023-10-14T10:37:16.975562+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:16.975436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:16.975436+0000","last_clean_scrub_stamp":"2023-10-14T10:37:16.975436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:31:42.878362+0000","scrub_duration":0.0016077960000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:29.970861+0000","last_change":"2023-10-14T10:37:29.970909+0000","last_active":"2023-10-14T10:37:29.970861+0000","last_peered":"2023-10-14T10:37:29.970861+0000","last_clean":"2023-10-14T10:37:29.970861+0000","last_became_active":"2023-10-14T10:37:00.073792+0000","last_became_peered":"2023-10-14T10:37:00.073792+0000","last_unstale":"2023-10-14T10:37:29.970861+0000","last_undegraded":"2023-10-14T10:37:29.970861+0000","last_fullsized":"2023-10-14T10:37:29.970861+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:29.970754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:29.970754+0000","last_clean_scrub_stamp":"2023-10-14T10:37:29.970754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T14:08:18.066248+0000","scrub_duration":0.001847558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:55.598835+0000","last_change":"2023-10-14T10:37:55.598884+0000","last_active":"2023-10-14T10:37:55.598835+0000","last_peered":"2023-10-14T10:37:55.598835+0000","last_clean":"2023-10-14T10:37:55.598835+0000","last_became_active":"2023-10-14T10:37:00.081451+0000","last_became_peered":"2023-10-14T10:37:00.081451+0000","last_unstale":"2023-10-14T10:37:55.598835+0000","last_undegraded":"2023-10-14T10:37:55.598835+0000","last_fullsized":"2023-10-14T10:37:55.598835+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:55.598725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:55.598725+0000","last_clean_scrub_stamp":"2023-10-14T10:37:55.598725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:22:14.704497+0000","scrub_duration":0.0021681589999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:49.039728+0000","last_change":"2023-10-14T10:37:49.039775+0000","last_active":"2023-10-14T10:37:49.039728+0000","last_peered":"2023-10-14T10:37:49.039728+0000","last_clean":"2023-10-14T10:37:49.039728+0000","last_became_active":"2023-10-14T10:37:00.070667+0000","last_became_peered":"2023-10-14T10:37:00.070667+0000","last_unstale":"2023-10-14T10:37:49.039728+0000","last_undegraded":"2023-10-14T10:37:49.039728+0000","last_fullsized":"2023-10-14T10:37:49.039728+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:49.039605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:49.039605+0000","last_clean_scrub_stamp":"2023-10-14T10:37:49.039605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:57:52.753077+0000","scrub_duration":0.001714559,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:27.985170+0000","last_change":"2023-10-14T10:37:27.985252+0000","last_active":"2023-10-14T10:37:27.985170+0000","last_peered":"2023-10-14T10:37:27.985170+0000","last_clean":"2023-10-14T10:37:27.985170+0000","last_became_active":"2023-10-14T10:37:00.072639+0000","last_became_peered":"2023-10-14T10:37:00.072639+0000","last_unstale":"2023-10-14T10:37:27.985170+0000","last_undegraded":"2023-10-14T10:37:27.985170+0000","last_fullsized":"2023-10-14T10:37:27.985170+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:27.985038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:27.985038+0000","last_clean_scrub_stamp":"2023-10-14T10:37:27.985038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:26:54.065382+0000","scrub_duration":0.001768588,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:42.762513+0000","last_change":"2023-10-14T10:37:42.762560+0000","last_active":"2023-10-14T10:37:42.762513+0000","last_peered":"2023-10-14T10:37:42.762513+0000","last_clean":"2023-10-14T10:37:42.762513+0000","last_became_active":"2023-10-14T10:37:00.069373+0000","last_became_peered":"2023-10-14T10:37:00.069373+0000","last_unstale":"2023-10-14T10:37:42.762513+0000","last_undegraded":"2023-10-14T10:37:42.762513+0000","last_fullsized":"2023-10-14T10:37:42.762513+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:42.762373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:42.762373+0000","last_clean_scrub_stamp":"2023-10-14T10:37:42.762373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:27:08.842717+0000","scrub_duration":0.002121197,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:19.586632+0000","last_change":"2023-10-14T10:37:19.586687+0000","last_active":"2023-10-14T10:37:19.586632+0000","last_peered":"2023-10-14T10:37:19.586632+0000","last_clean":"2023-10-14T10:37:19.586632+0000","last_became_active":"2023-10-14T10:37:00.074473+0000","last_became_peered":"2023-10-14T10:37:00.074473+0000","last_unstale":"2023-10-14T10:37:19.586632+0000","last_undegraded":"2023-10-14T10:37:19.586632+0000","last_fullsized":"2023-10-14T10:37:19.586632+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:19.586500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:19.586500+0000","last_clean_scrub_stamp":"2023-10-14T10:37:19.586500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:57:11.433994+0000","scrub_duration":0.001596793,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:42.660367+0000","last_change":"2023-10-14T10:37:42.660417+0000","last_active":"2023-10-14T10:37:42.660367+0000","last_peered":"2023-10-14T10:37:42.660367+0000","last_clean":"2023-10-14T10:37:42.660367+0000","last_became_active":"2023-10-14T10:37:00.073651+0000","last_became_peered":"2023-10-14T10:37:00.073651+0000","last_unstale":"2023-10-14T10:37:42.660367+0000","last_undegraded":"2023-10-14T10:37:42.660367+0000","last_fullsized":"2023-10-14T10:37:42.660367+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:42.660244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:42.660244+0000","last_clean_scrub_stamp":"2023-10-14T10:37:42.660244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T21:22:40.618175+0000","scrub_duration":0.0016045129999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:38:04.869623+0000","last_change":"2023-10-14T10:38:04.869673+0000","last_active":"2023-10-14T10:38:04.869623+0000","last_peered":"2023-10-14T10:38:04.869623+0000","last_clean":"2023-10-14T10:38:04.869623+0000","last_became_active":"2023-10-14T10:37:00.071999+0000","last_became_peered":"2023-10-14T10:37:00.071999+0000","last_unstale":"2023-10-14T10:38:04.869623+0000","last_undegraded":"2023-10-14T10:38:04.869623+0000","last_fullsized":"2023-10-14T10:38:04.869623+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:38:04.869508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:38:04.869508+0000","last_clean_scrub_stamp":"2023-10-14T10:38:04.869508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T20:09:45.600292+0000","scrub_duration":0.0015255049999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:22.094919+0000","last_change":"2023-10-14T10:37:22.094971+0000","last_active":"2023-10-14T10:37:22.094919+0000","last_peered":"2023-10-14T10:37:22.094919+0000","last_clean":"2023-10-14T10:37:22.094919+0000","last_became_active":"2023-10-14T10:37:00.071509+0000","last_became_peered":"2023-10-14T10:37:00.071509+0000","last_unstale":"2023-10-14T10:37:22.094919+0000","last_undegraded":"2023-10-14T10:37:22.094919+0000","last_fullsized":"2023-10-14T10:37:22.094919+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:22.094817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:22.094817+0000","last_clean_scrub_stamp":"2023-10-14T10:37:22.094817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:57:03.789898+0000","scrub_duration":0.0017607390000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:21.067339+0000","last_change":"2023-10-14T10:37:21.067417+0000","last_active":"2023-10-14T10:37:21.067339+0000","last_peered":"2023-10-14T10:37:21.067339+0000","last_clean":"2023-10-14T10:37:21.067339+0000","last_became_active":"2023-10-14T10:37:00.071946+0000","last_became_peered":"2023-10-14T10:37:00.071946+0000","last_unstale":"2023-10-14T10:37:21.067339+0000","last_undegraded":"2023-10-14T10:37:21.067339+0000","last_fullsized":"2023-10-14T10:37:21.067339+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:21.067219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:21.067219+0000","last_clean_scrub_stamp":"2023-10-14T10:37:21.067219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T12:48:36.500809+0000","scrub_duration":0.0016934249999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:38:05.874079+0000","last_change":"2023-10-14T10:38:05.874167+0000","last_active":"2023-10-14T10:38:05.874079+0000","last_peered":"2023-10-14T10:38:05.874079+0000","last_clean":"2023-10-14T10:38:05.874079+0000","last_became_active":"2023-10-14T10:37:00.080288+0000","last_became_peered":"2023-10-14T10:37:00.080288+0000","last_unstale":"2023-10-14T10:38:05.874079+0000","last_undegraded":"2023-10-14T10:38:05.874079+0000","last_fullsized":"2023-10-14T10:38:05.874079+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:38:05.873937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:38:05.873937+0000","last_clean_scrub_stamp":"2023-10-14T10:38:05.873937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:03:25.513162+0000","scrub_duration":0.001637482,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:13.890326+0000","last_change":"2023-10-14T10:37:13.890365+0000","last_active":"2023-10-14T10:37:13.890326+0000","last_peered":"2023-10-14T10:37:13.890326+0000","last_clean":"2023-10-14T10:37:13.890326+0000","last_became_active":"2023-10-14T10:37:00.080186+0000","last_became_peered":"2023-10-14T10:37:00.080186+0000","last_unstale":"2023-10-14T10:37:13.890326+0000","last_undegraded":"2023-10-14T10:37:13.890326+0000","last_fullsized":"2023-10-14T10:37:13.890326+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:13.890234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:13.890234+0000","last_clean_scrub_stamp":"2023-10-14T10:37:13.890234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:48:50.907546+0000","scrub_duration":0.001362254,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:26.440259+0000","last_change":"2023-10-14T10:37:26.440336+0000","last_active":"2023-10-14T10:37:26.440259+0000","last_peered":"2023-10-14T10:37:26.440259+0000","last_clean":"2023-10-14T10:37:26.440259+0000","last_became_active":"2023-10-14T10:37:00.071796+0000","last_became_peered":"2023-10-14T10:37:00.071796+0000","last_unstale":"2023-10-14T10:37:26.440259+0000","last_undegraded":"2023-10-14T10:37:26.440259+0000","last_fullsized":"2023-10-14T10:37:26.440259+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:26.440115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:26.440115+0000","last_clean_scrub_stamp":"2023-10-14T10:37:26.440115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:54:37.394808+0000","scrub_duration":0.0021402970000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:38:07.534151+0000","last_change":"2023-10-14T10:38:07.534199+0000","last_active":"2023-10-14T10:38:07.534151+0000","last_peered":"2023-10-14T10:38:07.534151+0000","last_clean":"2023-10-14T10:38:07.534151+0000","last_became_active":"2023-10-14T10:37:00.080526+0000","last_became_peered":"2023-10-14T10:37:00.080526+0000","last_unstale":"2023-10-14T10:38:07.534151+0000","last_undegraded":"2023-10-14T10:38:07.534151+0000","last_fullsized":"2023-10-14T10:38:07.534151+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:38:07.534040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:38:07.534040+0000","last_clean_scrub_stamp":"2023-10-14T10:38:07.534040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:57:06.545656+0000","scrub_duration":0.0023982069999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:38:05.569323+0000","last_change":"2023-10-14T10:38:05.569389+0000","last_active":"2023-10-14T10:38:05.569323+0000","last_peered":"2023-10-14T10:38:05.569323+0000","last_clean":"2023-10-14T10:38:05.569323+0000","last_became_active":"2023-10-14T10:37:00.080062+0000","last_became_peered":"2023-10-14T10:37:00.080062+0000","last_unstale":"2023-10-14T10:38:05.569323+0000","last_undegraded":"2023-10-14T10:38:05.569323+0000","last_fullsized":"2023-10-14T10:38:05.569323+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:38:05.569199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:38:05.569199+0000","last_clean_scrub_stamp":"2023-10-14T10:38:05.569199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:13:10.096979+0000","scrub_duration":0.00195612,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:24.101006+0000","last_change":"2023-10-14T10:37:24.101056+0000","last_active":"2023-10-14T10:37:24.101006+0000","last_peered":"2023-10-14T10:37:24.101006+0000","last_clean":"2023-10-14T10:37:24.101006+0000","last_became_active":"2023-10-14T10:37:00.070897+0000","last_became_peered":"2023-10-14T10:37:00.070897+0000","last_unstale":"2023-10-14T10:37:24.101006+0000","last_undegraded":"2023-10-14T10:37:24.101006+0000","last_fullsized":"2023-10-14T10:37:24.101006+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:24.100899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:24.100899+0000","last_clean_scrub_stamp":"2023-10-14T10:37:24.100899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:58:06.157813+0000","scrub_duration":0.001908496,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:26.820065+0000","last_change":"2023-10-14T10:37:26.820141+0000","last_active":"2023-10-14T10:37:26.820065+0000","last_peered":"2023-10-14T10:37:26.820065+0000","last_clean":"2023-10-14T10:37:26.820065+0000","last_became_active":"2023-10-14T10:37:00.069246+0000","last_became_peered":"2023-10-14T10:37:00.069246+0000","last_unstale":"2023-10-14T10:37:26.820065+0000","last_undegraded":"2023-10-14T10:37:26.820065+0000","last_fullsized":"2023-10-14T10:37:26.820065+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:26.819916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:26.819916+0000","last_clean_scrub_stamp":"2023-10-14T10:37:26.819916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T21:40:50.616911+0000","scrub_duration":0.0021912780000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:47.037229+0000","last_change":"2023-10-14T10:37:47.037305+0000","last_active":"2023-10-14T10:37:47.037229+0000","last_peered":"2023-10-14T10:37:47.037229+0000","last_clean":"2023-10-14T10:37:47.037229+0000","last_became_active":"2023-10-14T10:37:00.069871+0000","last_became_peered":"2023-10-14T10:37:00.069871+0000","last_unstale":"2023-10-14T10:37:47.037229+0000","last_undegraded":"2023-10-14T10:37:47.037229+0000","last_fullsized":"2023-10-14T10:37:47.037229+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:47.037074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:47.037074+0000","last_clean_scrub_stamp":"2023-10-14T10:37:47.037074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T20:01:24.085952+0000","scrub_duration":0.001819242,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:38:08.819110+0000","last_change":"2023-10-14T10:38:08.819185+0000","last_active":"2023-10-14T10:38:08.819110+0000","last_peered":"2023-10-14T10:38:08.819110+0000","last_clean":"2023-10-14T10:38:08.819110+0000","last_became_active":"2023-10-14T10:37:00.076169+0000","last_became_peered":"2023-10-14T10:37:00.076169+0000","last_unstale":"2023-10-14T10:38:08.819110+0000","last_undegraded":"2023-10-14T10:38:08.819110+0000","last_fullsized":"2023-10-14T10:38:08.819110+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:38:08.818958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:38:08.818958+0000","last_clean_scrub_stamp":"2023-10-14T10:38:08.818958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:21:43.128457+0000","scrub_duration":0.0024136790000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:42.041674+0000","last_change":"2023-10-14T10:37:42.041756+0000","last_active":"2023-10-14T10:37:42.041674+0000","last_peered":"2023-10-14T10:37:42.041674+0000","last_clean":"2023-10-14T10:37:42.041674+0000","last_became_active":"2023-10-14T10:37:00.076263+0000","last_became_peered":"2023-10-14T10:37:00.076263+0000","last_unstale":"2023-10-14T10:37:42.041674+0000","last_undegraded":"2023-10-14T10:37:42.041674+0000","last_fullsized":"2023-10-14T10:37:42.041674+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:42.041529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:42.041529+0000","last_clean_scrub_stamp":"2023-10-14T10:37:42.041529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T14:08:30.137022+0000","scrub_duration":0.0018411619999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:30.765830+0000","last_change":"2023-10-14T10:37:30.765882+0000","last_active":"2023-10-14T10:37:30.765830+0000","last_peered":"2023-10-14T10:37:30.765830+0000","last_clean":"2023-10-14T10:37:30.765830+0000","last_became_active":"2023-10-14T10:37:00.079178+0000","last_became_peered":"2023-10-14T10:37:00.079178+0000","last_unstale":"2023-10-14T10:37:30.765830+0000","last_undegraded":"2023-10-14T10:37:30.765830+0000","last_fullsized":"2023-10-14T10:37:30.765830+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:30.765720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:30.765720+0000","last_clean_scrub_stamp":"2023-10-14T10:37:30.765720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T12:48:46.199309+0000","scrub_duration":0.0022229910000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:42.718659+0000","last_change":"2023-10-14T10:37:42.718740+0000","last_active":"2023-10-14T10:37:42.718659+0000","last_peered":"2023-10-14T10:37:42.718659+0000","last_clean":"2023-10-14T10:37:42.718659+0000","last_became_active":"2023-10-14T10:37:00.080171+0000","last_became_peered":"2023-10-14T10:37:00.080171+0000","last_unstale":"2023-10-14T10:37:42.718659+0000","last_undegraded":"2023-10-14T10:37:42.718659+0000","last_fullsized":"2023-10-14T10:37:42.718659+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:42.718545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:42.718545+0000","last_clean_scrub_stamp":"2023-10-14T10:37:42.718545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T18:34:28.005587+0000","scrub_duration":0.001795675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:30.956480+0000","last_change":"2023-10-14T10:37:30.956555+0000","last_active":"2023-10-14T10:37:30.956480+0000","last_peered":"2023-10-14T10:37:30.956480+0000","last_clean":"2023-10-14T10:37:30.956480+0000","last_became_active":"2023-10-14T10:37:00.071882+0000","last_became_peered":"2023-10-14T10:37:00.071882+0000","last_unstale":"2023-10-14T10:37:30.956480+0000","last_undegraded":"2023-10-14T10:37:30.956480+0000","last_fullsized":"2023-10-14T10:37:30.956480+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:30.956369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:30.956369+0000","last_clean_scrub_stamp":"2023-10-14T10:37:30.956369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:18:44.736053+0000","scrub_duration":0.001666013,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:35.120416+0000","last_change":"2023-10-14T10:37:35.120460+0000","last_active":"2023-10-14T10:37:35.120416+0000","last_peered":"2023-10-14T10:37:35.120416+0000","last_clean":"2023-10-14T10:37:35.120416+0000","last_became_active":"2023-10-14T10:37:00.081312+0000","last_became_peered":"2023-10-14T10:37:00.081312+0000","last_unstale":"2023-10-14T10:37:35.120416+0000","last_undegraded":"2023-10-14T10:37:35.120416+0000","last_fullsized":"2023-10-14T10:37:35.120416+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:35.120311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:35.120311+0000","last_clean_scrub_stamp":"2023-10-14T10:37:35.120311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:58:17.177225+0000","scrub_duration":0.001906561,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:15.957200+0000","last_change":"2023-10-14T10:37:15.957269+0000","last_active":"2023-10-14T10:37:15.957200+0000","last_peered":"2023-10-14T10:37:15.957200+0000","last_clean":"2023-10-14T10:37:15.957200+0000","last_became_active":"2023-10-14T10:37:00.078441+0000","last_became_peered":"2023-10-14T10:37:00.078441+0000","last_unstale":"2023-10-14T10:37:15.957200+0000","last_undegraded":"2023-10-14T10:37:15.957200+0000","last_fullsized":"2023-10-14T10:37:15.957200+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:15.957063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:15.957063+0000","last_clean_scrub_stamp":"2023-10-14T10:37:15.957063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:48:52.974375+0000","scrub_duration":0.001645093,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:25.103789+0000","last_change":"2023-10-14T10:37:25.103841+0000","last_active":"2023-10-14T10:37:25.103789+0000","last_peered":"2023-10-14T10:37:25.103789+0000","last_clean":"2023-10-14T10:37:25.103789+0000","last_became_active":"2023-10-14T10:37:00.070945+0000","last_became_peered":"2023-10-14T10:37:00.070945+0000","last_unstale":"2023-10-14T10:37:25.103789+0000","last_undegraded":"2023-10-14T10:37:25.103789+0000","last_fullsized":"2023-10-14T10:37:25.103789+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:25.103682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:25.103682+0000","last_clean_scrub_stamp":"2023-10-14T10:37:25.103682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:53:19.037516+0000","scrub_duration":0.0021103710000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":25,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:21.985058+0000","last_change":"2023-10-14T10:37:21.985135+0000","last_active":"2023-10-14T10:37:21.985058+0000","last_peered":"2023-10-14T10:37:21.985058+0000","last_clean":"2023-10-14T10:37:21.985058+0000","last_became_active":"2023-10-14T10:36:59.066488+0000","last_became_peered":"2023-10-14T10:36:59.066488+0000","last_unstale":"2023-10-14T10:37:21.985058+0000","last_undegraded":"2023-10-14T10:37:21.985058+0000","last_fullsized":"2023-10-14T10:37:21.985058+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'4","last_scrub_stamp":"2023-10-14T10:37:21.984953+0000","last_deep_scrub":"32'4","last_deep_scrub_stamp":"2023-10-14T10:37:21.984953+0000","last_clean_scrub_stamp":"2023-10-14T10:37:21.984953+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:31:39.071011+0000","scrub_duration":0.0021399549999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:31.160865+0000","last_change":"2023-10-14T10:37:31.160912+0000","last_active":"2023-10-14T10:37:31.160865+0000","last_peered":"2023-10-14T10:37:31.160865+0000","last_clean":"2023-10-14T10:37:31.160865+0000","last_became_active":"2023-10-14T10:37:00.077734+0000","last_became_peered":"2023-10-14T10:37:00.077734+0000","last_unstale":"2023-10-14T10:37:31.160865+0000","last_undegraded":"2023-10-14T10:37:31.160865+0000","last_fullsized":"2023-10-14T10:37:31.160865+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:31.160755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:31.160755+0000","last_clean_scrub_stamp":"2023-10-14T10:37:31.160755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T14:19:54.429928+0000","scrub_duration":0.0014218779999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:22.082855+0000","last_change":"2023-10-14T10:37:22.082911+0000","last_active":"2023-10-14T10:37:22.082855+0000","last_peered":"2023-10-14T10:37:22.082855+0000","last_clean":"2023-10-14T10:37:22.082855+0000","last_became_active":"2023-10-14T10:36:59.070463+0000","last_became_peered":"2023-10-14T10:36:59.070463+0000","last_unstale":"2023-10-14T10:37:22.082855+0000","last_undegraded":"2023-10-14T10:37:22.082855+0000","last_fullsized":"2023-10-14T10:37:22.082855+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1","last_scrub_stamp":"2023-10-14T10:37:22.082747+0000","last_deep_scrub":"32'1","last_deep_scrub_stamp":"2023-10-14T10:37:22.082747+0000","last_clean_scrub_stamp":"2023-10-14T10:37:22.082747+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:06:51.432100+0000","scrub_duration":0.0021442739999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:50.635047+0000","last_change":"2023-10-14T10:37:50.635125+0000","last_active":"2023-10-14T10:37:50.635047+0000","last_peered":"2023-10-14T10:37:50.635047+0000","last_clean":"2023-10-14T10:37:50.635047+0000","last_became_active":"2023-10-14T10:37:00.080160+0000","last_became_peered":"2023-10-14T10:37:00.080160+0000","last_unstale":"2023-10-14T10:37:50.635047+0000","last_undegraded":"2023-10-14T10:37:50.635047+0000","last_fullsized":"2023-10-14T10:37:50.635047+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:50.634876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:50.634876+0000","last_clean_scrub_stamp":"2023-10-14T10:37:50.634876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T18:35:52.324078+0000","scrub_duration":0.0023350100000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":23,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:29.788305+0000","last_change":"2023-10-14T10:37:29.788362+0000","last_active":"2023-10-14T10:37:29.788305+0000","last_peered":"2023-10-14T10:37:29.788305+0000","last_clean":"2023-10-14T10:37:29.788305+0000","last_became_active":"2023-10-14T10:36:59.067616+0000","last_became_peered":"2023-10-14T10:36:59.067616+0000","last_unstale":"2023-10-14T10:37:29.788305+0000","last_undegraded":"2023-10-14T10:37:29.788305+0000","last_fullsized":"2023-10-14T10:37:29.788305+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2023-10-14T10:37:29.788187+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2023-10-14T10:37:29.788187+0000","last_clean_scrub_stamp":"2023-10-14T10:37:29.788187+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:26:55.868531+0000","scrub_duration":0.0020300269999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:50.083851+0000","last_change":"2023-10-14T10:37:50.083896+0000","last_active":"2023-10-14T10:37:50.083851+0000","last_peered":"2023-10-14T10:37:50.083851+0000","last_clean":"2023-10-14T10:37:50.083851+0000","last_became_active":"2023-10-14T10:37:00.078829+0000","last_became_peered":"2023-10-14T10:37:00.078829+0000","last_unstale":"2023-10-14T10:37:50.083851+0000","last_undegraded":"2023-10-14T10:37:50.083851+0000","last_fullsized":"2023-10-14T10:37:50.083851+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:50.083751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:50.083751+0000","last_clean_scrub_stamp":"2023-10-14T10:37:50.083751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T20:09:30.814536+0000","scrub_duration":0.0019669029999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":23,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:12.636047+0000","last_change":"2023-10-14T10:37:12.636098+0000","last_active":"2023-10-14T10:37:12.636047+0000","last_peered":"2023-10-14T10:37:12.636047+0000","last_clean":"2023-10-14T10:37:12.636047+0000","last_became_active":"2023-10-14T10:36:59.065608+0000","last_became_peered":"2023-10-14T10:36:59.065608+0000","last_unstale":"2023-10-14T10:37:12.636047+0000","last_undegraded":"2023-10-14T10:37:12.636047+0000","last_fullsized":"2023-10-14T10:37:12.636047+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2023-10-14T10:37:12.635967+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2023-10-14T10:37:12.635967+0000","last_clean_scrub_stamp":"2023-10-14T10:37:12.635967+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:31:27.596001+0000","scrub_duration":0.0015686820000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:40.965920+0000","last_change":"2023-10-14T10:37:40.965968+0000","last_active":"2023-10-14T10:37:40.965920+0000","last_peered":"2023-10-14T10:37:40.965920+0000","last_clean":"2023-10-14T10:37:40.965920+0000","last_became_active":"2023-10-14T10:37:00.076840+0000","last_became_peered":"2023-10-14T10:37:00.076840+0000","last_unstale":"2023-10-14T10:37:40.965920+0000","last_undegraded":"2023-10-14T10:37:40.965920+0000","last_fullsized":"2023-10-14T10:37:40.965920+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:40.965808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:40.965808+0000","last_clean_scrub_stamp":"2023-10-14T10:37:40.965808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T21:22:38.923740+0000","scrub_duration":0.001657047,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:15.636426+0000","last_change":"2023-10-14T10:37:15.636479+0000","last_active":"2023-10-14T10:37:15.636426+0000","last_peered":"2023-10-14T10:37:15.636426+0000","last_clean":"2023-10-14T10:37:15.636426+0000","last_became_active":"2023-10-14T10:36:59.069192+0000","last_became_peered":"2023-10-14T10:36:59.069192+0000","last_unstale":"2023-10-14T10:37:15.636426+0000","last_undegraded":"2023-10-14T10:37:15.636426+0000","last_fullsized":"2023-10-14T10:37:15.636426+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1","last_scrub_stamp":"2023-10-14T10:37:15.636316+0000","last_deep_scrub":"32'1","last_deep_scrub_stamp":"2023-10-14T10:37:15.636316+0000","last_clean_scrub_stamp":"2023-10-14T10:37:15.636316+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:18:11.324774+0000","scrub_duration":0.0019544810000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:38:08.978179+0000","last_change":"2023-10-14T10:38:08.978255+0000","last_active":"2023-10-14T10:38:08.978179+0000","last_peered":"2023-10-14T10:38:08.978179+0000","last_clean":"2023-10-14T10:38:08.978179+0000","last_became_active":"2023-10-14T10:37:00.068517+0000","last_became_peered":"2023-10-14T10:37:00.068517+0000","last_unstale":"2023-10-14T10:38:08.978179+0000","last_undegraded":"2023-10-14T10:38:08.978179+0000","last_fullsized":"2023-10-14T10:38:08.978179+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:38:08.978003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:38:08.978003+0000","last_clean_scrub_stamp":"2023-10-14T10:38:08.978003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:13:13.505783+0000","scrub_duration":0.0019791380000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:41.694588+0000","last_change":"2023-10-14T10:37:41.694642+0000","last_active":"2023-10-14T10:37:41.694588+0000","last_peered":"2023-10-14T10:37:41.694588+0000","last_clean":"2023-10-14T10:37:41.694588+0000","last_became_active":"2023-10-14T10:36:59.069151+0000","last_became_peered":"2023-10-14T10:36:59.069151+0000","last_unstale":"2023-10-14T10:37:41.694588+0000","last_undegraded":"2023-10-14T10:37:41.694588+0000","last_fullsized":"2023-10-14T10:37:41.694588+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1","last_scrub_stamp":"2023-10-14T10:37:41.694476+0000","last_deep_scrub":"32'1","last_deep_scrub_stamp":"2023-10-14T10:37:41.694476+0000","last_clean_scrub_stamp":"2023-10-14T10:37:41.694476+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T18:48:53.868548+0000","scrub_duration":0.00202717,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:56.583801+0000","last_change":"2023-10-14T10:37:56.583850+0000","last_active":"2023-10-14T10:37:56.583801+0000","last_peered":"2023-10-14T10:37:56.583801+0000","last_clean":"2023-10-14T10:37:56.583801+0000","last_became_active":"2023-10-14T10:37:00.076706+0000","last_became_peered":"2023-10-14T10:37:00.076706+0000","last_unstale":"2023-10-14T10:37:56.583801+0000","last_undegraded":"2023-10-14T10:37:56.583801+0000","last_fullsized":"2023-10-14T10:37:56.583801+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:56.583687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:56.583687+0000","last_clean_scrub_stamp":"2023-10-14T10:37:56.583687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T21:22:54.541618+0000","scrub_duration":0.001505218,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:26.865191+0000","last_change":"2023-10-14T10:37:26.865278+0000","last_active":"2023-10-14T10:37:26.865191+0000","last_peered":"2023-10-14T10:37:26.865191+0000","last_clean":"2023-10-14T10:37:26.865191+0000","last_became_active":"2023-10-14T10:36:59.062075+0000","last_became_peered":"2023-10-14T10:36:59.062075+0000","last_unstale":"2023-10-14T10:37:26.865191+0000","last_undegraded":"2023-10-14T10:37:26.865191+0000","last_fullsized":"2023-10-14T10:37:26.865191+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:26.865054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:26.865054+0000","last_clean_scrub_stamp":"2023-10-14T10:37:26.865054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:38:54.815977+0000","scrub_duration":0.00159884,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:35.070204+0000","last_change":"2023-10-14T10:37:35.070257+0000","last_active":"2023-10-14T10:37:35.070204+0000","last_peered":"2023-10-14T10:37:35.070204+0000","last_clean":"2023-10-14T10:37:35.070204+0000","last_became_active":"2023-10-14T10:37:00.067187+0000","last_became_peered":"2023-10-14T10:37:00.067187+0000","last_unstale":"2023-10-14T10:37:35.070204+0000","last_undegraded":"2023-10-14T10:37:35.070204+0000","last_fullsized":"2023-10-14T10:37:35.070204+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:35.070094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:35.070094+0000","last_clean_scrub_stamp":"2023-10-14T10:37:35.070094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:17:05.211354+0000","scrub_duration":0.00188517,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:29.738486+0000","last_change":"2023-10-14T10:37:29.738579+0000","last_active":"2023-10-14T10:37:29.738486+0000","last_peered":"2023-10-14T10:37:29.738486+0000","last_clean":"2023-10-14T10:37:29.738486+0000","last_became_active":"2023-10-14T10:36:59.064990+0000","last_became_peered":"2023-10-14T10:36:59.064990+0000","last_unstale":"2023-10-14T10:37:29.738486+0000","last_undegraded":"2023-10-14T10:37:29.738486+0000","last_fullsized":"2023-10-14T10:37:29.738486+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:29.738368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:29.738368+0000","last_clean_scrub_stamp":"2023-10-14T10:37:29.738368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:27:09.238644+0000","scrub_duration":0.0016393549999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:55.826273+0000","last_change":"2023-10-14T10:37:55.826370+0000","last_active":"2023-10-14T10:37:55.826273+0000","last_peered":"2023-10-14T10:37:55.826273+0000","last_clean":"2023-10-14T10:37:55.826273+0000","last_became_active":"2023-10-14T10:37:00.077152+0000","last_became_peered":"2023-10-14T10:37:00.077152+0000","last_unstale":"2023-10-14T10:37:55.826273+0000","last_undegraded":"2023-10-14T10:37:55.826273+0000","last_fullsized":"2023-10-14T10:37:55.826273+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:55.826145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:55.826145+0000","last_clean_scrub_stamp":"2023-10-14T10:37:55.826145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T20:01:32.875022+0000","scrub_duration":0.0016613439999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:33.751083+0000","last_change":"2023-10-14T10:37:33.751175+0000","last_active":"2023-10-14T10:37:33.751083+0000","last_peered":"2023-10-14T10:37:33.751083+0000","last_clean":"2023-10-14T10:37:33.751083+0000","last_became_active":"2023-10-14T10:36:59.061597+0000","last_became_peered":"2023-10-14T10:36:59.061597+0000","last_unstale":"2023-10-14T10:37:33.751083+0000","last_undegraded":"2023-10-14T10:37:33.751083+0000","last_fullsized":"2023-10-14T10:37:33.751083+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:33.750809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:33.750809+0000","last_clean_scrub_stamp":"2023-10-14T10:37:33.750809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T14:08:21.846303+0000","scrub_duration":0.0020450709999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:38:02.821566+0000","last_change":"2023-10-14T10:38:02.821622+0000","last_active":"2023-10-14T10:38:02.821566+0000","last_peered":"2023-10-14T10:38:02.821566+0000","last_clean":"2023-10-14T10:38:02.821566+0000","last_became_active":"2023-10-14T10:37:00.075889+0000","last_became_peered":"2023-10-14T10:37:00.075889+0000","last_unstale":"2023-10-14T10:38:02.821566+0000","last_undegraded":"2023-10-14T10:38:02.821566+0000","last_fullsized":"2023-10-14T10:38:02.821566+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:38:02.821454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:38:02.821454+0000","last_clean_scrub_stamp":"2023-10-14T10:38:02.821454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:31:40.968594+0000","scrub_duration":0.001686777,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:20.017238+0000","last_change":"2023-10-14T10:37:20.017309+0000","last_active":"2023-10-14T10:37:20.017238+0000","last_peered":"2023-10-14T10:37:20.017238+0000","last_clean":"2023-10-14T10:37:20.017238+0000","last_became_active":"2023-10-14T10:36:59.069731+0000","last_became_peered":"2023-10-14T10:36:59.069731+0000","last_unstale":"2023-10-14T10:37:20.017238+0000","last_undegraded":"2023-10-14T10:37:20.017238+0000","last_fullsized":"2023-10-14T10:37:20.017238+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:20.017076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:20.017076+0000","last_clean_scrub_stamp":"2023-10-14T10:37:20.017076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T18:58:11.111987+0000","scrub_duration":0.001527189,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:41.672519+0000","last_change":"2023-10-14T10:37:41.672603+0000","last_active":"2023-10-14T10:37:41.672519+0000","last_peered":"2023-10-14T10:37:41.672519+0000","last_clean":"2023-10-14T10:37:41.672519+0000","last_became_active":"2023-10-14T10:37:00.072288+0000","last_became_peered":"2023-10-14T10:37:00.072288+0000","last_unstale":"2023-10-14T10:37:41.672519+0000","last_undegraded":"2023-10-14T10:37:41.672519+0000","last_fullsized":"2023-10-14T10:37:41.672519+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:41.672393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:41.672393+0000","last_clean_scrub_stamp":"2023-10-14T10:37:41.672393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:22:00.778165+0000","scrub_duration":0.0019683190000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":24,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:24.791215+0000","last_change":"2023-10-14T10:37:24.791349+0000","last_active":"2023-10-14T10:37:24.791215+0000","last_peered":"2023-10-14T10:37:24.791215+0000","last_clean":"2023-10-14T10:37:24.791215+0000","last_became_active":"2023-10-14T10:36:59.061700+0000","last_became_peered":"2023-10-14T10:36:59.061700+0000","last_unstale":"2023-10-14T10:37:24.791215+0000","last_undegraded":"2023-10-14T10:37:24.791215+0000","last_fullsized":"2023-10-14T10:37:24.791215+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3","last_scrub_stamp":"2023-10-14T10:37:24.791071+0000","last_deep_scrub":"32'3","last_deep_scrub_stamp":"2023-10-14T10:37:24.791071+0000","last_clean_scrub_stamp":"2023-10-14T10:37:24.791071+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:49:01.808383+0000","scrub_duration":0.0023118180000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:32.745124+0000","last_change":"2023-10-14T10:37:32.745181+0000","last_active":"2023-10-14T10:37:32.745124+0000","last_peered":"2023-10-14T10:37:32.745124+0000","last_clean":"2023-10-14T10:37:32.745124+0000","last_became_active":"2023-10-14T10:37:00.074700+0000","last_became_peered":"2023-10-14T10:37:00.074700+0000","last_unstale":"2023-10-14T10:37:32.745124+0000","last_undegraded":"2023-10-14T10:37:32.745124+0000","last_fullsized":"2023-10-14T10:37:32.745124+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:32.745006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:32.745006+0000","last_clean_scrub_stamp":"2023-10-14T10:37:32.745006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:57:14.440088+0000","scrub_duration":0.0019991470000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"33'10","reported_seq":31,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:36.684773+0000","last_change":"2023-10-14T10:37:36.684835+0000","last_active":"2023-10-14T10:37:36.684773+0000","last_peered":"2023-10-14T10:37:36.684773+0000","last_clean":"2023-10-14T10:37:36.684773+0000","last_became_active":"2023-10-14T10:36:59.069043+0000","last_became_peered":"2023-10-14T10:36:59.069043+0000","last_unstale":"2023-10-14T10:37:36.684773+0000","last_undegraded":"2023-10-14T10:37:36.684773+0000","last_fullsized":"2023-10-14T10:37:36.684773+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'10","last_scrub_stamp":"2023-10-14T10:37:36.684645+0000","last_deep_scrub":"33'10","last_deep_scrub_stamp":"2023-10-14T10:37:36.684645+0000","last_clean_scrub_stamp":"2023-10-14T10:37:36.684645+0000","objects_scrubbed":4,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:18:50.464330+0000","scrub_duration":0.005487306,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:38.036544+0000","last_change":"2023-10-14T10:37:38.036631+0000","last_active":"2023-10-14T10:37:38.036544+0000","last_peered":"2023-10-14T10:37:38.036544+0000","last_clean":"2023-10-14T10:37:38.036544+0000","last_became_active":"2023-10-14T10:37:00.065345+0000","last_became_peered":"2023-10-14T10:37:00.065345+0000","last_unstale":"2023-10-14T10:37:38.036544+0000","last_undegraded":"2023-10-14T10:37:38.036544+0000","last_fullsized":"2023-10-14T10:37:38.036544+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:38.036400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:38.036400+0000","last_clean_scrub_stamp":"2023-10-14T10:37:38.036400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:39:05.987323+0000","scrub_duration":0.002126911,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"33'4","reported_seq":25,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:30.752534+0000","last_change":"2023-10-14T10:37:30.752613+0000","last_active":"2023-10-14T10:37:30.752534+0000","last_peered":"2023-10-14T10:37:30.752534+0000","last_clean":"2023-10-14T10:37:30.752534+0000","last_became_active":"2023-10-14T10:36:59.065569+0000","last_became_peered":"2023-10-14T10:36:59.065569+0000","last_unstale":"2023-10-14T10:37:30.752534+0000","last_undegraded":"2023-10-14T10:37:30.752534+0000","last_fullsized":"2023-10-14T10:37:30.752534+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'4","last_scrub_stamp":"2023-10-14T10:37:30.752424+0000","last_deep_scrub":"33'4","last_deep_scrub_stamp":"2023-10-14T10:37:30.752424+0000","last_clean_scrub_stamp":"2023-10-14T10:37:30.752424+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T12:29:20.109316+0000","scrub_duration":0.0039809850000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:49.868165+0000","last_change":"2023-10-14T10:37:49.868216+0000","last_active":"2023-10-14T10:37:49.868165+0000","last_peered":"2023-10-14T10:37:49.868165+0000","last_clean":"2023-10-14T10:37:49.868165+0000","last_became_active":"2023-10-14T10:37:00.078844+0000","last_became_peered":"2023-10-14T10:37:00.078844+0000","last_unstale":"2023-10-14T10:37:49.868165+0000","last_undegraded":"2023-10-14T10:37:49.868165+0000","last_fullsized":"2023-10-14T10:37:49.868165+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:49.868025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:49.868025+0000","last_clean_scrub_stamp":"2023-10-14T10:37:49.868025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:34:29.629255+0000","scrub_duration":0.001667027,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:21.918946+0000","last_change":"2023-10-14T10:37:21.919039+0000","last_active":"2023-10-14T10:37:21.918946+0000","last_peered":"2023-10-14T10:37:21.918946+0000","last_clean":"2023-10-14T10:37:21.918946+0000","last_became_active":"2023-10-14T10:36:59.066952+0000","last_became_peered":"2023-10-14T10:36:59.066952+0000","last_unstale":"2023-10-14T10:37:21.918946+0000","last_undegraded":"2023-10-14T10:37:21.918946+0000","last_fullsized":"2023-10-14T10:37:21.918946+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:21.918813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:21.918813+0000","last_clean_scrub_stamp":"2023-10-14T10:37:21.918813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:53:15.852647+0000","scrub_duration":0.002202907,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:33.943228+0000","last_change":"2023-10-14T10:37:33.943309+0000","last_active":"2023-10-14T10:37:33.943228+0000","last_peered":"2023-10-14T10:37:33.943228+0000","last_clean":"2023-10-14T10:37:33.943228+0000","last_became_active":"2023-10-14T10:37:00.078353+0000","last_became_peered":"2023-10-14T10:37:00.078353+0000","last_unstale":"2023-10-14T10:37:33.943228+0000","last_undegraded":"2023-10-14T10:37:33.943228+0000","last_fullsized":"2023-10-14T10:37:33.943228+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:33.943115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:33.943115+0000","last_clean_scrub_stamp":"2023-10-14T10:37:33.943115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T20:01:10.991993+0000","scrub_duration":0.001510136,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:19.819168+0000","last_change":"2023-10-14T10:37:19.819220+0000","last_active":"2023-10-14T10:37:19.819168+0000","last_peered":"2023-10-14T10:37:19.819168+0000","last_clean":"2023-10-14T10:37:19.819168+0000","last_became_active":"2023-10-14T10:36:59.062380+0000","last_became_peered":"2023-10-14T10:36:59.062380+0000","last_unstale":"2023-10-14T10:37:19.819168+0000","last_undegraded":"2023-10-14T10:37:19.819168+0000","last_fullsized":"2023-10-14T10:37:19.819168+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:19.819047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:19.819047+0000","last_clean_scrub_stamp":"2023-10-14T10:37:19.819047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:16:49.960306+0000","scrub_duration":0.001802477,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:24.980015+0000","last_change":"2023-10-14T10:37:24.980059+0000","last_active":"2023-10-14T10:37:24.980015+0000","last_peered":"2023-10-14T10:37:24.980015+0000","last_clean":"2023-10-14T10:37:24.980015+0000","last_became_active":"2023-10-14T10:37:00.077806+0000","last_became_peered":"2023-10-14T10:37:00.077806+0000","last_unstale":"2023-10-14T10:37:24.980015+0000","last_undegraded":"2023-10-14T10:37:24.980015+0000","last_fullsized":"2023-10-14T10:37:24.980015+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:24.979906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:24.979906+0000","last_clean_scrub_stamp":"2023-10-14T10:37:24.979906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T17:28:16.344297+0000","scrub_duration":0.0015380680000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":29,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:17.814572+0000","last_change":"2023-10-14T10:37:17.814685+0000","last_active":"2023-10-14T10:37:17.814572+0000","last_peered":"2023-10-14T10:37:17.814572+0000","last_clean":"2023-10-14T10:37:17.814572+0000","last_became_active":"2023-10-14T10:36:59.067727+0000","last_became_peered":"2023-10-14T10:36:59.067727+0000","last_unstale":"2023-10-14T10:37:17.814572+0000","last_undegraded":"2023-10-14T10:37:17.814572+0000","last_fullsized":"2023-10-14T10:37:17.814572+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'8","last_scrub_stamp":"2023-10-14T10:37:17.814452+0000","last_deep_scrub":"32'8","last_deep_scrub_stamp":"2023-10-14T10:37:17.814452+0000","last_clean_scrub_stamp":"2023-10-14T10:37:17.814452+0000","objects_scrubbed":4,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:25:42.460457+0000","scrub_duration":0.002353843,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:54.603529+0000","last_change":"2023-10-14T10:37:54.603578+0000","last_active":"2023-10-14T10:37:54.603529+0000","last_peered":"2023-10-14T10:37:54.603529+0000","last_clean":"2023-10-14T10:37:54.603529+0000","last_became_active":"2023-10-14T10:37:00.076658+0000","last_became_peered":"2023-10-14T10:37:00.076658+0000","last_unstale":"2023-10-14T10:37:54.603529+0000","last_undegraded":"2023-10-14T10:37:54.603529+0000","last_fullsized":"2023-10-14T10:37:54.603529+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:54.603416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:54.603416+0000","last_clean_scrub_stamp":"2023-10-14T10:37:54.603416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:08:18.816613+0000","scrub_duration":0.001634021,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:14.880170+0000","last_change":"2023-10-14T10:37:14.880213+0000","last_active":"2023-10-14T10:37:14.880170+0000","last_peered":"2023-10-14T10:37:14.880170+0000","last_clean":"2023-10-14T10:37:14.880170+0000","last_became_active":"2023-10-14T10:36:59.071406+0000","last_became_peered":"2023-10-14T10:36:59.071406+0000","last_unstale":"2023-10-14T10:37:14.880170+0000","last_undegraded":"2023-10-14T10:37:14.880170+0000","last_fullsized":"2023-10-14T10:37:14.880170+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1","last_scrub_stamp":"2023-10-14T10:37:14.880064+0000","last_deep_scrub":"32'1","last_deep_scrub_stamp":"2023-10-14T10:37:14.880064+0000","last_clean_scrub_stamp":"2023-10-14T10:37:14.880064+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T17:09:37.404058+0000","scrub_duration":0.0017514850000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:54.779853+0000","last_change":"2023-10-14T10:37:54.779902+0000","last_active":"2023-10-14T10:37:54.779853+0000","last_peered":"2023-10-14T10:37:54.779853+0000","last_clean":"2023-10-14T10:37:54.779853+0000","last_became_active":"2023-10-14T10:37:00.079795+0000","last_became_peered":"2023-10-14T10:37:00.079795+0000","last_unstale":"2023-10-14T10:37:54.779853+0000","last_undegraded":"2023-10-14T10:37:54.779853+0000","last_fullsized":"2023-10-14T10:37:54.779853+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:54.779744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:54.779744+0000","last_clean_scrub_stamp":"2023-10-14T10:37:54.779744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T18:35:56.468946+0000","scrub_duration":0.001370343,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"33'3","reported_seq":24,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:35.680521+0000","last_change":"2023-10-14T10:37:35.680617+0000","last_active":"2023-10-14T10:37:35.680521+0000","last_peered":"2023-10-14T10:37:35.680521+0000","last_clean":"2023-10-14T10:37:35.680521+0000","last_became_active":"2023-10-14T10:36:59.064959+0000","last_became_peered":"2023-10-14T10:36:59.064959+0000","last_unstale":"2023-10-14T10:37:35.680521+0000","last_undegraded":"2023-10-14T10:37:35.680521+0000","last_fullsized":"2023-10-14T10:37:35.680521+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'3","last_scrub_stamp":"2023-10-14T10:37:35.680397+0000","last_deep_scrub":"33'3","last_deep_scrub_stamp":"2023-10-14T10:37:35.680397+0000","last_clean_scrub_stamp":"2023-10-14T10:37:35.680397+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T21:40:59.477393+0000","scrub_duration":0.0031079279999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:44.984654+0000","last_change":"2023-10-14T10:37:44.984702+0000","last_active":"2023-10-14T10:37:44.984654+0000","last_peered":"2023-10-14T10:37:44.984654+0000","last_clean":"2023-10-14T10:37:44.984654+0000","last_became_active":"2023-10-14T10:37:00.077459+0000","last_became_peered":"2023-10-14T10:37:00.077459+0000","last_unstale":"2023-10-14T10:37:44.984654+0000","last_undegraded":"2023-10-14T10:37:44.984654+0000","last_fullsized":"2023-10-14T10:37:44.984654+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:44.984535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:44.984535+0000","last_clean_scrub_stamp":"2023-10-14T10:37:44.984535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:18:58.764220+0000","scrub_duration":0.0020937130000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"33'2","reported_seq":23,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:39.719846+0000","last_change":"2023-10-14T10:37:39.719936+0000","last_active":"2023-10-14T10:37:39.719846+0000","last_peered":"2023-10-14T10:37:39.719846+0000","last_clean":"2023-10-14T10:37:39.719846+0000","last_became_active":"2023-10-14T10:36:59.058558+0000","last_became_peered":"2023-10-14T10:36:59.058558+0000","last_unstale":"2023-10-14T10:37:39.719846+0000","last_undegraded":"2023-10-14T10:37:39.719846+0000","last_fullsized":"2023-10-14T10:37:39.719846+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2023-10-14T10:37:39.719723+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2023-10-14T10:37:39.719723+0000","last_clean_scrub_stamp":"2023-10-14T10:37:39.719723+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:17:09.860982+0000","scrub_duration":0.0042980579999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:31.349037+0000","last_change":"2023-10-14T10:37:31.349116+0000","last_active":"2023-10-14T10:37:31.349037+0000","last_peered":"2023-10-14T10:37:31.349037+0000","last_clean":"2023-10-14T10:37:31.349037+0000","last_became_active":"2023-10-14T10:37:00.071167+0000","last_became_peered":"2023-10-14T10:37:00.071167+0000","last_unstale":"2023-10-14T10:37:31.349037+0000","last_undegraded":"2023-10-14T10:37:31.349037+0000","last_fullsized":"2023-10-14T10:37:31.349037+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:31.348892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:31.348892+0000","last_clean_scrub_stamp":"2023-10-14T10:37:31.348892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T17:21:44.324153+0000","scrub_duration":0.001948319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:21.671764+0000","last_change":"2023-10-14T10:37:21.671823+0000","last_active":"2023-10-14T10:37:21.671764+0000","last_peered":"2023-10-14T10:37:21.671764+0000","last_clean":"2023-10-14T10:37:21.671764+0000","last_became_active":"2023-10-14T10:36:59.069110+0000","last_became_peered":"2023-10-14T10:36:59.069110+0000","last_unstale":"2023-10-14T10:37:21.671764+0000","last_undegraded":"2023-10-14T10:37:21.671764+0000","last_fullsized":"2023-10-14T10:37:21.671764+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:21.671648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:21.671648+0000","last_clean_scrub_stamp":"2023-10-14T10:37:21.671648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T14:19:44.940820+0000","scrub_duration":0.0022981799999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:29.416731+0000","last_change":"2023-10-14T10:37:29.416782+0000","last_active":"2023-10-14T10:37:29.416731+0000","last_peered":"2023-10-14T10:37:29.416731+0000","last_clean":"2023-10-14T10:37:29.416731+0000","last_became_active":"2023-10-14T10:37:00.077175+0000","last_became_peered":"2023-10-14T10:37:00.077175+0000","last_unstale":"2023-10-14T10:37:29.416731+0000","last_undegraded":"2023-10-14T10:37:29.416731+0000","last_fullsized":"2023-10-14T10:37:29.416731+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:29.416599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:29.416599+0000","last_clean_scrub_stamp":"2023-10-14T10:37:29.416599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:22:40.667629+0000","scrub_duration":0.00185724,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":23,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:18.035307+0000","last_change":"2023-10-14T10:37:18.035397+0000","last_active":"2023-10-14T10:37:18.035307+0000","last_peered":"2023-10-14T10:37:18.035307+0000","last_clean":"2023-10-14T10:37:18.035307+0000","last_became_active":"2023-10-14T10:36:59.067345+0000","last_became_peered":"2023-10-14T10:36:59.067345+0000","last_unstale":"2023-10-14T10:37:18.035307+0000","last_undegraded":"2023-10-14T10:37:18.035307+0000","last_fullsized":"2023-10-14T10:37:18.035307+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2023-10-14T10:37:18.035173+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2023-10-14T10:37:18.035173+0000","last_clean_scrub_stamp":"2023-10-14T10:37:18.035173+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:51:19.160977+0000","scrub_duration":0.0020979919999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:20.573039+0000","last_change":"2023-10-14T10:37:20.573138+0000","last_active":"2023-10-14T10:37:20.573039+0000","last_peered":"2023-10-14T10:37:20.573039+0000","last_clean":"2023-10-14T10:37:20.573039+0000","last_became_active":"2023-10-14T10:37:00.077465+0000","last_became_peered":"2023-10-14T10:37:00.077465+0000","last_unstale":"2023-10-14T10:37:20.573039+0000","last_undegraded":"2023-10-14T10:37:20.573039+0000","last_fullsized":"2023-10-14T10:37:20.573039+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:20.572921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:20.572921+0000","last_clean_scrub_stamp":"2023-10-14T10:37:20.572921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:27:44.149277+0000","scrub_duration":0.001569429,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:16.998159+0000","last_change":"2023-10-14T10:37:16.998214+0000","last_active":"2023-10-14T10:37:16.998159+0000","last_peered":"2023-10-14T10:37:16.998159+0000","last_clean":"2023-10-14T10:37:16.998159+0000","last_became_active":"2023-10-14T10:36:59.069269+0000","last_became_peered":"2023-10-14T10:36:59.069269+0000","last_unstale":"2023-10-14T10:37:16.998159+0000","last_undegraded":"2023-10-14T10:37:16.998159+0000","last_fullsized":"2023-10-14T10:37:16.998159+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:16.998041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:16.998041+0000","last_clean_scrub_stamp":"2023-10-14T10:37:16.998041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:52:55.851068+0000","scrub_duration":0.002349617,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:22.513537+0000","last_change":"2023-10-14T10:37:22.513590+0000","last_active":"2023-10-14T10:37:22.513537+0000","last_peered":"2023-10-14T10:37:22.513537+0000","last_clean":"2023-10-14T10:37:22.513537+0000","last_became_active":"2023-10-14T10:37:00.068620+0000","last_became_peered":"2023-10-14T10:37:00.068620+0000","last_unstale":"2023-10-14T10:37:22.513537+0000","last_undegraded":"2023-10-14T10:37:22.513537+0000","last_fullsized":"2023-10-14T10:37:22.513537+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:22.513405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:22.513405+0000","last_clean_scrub_stamp":"2023-10-14T10:37:22.513405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:28:45.771382+0000","scrub_duration":0.0016531670000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:17.591859+0000","last_change":"2023-10-14T10:37:17.591924+0000","last_active":"2023-10-14T10:37:17.591859+0000","last_peered":"2023-10-14T10:37:17.591859+0000","last_clean":"2023-10-14T10:37:17.591859+0000","last_became_active":"2023-10-14T10:36:59.061678+0000","last_became_peered":"2023-10-14T10:36:59.061678+0000","last_unstale":"2023-10-14T10:37:17.591859+0000","last_undegraded":"2023-10-14T10:37:17.591859+0000","last_fullsized":"2023-10-14T10:37:17.591859+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:17.591741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:17.591741+0000","last_clean_scrub_stamp":"2023-10-14T10:37:17.591741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T14:52:36.610528+0000","scrub_duration":0.0021650459999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:27.005340+0000","last_change":"2023-10-14T10:37:27.005388+0000","last_active":"2023-10-14T10:37:27.005340+0000","last_peered":"2023-10-14T10:37:27.005340+0000","last_clean":"2023-10-14T10:37:27.005340+0000","last_became_active":"2023-10-14T10:37:00.071692+0000","last_became_peered":"2023-10-14T10:37:00.071692+0000","last_unstale":"2023-10-14T10:37:27.005340+0000","last_undegraded":"2023-10-14T10:37:27.005340+0000","last_fullsized":"2023-10-14T10:37:27.005340+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:27.005226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:27.005226+0000","last_clean_scrub_stamp":"2023-10-14T10:37:27.005226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:38:54.956149+0000","scrub_duration":0.0015000110000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:13.801179+0000","last_change":"2023-10-14T10:37:13.801233+0000","last_active":"2023-10-14T10:37:13.801179+0000","last_peered":"2023-10-14T10:37:13.801179+0000","last_clean":"2023-10-14T10:37:13.801179+0000","last_became_active":"2023-10-14T10:36:59.063332+0000","last_became_peered":"2023-10-14T10:36:59.063332+0000","last_unstale":"2023-10-14T10:37:13.801179+0000","last_undegraded":"2023-10-14T10:37:13.801179+0000","last_fullsized":"2023-10-14T10:37:13.801179+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:13.801033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:13.801033+0000","last_clean_scrub_stamp":"2023-10-14T10:37:13.801033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T14:51:03.775865+0000","scrub_duration":0.0015480699999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:24.453404+0000","last_change":"2023-10-14T10:37:24.453463+0000","last_active":"2023-10-14T10:37:24.453404+0000","last_peered":"2023-10-14T10:37:24.453404+0000","last_clean":"2023-10-14T10:37:24.453404+0000","last_became_active":"2023-10-14T10:37:00.074859+0000","last_became_peered":"2023-10-14T10:37:00.074859+0000","last_unstale":"2023-10-14T10:37:24.453404+0000","last_undegraded":"2023-10-14T10:37:24.453404+0000","last_fullsized":"2023-10-14T10:37:24.453404+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:24.453276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:24.453276+0000","last_clean_scrub_stamp":"2023-10-14T10:37:24.453276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T21:33:20.669114+0000","scrub_duration":0.002218785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":23,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:23.963239+0000","last_change":"2023-10-14T10:37:23.963301+0000","last_active":"2023-10-14T10:37:23.963239+0000","last_peered":"2023-10-14T10:37:23.963239+0000","last_clean":"2023-10-14T10:37:23.963239+0000","last_became_active":"2023-10-14T10:36:59.069037+0000","last_became_peered":"2023-10-14T10:36:59.069037+0000","last_unstale":"2023-10-14T10:37:23.963239+0000","last_undegraded":"2023-10-14T10:37:23.963239+0000","last_fullsized":"2023-10-14T10:37:23.963239+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2023-10-14T10:37:23.963112+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2023-10-14T10:37:23.963112+0000","last_clean_scrub_stamp":"2023-10-14T10:37:23.963112+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:49:00.980424+0000","scrub_duration":0.0020882309999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:38:09.965420+0000","last_change":"2023-10-14T10:38:09.965499+0000","last_active":"2023-10-14T10:38:09.965420+0000","last_peered":"2023-10-14T10:38:09.965420+0000","last_clean":"2023-10-14T10:38:09.965420+0000","last_became_active":"2023-10-14T10:37:00.075491+0000","last_became_peered":"2023-10-14T10:37:00.075491+0000","last_unstale":"2023-10-14T10:38:09.965420+0000","last_undegraded":"2023-10-14T10:38:09.965420+0000","last_fullsized":"2023-10-14T10:38:09.965420+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:38:09.965300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:38:09.965300+0000","last_clean_scrub_stamp":"2023-10-14T10:38:09.965300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:57:08.976916+0000","scrub_duration":0.0022393280000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"33'1","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:19.004875+0000","last_change":"2023-10-14T10:37:19.004932+0000","last_active":"2023-10-14T10:37:19.004875+0000","last_peered":"2023-10-14T10:37:19.004875+0000","last_clean":"2023-10-14T10:37:19.004875+0000","last_became_active":"2023-10-14T10:36:59.066146+0000","last_became_peered":"2023-10-14T10:36:59.066146+0000","last_unstale":"2023-10-14T10:37:19.004875+0000","last_undegraded":"2023-10-14T10:37:19.004875+0000","last_fullsized":"2023-10-14T10:37:19.004875+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'1","last_scrub_stamp":"2023-10-14T10:37:19.004745+0000","last_deep_scrub":"33'1","last_deep_scrub_stamp":"2023-10-14T10:37:19.004745+0000","last_clean_scrub_stamp":"2023-10-14T10:37:19.004745+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T10:48:23.006435+0000","scrub_duration":0.0020016869999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:26.115864+0000","last_change":"2023-10-14T10:37:26.115959+0000","last_active":"2023-10-14T10:37:26.115864+0000","last_peered":"2023-10-14T10:37:26.115864+0000","last_clean":"2023-10-14T10:37:26.115864+0000","last_became_active":"2023-10-14T10:37:00.082570+0000","last_became_peered":"2023-10-14T10:37:00.082570+0000","last_unstale":"2023-10-14T10:37:26.115864+0000","last_undegraded":"2023-10-14T10:37:26.115864+0000","last_fullsized":"2023-10-14T10:37:26.115864+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:26.115725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:26.115725+0000","last_clean_scrub_stamp":"2023-10-14T10:37:26.115725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T16:49:03.133037+0000","scrub_duration":0.0016919400000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:33.077799+0000","last_change":"2023-10-14T10:37:33.077851+0000","last_active":"2023-10-14T10:37:33.077799+0000","last_peered":"2023-10-14T10:37:33.077799+0000","last_clean":"2023-10-14T10:37:33.077799+0000","last_became_active":"2023-10-14T10:36:59.061642+0000","last_became_peered":"2023-10-14T10:36:59.061642+0000","last_unstale":"2023-10-14T10:37:33.077799+0000","last_undegraded":"2023-10-14T10:37:33.077799+0000","last_fullsized":"2023-10-14T10:37:33.077799+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:33.077684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:33.077684+0000","last_clean_scrub_stamp":"2023-10-14T10:37:33.077684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:25:57.723689+0000","scrub_duration":0.0016094810000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:23.976059+0000","last_change":"2023-10-14T10:37:23.976112+0000","last_active":"2023-10-14T10:37:23.976059+0000","last_peered":"2023-10-14T10:37:23.976059+0000","last_clean":"2023-10-14T10:37:23.976059+0000","last_became_active":"2023-10-14T10:37:00.071244+0000","last_became_peered":"2023-10-14T10:37:00.071244+0000","last_unstale":"2023-10-14T10:37:23.976059+0000","last_undegraded":"2023-10-14T10:37:23.976059+0000","last_fullsized":"2023-10-14T10:37:23.976059+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:23.975945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:23.975945+0000","last_clean_scrub_stamp":"2023-10-14T10:37:23.975945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T10:48:27.977635+0000","scrub_duration":0.0019053919999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:12.061873+0000","last_change":"2023-10-14T10:37:12.061941+0000","last_active":"2023-10-14T10:37:12.061873+0000","last_peered":"2023-10-14T10:37:12.061873+0000","last_clean":"2023-10-14T10:37:12.061873+0000","last_became_active":"2023-10-14T10:36:59.066592+0000","last_became_peered":"2023-10-14T10:36:59.066592+0000","last_unstale":"2023-10-14T10:37:12.061873+0000","last_undegraded":"2023-10-14T10:37:12.061873+0000","last_fullsized":"2023-10-14T10:37:12.061873+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:12.061781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:12.061781+0000","last_clean_scrub_stamp":"2023-10-14T10:37:12.061781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:52:50.914807+0000","scrub_duration":0.0016784320000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:23.082899+0000","last_change":"2023-10-14T10:37:23.082942+0000","last_active":"2023-10-14T10:37:23.082899+0000","last_peered":"2023-10-14T10:37:23.082899+0000","last_clean":"2023-10-14T10:37:23.082899+0000","last_became_active":"2023-10-14T10:37:00.075794+0000","last_became_peered":"2023-10-14T10:37:00.075794+0000","last_unstale":"2023-10-14T10:37:23.082899+0000","last_undegraded":"2023-10-14T10:37:23.082899+0000","last_fullsized":"2023-10-14T10:37:23.082899+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:23.082797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:23.082797+0000","last_clean_scrub_stamp":"2023-10-14T10:37:23.082797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T17:09:45.606791+0000","scrub_duration":0.0017434429999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:12.796373+0000","last_change":"2023-10-14T10:37:12.796416+0000","last_active":"2023-10-14T10:37:12.796373+0000","last_peered":"2023-10-14T10:37:12.796373+0000","last_clean":"2023-10-14T10:37:12.796373+0000","last_became_active":"2023-10-14T10:36:59.065163+0000","last_became_peered":"2023-10-14T10:36:59.065163+0000","last_unstale":"2023-10-14T10:37:12.796373+0000","last_undegraded":"2023-10-14T10:37:12.796373+0000","last_fullsized":"2023-10-14T10:37:12.796373+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:12.796279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:12.796279+0000","last_clean_scrub_stamp":"2023-10-14T10:37:12.796279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T19:06:42.145631+0000","scrub_duration":0.001883366,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:39.741488+0000","last_change":"2023-10-14T10:37:39.741533+0000","last_active":"2023-10-14T10:37:39.741488+0000","last_peered":"2023-10-14T10:37:39.741488+0000","last_clean":"2023-10-14T10:37:39.741488+0000","last_became_active":"2023-10-14T10:37:00.077621+0000","last_became_peered":"2023-10-14T10:37:00.077621+0000","last_unstale":"2023-10-14T10:37:39.741488+0000","last_undegraded":"2023-10-14T10:37:39.741488+0000","last_fullsized":"2023-10-14T10:37:39.741488+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:39.741387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:39.741387+0000","last_clean_scrub_stamp":"2023-10-14T10:37:39.741387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T13:08:03.954584+0000","scrub_duration":0.0017780370000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:36.714140+0000","last_change":"2023-10-14T10:37:36.714192+0000","last_active":"2023-10-14T10:37:36.714140+0000","last_peered":"2023-10-14T10:37:36.714140+0000","last_clean":"2023-10-14T10:37:36.714140+0000","last_became_active":"2023-10-14T10:36:59.064336+0000","last_became_peered":"2023-10-14T10:36:59.064336+0000","last_unstale":"2023-10-14T10:37:36.714140+0000","last_undegraded":"2023-10-14T10:37:36.714140+0000","last_fullsized":"2023-10-14T10:37:36.714140+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:36.714012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:36.714012+0000","last_clean_scrub_stamp":"2023-10-14T10:37:36.714012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:26:01.360017+0000","scrub_duration":0.0015309290000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:45.701835+0000","last_change":"2023-10-14T10:37:45.701887+0000","last_active":"2023-10-14T10:37:45.701835+0000","last_peered":"2023-10-14T10:37:45.701835+0000","last_clean":"2023-10-14T10:37:45.701835+0000","last_became_active":"2023-10-14T10:37:00.080340+0000","last_became_peered":"2023-10-14T10:37:00.080340+0000","last_unstale":"2023-10-14T10:37:45.701835+0000","last_undegraded":"2023-10-14T10:37:45.701835+0000","last_fullsized":"2023-10-14T10:37:45.701835+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:45.701714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:45.701714+0000","last_clean_scrub_stamp":"2023-10-14T10:37:45.701714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T14:08:33.797207+0000","scrub_duration":0.002227639,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:16.807142+0000","last_change":"2023-10-14T10:37:16.807232+0000","last_active":"2023-10-14T10:37:16.807142+0000","last_peered":"2023-10-14T10:37:16.807142+0000","last_clean":"2023-10-14T10:37:16.807142+0000","last_became_active":"2023-10-14T10:36:59.063130+0000","last_became_peered":"2023-10-14T10:36:59.063130+0000","last_unstale":"2023-10-14T10:37:16.807142+0000","last_undegraded":"2023-10-14T10:37:16.807142+0000","last_fullsized":"2023-10-14T10:37:16.807142+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:16.806982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:16.806982+0000","last_clean_scrub_stamp":"2023-10-14T10:37:16.806982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T21:40:40.603978+0000","scrub_duration":0.0021981460000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:28.948640+0000","last_change":"2023-10-14T10:37:28.948692+0000","last_active":"2023-10-14T10:37:28.948640+0000","last_peered":"2023-10-14T10:37:28.948640+0000","last_clean":"2023-10-14T10:37:28.948640+0000","last_became_active":"2023-10-14T10:37:00.068556+0000","last_became_peered":"2023-10-14T10:37:00.068556+0000","last_unstale":"2023-10-14T10:37:28.948640+0000","last_undegraded":"2023-10-14T10:37:28.948640+0000","last_fullsized":"2023-10-14T10:37:28.948640+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:28.948488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:28.948488+0000","last_clean_scrub_stamp":"2023-10-14T10:37:28.948488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:15:09.923968+0000","scrub_duration":0.0020176769999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:14.983447+0000","last_change":"2023-10-14T10:37:14.983500+0000","last_active":"2023-10-14T10:37:14.983447+0000","last_peered":"2023-10-14T10:37:14.983447+0000","last_clean":"2023-10-14T10:37:14.983447+0000","last_became_active":"2023-10-14T10:36:59.066424+0000","last_became_peered":"2023-10-14T10:36:59.066424+0000","last_unstale":"2023-10-14T10:37:14.983447+0000","last_undegraded":"2023-10-14T10:37:14.983447+0000","last_fullsized":"2023-10-14T10:37:14.983447+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-14T10:37:14.983339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-14T10:37:14.983339+0000","last_clean_scrub_stamp":"2023-10-14T10:37:14.983339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T22:28:20.478594+0000","scrub_duration":0.0016413619999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"33'119","reported_seq":188,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-14T10:37:14.949621+0000","last_change":"2023-10-14T10:37:14.949621+0000","last_active":"2023-10-14T10:37:14.949621+0000","last_peered":"2023-10-14T10:37:14.949621+0000","last_clean":"2023-10-14T10:37:14.949621+0000","last_became_active":"2023-10-14T10:35:59.769199+0000","last_became_peered":"2023-10-14T10:35:59.769199+0000","last_unstale":"2023-10-14T10:37:14.949621+0000","last_undegraded":"2023-10-14T10:37:14.949621+0000","last_fullsized":"2023-10-14T10:37:14.949621+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'119","last_scrub_stamp":"2023-10-14T10:37:14.949400+0000","last_deep_scrub":"33'119","last_deep_scrub_stamp":"2023-10-14T10:37:14.949400+0000","last_clean_scrub_stamp":"2023-10-14T10:37:14.949400+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T15:53:08.883234+0000","scrub_duration":0.0050812649999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":47,"ondisk_log_size":47,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542175,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27960,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343880,"statfs":{"total":96636764160,"available":96608133120,"internally_reserved":0,"allocated":966656,"data_stored":732594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 10:38:02 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.65200000000000002,"15min":0.66400000000000003},"min":{"1min":0.38500000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.73099999999999998,"5min":2.4649999999999999,"15min":2.4649999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.52600000000000002,"15min":0.52800000000000002},"min":{"1min":0.38800000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.61299999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.63800000000000001}]},{"osd":1,"last update":"Sat Oct 14 10:38:02 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.53800000000000003,"15min":0.53700000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.73599999999999999,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.68300000000000005,"15min":0.69699999999999995},"min":{"1min":0.43099999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.77000000000000002,"5min":2.4289999999999998,"15min":2.4289999999999998},"last":0.69499999999999995}]},{"osd":2,"last update":"Sat Oct 14 10:38:02 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.63500000000000001,"15min":0.63900000000000001},"min":{"1min":0.46300000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.748,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.57799999999999996,"15min":0.57599999999999996},"min":{"1min":0.376,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.75700000000000001,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.60899999999999999}]},{"osd":3,"last update":"Sat Oct 14 10:38:02 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.60799999999999998,"15min":0.61399999999999999},"min":{"1min":0.39000000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.81299999999999994,"5min":1.405,"15min":1.405},"last":0.62},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.59699999999999998,"15min":0.60499999999999998},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.71599999999999997,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.71899999999999997}]},{"osd":4,"last update":"Sat Oct 14 10:38:02 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.52300000000000002,"15min":0.53200000000000003},"min":{"1min":0.30199999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.71999999999999997,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.59699999999999998,"15min":0.60399999999999998},"min":{"1min":0.38400000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.73699999999999999,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.67400000000000004}]},{"osd":5,"last update":"Sat Oct 14 10:38:02 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.57199999999999995,"15min":0.58599999999999997},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.70699999999999996,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.58699999999999997,"15min":0.59899999999999998},"min":{"1min":0.35899999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.68400000000000005,"5min":1.016,"15min":1.016},"last":0.501}]},{"osd":6,"last update":"Sat Oct 14 10:38:02 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.58499999999999996,"15min":0.59099999999999997},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.68100000000000005,"5min":1.056,"15min":1.056},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.58199999999999996,"15min":0.59299999999999997},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.68799999999999994,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.52100000000000002}]}]},{"osd":6,"up_from":14,"seq":60129542176,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":376832,"data_stored":142752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 10:38:00 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.495,"15min":0.48699999999999999},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.83899999999999997,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.503,"15min":0.497},"min":{"1min":0.374,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.77300000000000002,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.50800000000000001}]},{"osd":1,"last update":"Sat Oct 14 10:38:00 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.499,"15min":0.48199999999999998},"min":{"1min":0.39400000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.59899999999999998,"15min":0.59599999999999997},"min":{"1min":0.40600000000000003,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.74099999999999999,"5min":1.389,"15min":1.389},"last":0.64100000000000001}]},{"osd":2,"last update":"Sat Oct 14 10:38:00 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.51600000000000001,"15min":0.50900000000000001},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.77600000000000002,"5min":0.875,"15min":0.875},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.54400000000000004,"15min":0.53700000000000003},"min":{"1min":0.44600000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.74299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.54700000000000004}]},{"osd":3,"last update":"Sat Oct 14 10:38:00 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.54200000000000004,"15min":0.53400000000000003},"min":{"1min":0.34599999999999997,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.748,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.434},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.72599999999999998,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.52800000000000002}]},{"osd":4,"last update":"Sat Oct 14 10:38:00 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.51700000000000002,"15min":0.52100000000000002},"min":{"1min":0.35799999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.64800000000000002,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.57499999999999996}]},{"osd":5,"last update":"Sat Oct 14 10:38:00 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.48499999999999999,"15min":0.48299999999999998},"min":{"1min":0.34200000000000003,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.623,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.38900000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.624}]},{"osd":7,"last update":"Sat Oct 14 10:38:00 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.56200000000000006,"15min":0.56599999999999995},"min":{"1min":0.39000000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.79200000000000004,"5min":1.024,"15min":1.024},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.53400000000000003,"15min":0.53900000000000003},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.68200000000000005,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.56499999999999995}]}]},{"osd":1,"up_from":14,"seq":60129542175,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":376,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":385024,"data_stored":143770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 10:38:03 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":1.0860000000000001,"15min":1.198},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.92500000000000004,"5min":16.855,"15min":16.855},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.443,"5min":1.2370000000000001,"15min":1.369},"min":{"1min":0.33200000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.83199999999999996,"5min":19.353000000000002,"15min":19.353000000000002},"last":0.47299999999999998}]},{"osd":2,"last update":"Sat Oct 14 10:38:03 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.44500000000000001,"15min":0.45600000000000002},"min":{"1min":0.23699999999999999,"5min":0.219,"15min":0.219},"max":{"1min":0.72899999999999998,"5min":3.0670000000000002,"15min":3.0670000000000002},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.42499999999999999,"15min":0.42099999999999999},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.44700000000000001}]},{"osd":3,"last update":"Sat Oct 14 10:38:03 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.45000000000000001,"15min":0.44600000000000001},"min":{"1min":0.30399999999999999,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.499},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.81999999999999995,"15min":0.88100000000000001},"min":{"1min":0.28399999999999997,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.755,"5min":10.005000000000001,"15min":10.005000000000001},"last":0.53300000000000003}]},{"osd":4,"last update":"Sat Oct 14 10:38:03 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.67800000000000005,"15min":0.69299999999999995},"min":{"1min":0.42299999999999999,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.96399999999999997,"5min":2.9500000000000002,"15min":2.9500000000000002},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.84999999999999998,"15min":0.88500000000000001},"min":{"1min":0.46899999999999997,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0669999999999999,"5min":5.7270000000000003,"15min":5.7270000000000003},"last":0.67400000000000004}]},{"osd":5,"last update":"Sat Oct 14 10:38:03 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":1.0329999999999999,"15min":1.097},"min":{"1min":0.498,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.046,"5min":9.8800000000000008,"15min":9.8800000000000008},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.76600000000000001,"15min":0.78400000000000003},"min":{"1min":0.46800000000000003,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.95399999999999996,"5min":3.0350000000000001,"15min":3.0350000000000001},"last":0.76200000000000001}]},{"osd":6,"last update":"Sat Oct 14 10:38:03 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.90200000000000002,"15min":0.94099999999999995},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.032,"5min":5.681,"15min":5.681},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.66800000000000004,"5min":1.0720000000000001,"15min":1.139},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.1160000000000001,"5min":9.9380000000000006,"15min":9.9380000000000006},"last":0.69699999999999995}]},{"osd":7,"last update":"Sat Oct 14 10:38:03 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":1.0589999999999999,"15min":1.123},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0880000000000001,"5min":9.9730000000000008,"15min":9.9730000000000008},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.75,"15min":0.76600000000000001},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.0069999999999999,"5min":3.0009999999999999,"15min":3.0009999999999999},"last":0.78900000000000003}]}]},{"osd":0,"up_from":14,"seq":60129542174,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343884,"statfs":{"total":96636764160,"available":96608137216,"internally_reserved":0,"allocated":962560,"data_stored":732504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 10:37:59 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.375,"15min":0.36399999999999999},"min":{"1min":0.34200000000000003,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.59099999999999997,"5min":0.62,"15min":0.62},"last":0.373},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.39000000000000001,"15min":0.376},"min":{"1min":0.28000000000000003,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"last":0.49299999999999999}]},{"osd":2,"last update":"Sat Oct 14 10:37:59 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.36199999999999999,"15min":0.35199999999999998},"min":{"1min":0.28799999999999998,"5min":0.25,"15min":0.25},"max":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.44},{"interface":"front","average":{"1min":0.437,"5min":0.373,"15min":0.36199999999999999},"min":{"1min":0.28399999999999997,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.39800000000000002}]},{"osd":3,"last update":"Sat Oct 14 10:37:59 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.35899999999999999,"15min":0.34499999999999997},"min":{"1min":0.315,"5min":0.253,"15min":0.253},"max":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.497,"5min":0.42099999999999999,"15min":0.40799999999999997},"min":{"1min":0.32400000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.53300000000000003}]},{"osd":4,"last update":"Sat Oct 14 10:37:59 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.56499999999999995,"15min":0.55100000000000005},"min":{"1min":0.46000000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.56599999999999995,"15min":0.55400000000000005},"min":{"1min":0.439,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.77200000000000002,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.66700000000000004}]},{"osd":5,"last update":"Sat Oct 14 10:37:59 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.59199999999999997,"15min":0.57599999999999996},"min":{"1min":0.55100000000000005,"5min":0.433,"15min":0.433},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.58499999999999996,"15min":0.57099999999999995},"min":{"1min":0.42199999999999999,"5min":0.377,"15min":0.377},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.57299999999999995}]},{"osd":6,"last update":"Sat Oct 14 10:37:59 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.57899999999999996,"15min":0.56799999999999995},"min":{"1min":0.48099999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.61799999999999999,"15min":0.60399999999999998},"min":{"1min":0.47399999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.68600000000000005}]},{"osd":7,"last update":"Sat Oct 14 10:37:59 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.57799999999999996,"15min":0.56100000000000005},"min":{"1min":0.52600000000000002,"5min":0.37,"15min":0.37},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.61299999999999999,"15min":0.59799999999999998},"min":{"1min":0.46100000000000002,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.88700000000000001,"5min":1.081,"15min":1.081},"last":0.71699999999999997}]}]},{"osd":2,"up_from":14,"seq":60129542175,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":372736,"data_stored":142206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 10:38:00 2023","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.38400000000000001,"15min":0.39000000000000001},"min":{"1min":0.246,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.443,"5min":1.429,"15min":1.429},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.371,"5min":0.36499999999999999,"15min":0.36399999999999999},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.498,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.42499999999999999}]},{"osd":1,"last update":"Sat Oct 14 10:38:00 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.47199999999999998,"15min":0.48199999999999998},"min":{"1min":0.28899999999999998,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.59199999999999997,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.45300000000000001,"15min":0.45400000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.59599999999999997,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.52100000000000002}]},{"osd":3,"last update":"Sat Oct 14 10:38:00 2023","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.53200000000000003,"15min":0.55600000000000005},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.52600000000000002,"5min":4.0629999999999997,"15min":4.0629999999999997},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.46600000000000003,"15min":0.46800000000000003},"min":{"1min":0.28799999999999998,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.57799999999999996,"5min":1.381,"15min":1.381},"last":0.55200000000000005}]},{"osd":4,"last update":"Sat Oct 14 10:38:00 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.61499999999999999,"15min":0.61299999999999999},"min":{"1min":0.40500000000000003,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.79600000000000004,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.60999999999999999,"15min":0.60799999999999998},"min":{"1min":0.41299999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.94799999999999995,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.872}]},{"osd":5,"last update":"Sat Oct 14 10:38:00 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.746,"15min":0.76400000000000001},"min":{"1min":0.39300000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.85799999999999998,"5min":3.8599999999999999,"15min":3.8599999999999999},"last":1.0009999999999999},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.65700000000000003,"15min":0.65600000000000003},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.88700000000000001,"5min":1.464,"15min":1.464},"last":0.82599999999999996}]},{"osd":6,"last update":"Sat Oct 14 10:38:00 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.628,"15min":0.626},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.90000000000000002,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.77100000000000002,"15min":0.79200000000000004},"min":{"1min":0.45600000000000002,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.84499999999999997,"5min":3.9820000000000002,"15min":3.9820000000000002},"last":0.95299999999999996}]},{"osd":7,"last update":"Sat Oct 14 10:38:00 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.64200000000000002,"15min":0.63800000000000001},"min":{"1min":0.51000000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.872,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.80900000000000005,"15min":0.83399999999999996},"min":{"1min":0.5,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.90200000000000002,"5min":3.9049999999999998,"15min":3.9049999999999998},"last":0.89700000000000002}]}]},{"osd":3,"up_from":14,"seq":60129542175,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":376,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":385024,"data_stored":143770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 10:37:59 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.38600000000000001,"15min":0.38500000000000001},"min":{"1min":0.24199999999999999,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.70199999999999996,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.35699999999999998,"15min":0.34899999999999998},"min":{"1min":0.26800000000000002,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.39400000000000002}]},{"osd":1,"last update":"Sat Oct 14 10:37:59 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.36399999999999999,"15min":0.35999999999999999},"min":{"1min":0.251,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.56000000000000005,"5min":0.58399999999999996,"15min":0.58399999999999996},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.434,"5min":0.40300000000000002,"15min":0.39800000000000002},"min":{"1min":0.33000000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"last":0.45400000000000001}]},{"osd":2,"last update":"Sat Oct 14 10:37:59 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.39900000000000002,"15min":0.38900000000000001},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.442,"15min":0.438},"min":{"1min":0.34000000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.66200000000000003,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.56599999999999995}]},{"osd":4,"last update":"Sat Oct 14 10:37:59 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.58699999999999997,"15min":0.58099999999999996},"min":{"1min":0.46000000000000002,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.82899999999999996,"15min":0.86399999999999999},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.84699999999999998,"5min":6.0350000000000001,"15min":6.0350000000000001},"last":0.66600000000000004}]},{"osd":5,"last update":"Sat Oct 14 10:37:59 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.59999999999999998,"15min":0.59099999999999997},"min":{"1min":0.46200000000000002,"5min":0.437,"15min":0.437},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.85899999999999999,"15min":0.89400000000000002},"min":{"1min":0.44600000000000001,"5min":0.44,"15min":0.44},"max":{"1min":1.0289999999999999,"5min":6.0540000000000003,"15min":6.0540000000000003},"last":0.76600000000000001}]},{"osd":6,"last update":"Sat Oct 14 10:37:59 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.60299999999999998,"15min":0.59599999999999997},"min":{"1min":0.47599999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.61899999999999999,"15min":0.60899999999999999},"min":{"1min":0.51800000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.70599999999999996}]},{"osd":7,"last update":"Sat Oct 14 10:37:59 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.878,"15min":0.91200000000000003},"min":{"1min":0.48999999999999999,"5min":0.375,"15min":0.375},"max":{"1min":1.0760000000000001,"5min":6.0819999999999999,"15min":6.0819999999999999},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.61599999999999999,"15min":0.60899999999999999},"min":{"1min":0.45100000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.68700000000000006}]}]},{"osd":4,"up_from":14,"seq":60129542174,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":376832,"data_stored":142240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 10:37:57 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.48499999999999999,"15min":0.46999999999999997},"min":{"1min":0.375,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.49399999999999999,"15min":0.48099999999999998},"min":{"1min":0.33800000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.50800000000000001}]},{"osd":1,"last update":"Sat Oct 14 10:37:57 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.51000000000000001,"15min":0.49299999999999999},"min":{"1min":0.40600000000000003,"5min":0.375,"15min":0.375},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.433},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.54300000000000004,"15min":0.53800000000000003},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.63300000000000001}]},{"osd":2,"last update":"Sat Oct 14 10:37:57 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.52400000000000002,"15min":0.51400000000000001},"min":{"1min":0.44800000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.57999999999999996,"15min":0.57399999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.61599999999999999}]},{"osd":3,"last update":"Sat Oct 14 10:37:57 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.55000000000000004,"15min":0.54400000000000004},"min":{"1min":0.45800000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.71699999999999997,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.59899999999999998,"15min":0.59299999999999997},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.66800000000000004}]},{"osd":5,"last update":"Sat Oct 14 10:37:57 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.56100000000000005,"15min":0.57899999999999996},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.72399999999999998,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.57399999999999995,"15min":0.58199999999999996},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.73999999999999999,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.52900000000000003}]},{"osd":6,"last update":"Sat Oct 14 10:37:57 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.58899999999999997,"15min":0.60599999999999998},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.73999999999999999,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.65900000000000003,"15min":0.67100000000000004},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.84099999999999997,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.72499999999999998}]},{"osd":7,"last update":"Sat Oct 14 10:37:57 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.621,"15min":0.63100000000000001},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.81799999999999995,"5min":0.872,"15min":0.872},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.624,"15min":0.63200000000000001},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.75800000000000001,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.68999999999999995}]}]},{"osd":5,"up_from":14,"seq":60129542175,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":376832,"data_stored":142808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 10:38:01 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.52300000000000002,"15min":0.52000000000000002},"min":{"1min":0.42099999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.67200000000000004,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.32500000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.75,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.61599999999999999}]},{"osd":1,"last update":"Sat Oct 14 10:38:01 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.57299999999999995,"15min":0.57099999999999995},"min":{"1min":0.47899999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.53900000000000003,"15min":0.52900000000000003},"min":{"1min":0.45700000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.78600000000000003,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.55500000000000005}]},{"osd":2,"last update":"Sat Oct 14 10:38:01 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.54700000000000004,"15min":0.53900000000000003},"min":{"1min":0.433,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.58299999999999996,"15min":0.57999999999999996},"min":{"1min":0.441,"5min":0.312,"15min":0.312},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.58099999999999996}]},{"osd":3,"last update":"Sat Oct 14 10:38:01 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.57499999999999996,"15min":0.57699999999999996},"min":{"1min":0.434,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.68200000000000005,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.63,"5min":0.64900000000000002,"15min":0.65200000000000002},"min":{"1min":0.51600000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.76500000000000001,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.70499999999999996}]},{"osd":4,"last update":"Sat Oct 14 10:38:01 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.495,"15min":0.504},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.57999999999999996,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.501,"5min":0.58999999999999997,"15min":0.60499999999999998},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.72699999999999998,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.44}]},{"osd":6,"last update":"Sat Oct 14 10:38:01 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.56999999999999995,"15min":0.58299999999999996},"min":{"1min":0.33500000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.74099999999999999,"5min":1.008,"15min":1.008},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.61799999999999999,"15min":0.628},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.70499999999999996,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.47899999999999998}]},{"osd":7,"last update":"Sat Oct 14 10:38:01 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.624,"15min":0.63900000000000001},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.71699999999999997,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.504},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.64100000000000001,"15min":0.65700000000000003},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.745,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.51600000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-14T10:38:16.453 DEBUG:teuthology.orchestra.run.smithi067:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-10-14T10:38:16.577 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.575+0000 7f63682ff700 1 Processor -- start 2023-10-14T10:38:16.578 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.575+0000 7f63682ff700 1 -- start start 2023-10-14T10:38:16.580 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.576+0000 7f63682ff700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f636015eab0 0x7f63600cf190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:38:16.580 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.576+0000 7f63682ff700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f63600d1d90 0x7f63600cbc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:38:16.581 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.576+0000 7f63682ff700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f636015ee90 con 0x7f63600cf6d0 2023-10-14T10:38:16.581 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.576+0000 7f63682ff700 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63600cfb50 con 0x7f63600d1d90 2023-10-14T10:38:16.581 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.576+0000 7f63682ff700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6360158370 con 0x7f636015eab0 2023-10-14T10:38:16.581 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.576+0000 7f636689c700 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f63600d1d90 0x7f63600cbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:38:16.582 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.576+0000 7f636589a700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f63600cf6d0 0x7f63600d1680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:54106/0 (socket says 172.21.15.67:54106) 2023-10-14T10:38:16.582 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.576+0000 7f636589a700 1 -- 172.21.15.67:0/2486816995 learned_addr learned my addr 172.21.15.67:0/2486816995 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:38:16.582 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.577+0000 7f636609b700 1 --2- 172.21.15.67:0/2486816995 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f636015eab0 0x7f63600cf190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:38:16.583 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.577+0000 7f636689c700 1 -- 172.21.15.67:0/2486816995 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f636015eab0 msgr2=0x7f63600cf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:38:16.583 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.577+0000 7f636689c700 1 --2- 172.21.15.67:0/2486816995 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f636015eab0 0x7f63600cf190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:38:16.583 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.577+0000 7f636689c700 1 -- 172.21.15.67:0/2486816995 >> v1:172.21.15.67:6789/0 conn(0x7f63600cf6d0 legacy=0x7f63600d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:38:16.583 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.577+0000 7f636689c700 1 -- 172.21.15.67:0/2486816995 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63600d23b0 con 0x7f63600d1d90 2023-10-14T10:38:16.584 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.577+0000 7f6365099700 1 -- 172.21.15.67:0/2486816995 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 577262248 0 0) 0x7f636015ee90 con 0x7f63600cf6d0 2023-10-14T10:38:16.584 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.577+0000 7f636689c700 1 --2- 172.21.15.67:0/2486816995 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63600d1d90 0x7f63600cbc10 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f636015f020 tx=0x7f635800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b3311937ec3f9e0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:38:16.584 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.577+0000 7f6365099700 1 -- 172.21.15.67:0/2486816995 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6358016070 con 0x7f63600d1d90 2023-10-14T10:38:16.584 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.577+0000 7f6365099700 1 -- 172.21.15.67:0/2486816995 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f635800bb20 con 0x7f63600d1d90 2023-10-14T10:38:16.585 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.577+0000 7f6365099700 1 -- 172.21.15.67:0/2486816995 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635800e070 con 0x7f63600d1d90 2023-10-14T10:38:16.585 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.578+0000 7f63682ff700 1 -- 172.21.15.67:0/2486816995 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63600d1d90 msgr2=0x7f63600cbc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:38:16.585 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.578+0000 7f63682ff700 1 --2- 172.21.15.67:0/2486816995 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63600d1d90 0x7f63600cbc10 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f636015f020 tx=0x7f635800b330 comp rx=0 tx=0).stop 2023-10-14T10:38:16.585 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.578+0000 7f63682ff700 1 -- 172.21.15.67:0/2486816995 shutdown_connections 2023-10-14T10:38:16.586 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.578+0000 7f63682ff700 1 --2- 172.21.15.67:0/2486816995 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63600d1d90 0x7f63600cbc10 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:38:16.586 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.578+0000 7f63682ff700 1 --2- 172.21.15.67:0/2486816995 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f636015eab0 0x7f63600cf190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:38:16.586 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.578+0000 7f63682ff700 1 -- 172.21.15.67:0/2486816995 >> 172.21.15.67:0/2486816995 conn(0x7f63600665a0 msgr2=0x7f6360066980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:38:16.587 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.578+0000 7f63682ff700 1 -- 172.21.15.67:0/2486816995 shutdown_connections 2023-10-14T10:38:16.587 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.578+0000 7f63682ff700 1 -- 172.21.15.67:0/2486816995 wait complete. 2023-10-14T10:38:16.590 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.579+0000 7f63682ff700 1 Processor -- start 2023-10-14T10:38:16.590 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.579+0000 7f63682ff700 1 -- start start 2023-10-14T10:38:16.590 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.580+0000 7f63682ff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63600cf6d0 0x7f6360160ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:38:16.591 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.580+0000 7f63682ff700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f63600d1d90 0x7f6360156960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:38:16.591 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.580+0000 7f63682ff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f636015eab0 0x7f63600c68d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:38:16.591 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.580+0000 7f63682ff700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6360161220 con 0x7f63600cf6d0 2023-10-14T10:38:16.591 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.580+0000 7f63682ff700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63600b6b00 con 0x7f636015eab0 2023-10-14T10:38:16.592 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.580+0000 7f63682ff700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63600b6de0 con 0x7f63600d1d90 2023-10-14T10:38:16.592 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.580+0000 7f636609b700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63600cf6d0 0x7f6360160ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:38:16.592 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.580+0000 7f636609b700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63600cf6d0 0x7f6360160ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46342/0 (socket says 172.21.15.67:46342) 2023-10-14T10:38:16.593 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.580+0000 7f636609b700 1 -- 172.21.15.67:0/3130649089 learned_addr learned my addr 172.21.15.67:0/3130649089 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:38:16.593 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.580+0000 7f636589a700 1 --2- 172.21.15.67:0/3130649089 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f63600d1d90 0x7f6360156960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:38:16.593 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.580+0000 7f636689c700 1 --2- 172.21.15.67:0/3130649089 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f636015eab0 0x7f63600c68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:38:16.593 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.580+0000 7f636609b700 1 -- 172.21.15.67:0/3130649089 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f63600d1d90 msgr2=0x7f6360156960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:38:16.594 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.580+0000 7f636609b700 1 --2- 172.21.15.67:0/3130649089 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f63600d1d90 0x7f6360156960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:38:16.594 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.580+0000 7f636609b700 1 -- 172.21.15.67:0/3130649089 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f636015eab0 msgr2=0x7f63600c68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:38:16.594 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.580+0000 7f636609b700 1 --2- 172.21.15.67:0/3130649089 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f636015eab0 0x7f63600c68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:38:16.595 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.580+0000 7f636609b700 1 -- 172.21.15.67:0/3130649089 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6358009a70 con 0x7f63600cf6d0 2023-10-14T10:38:16.595 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.580+0000 7f636589a700 1 --2- 172.21.15.67:0/3130649089 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f63600d1d90 0x7f6360156960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-14T10:38:16.595 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.580+0000 7f636609b700 1 --2- 172.21.15.67:0/3130649089 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63600cf6d0 0x7f6360160ce0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f635000e850 tx=0x7f635000eb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:38:16.595 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.581+0000 7f6356ffd700 1 -- 172.21.15.67:0/3130649089 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6350013680 con 0x7f63600cf6d0 2023-10-14T10:38:16.596 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.581+0000 7f6356ffd700 1 -- 172.21.15.67:0/3130649089 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6350013800 con 0x7f63600cf6d0 2023-10-14T10:38:16.596 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.581+0000 7f6356ffd700 1 -- 172.21.15.67:0/3130649089 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6350013bb0 con 0x7f63600cf6d0 2023-10-14T10:38:16.596 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.581+0000 7f63682ff700 1 -- 172.21.15.67:0/3130649089 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63600b47a0 con 0x7f63600cf6d0 2023-10-14T10:38:16.596 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.581+0000 7f63682ff700 1 -- 172.21.15.67:0/3130649089 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6360156ea0 con 0x7f63600cf6d0 2023-10-14T10:38:16.597 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.582+0000 7f63682ff700 1 -- 172.21.15.67:0/3130649089 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63600b4cc0 con 0x7f63600cf6d0 2023-10-14T10:38:16.597 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.582+0000 7f6356ffd700 1 -- 172.21.15.67:0/3130649089 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f6350013d10 con 0x7f63600cf6d0 2023-10-14T10:38:16.597 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.582+0000 7f6356ffd700 1 --2- 172.21.15.67:0/3130649089 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f634406af00 0x7f634406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:38:16.597 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.582+0000 7f6356ffd700 1 -- 172.21.15.67:0/3130649089 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5729+0+0 (secure 0 0 0) 0x7f6350089450 con 0x7f63600cf6d0 2023-10-14T10:38:16.598 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.583+0000 7f636589a700 1 --2- 172.21.15.67:0/3130649089 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f634406af00 0x7f634406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:38:16.598 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.583+0000 7f636589a700 1 --2- 172.21.15.67:0/3130649089 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f634406af00 0x7f634406d3c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f63600b5b20 tx=0x7f635c007ca0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:38:16.598 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.587+0000 7f6356ffd700 1 -- 172.21.15.67:0/3130649089 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f63600b4cc0 con 0x7f63600cf6d0 2023-10-14T10:38:16.789 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.787+0000 7f63682ff700 1 -- 172.21.15.67:0/3130649089 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f63600b4cc0 con 0x7f63600cf6d0 2023-10-14T10:38:16.792 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.790+0000 7f6356ffd700 1 -- 172.21.15.67:0/3130649089 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v11)=0 v11) v1 ==== 123+0+0 (secure 0 0 0) 0x7f63600b4cc0 con 0x7f63600cf6d0 2023-10-14T10:38:16.793 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.791+0000 7f6356ffd700 1 -- 172.21.15.67:0/3130649089 <== mon.0 v2:172.21.15.67:3300/0 8 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f6350054570 con 0x7f63600cf6d0 2023-10-14T10:38:16.795 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.792+0000 7f63682ff700 1 -- 172.21.15.67:0/3130649089 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f634406af00 msgr2=0x7f634406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:38:16.795 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.792+0000 7f63682ff700 1 --2- 172.21.15.67:0/3130649089 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f634406af00 0x7f634406d3c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f63600b5b20 tx=0x7f635c007ca0 comp rx=0 tx=0).stop 2023-10-14T10:38:16.795 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.792+0000 7f63682ff700 1 -- 172.21.15.67:0/3130649089 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63600cf6d0 msgr2=0x7f6360160ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:38:16.796 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.792+0000 7f63682ff700 1 --2- 172.21.15.67:0/3130649089 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63600cf6d0 0x7f6360160ce0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f635000e850 tx=0x7f635000eb60 comp rx=0 tx=0).stop 2023-10-14T10:38:16.796 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.792+0000 7f63682ff700 1 -- 172.21.15.67:0/3130649089 shutdown_connections 2023-10-14T10:38:16.796 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.792+0000 7f63682ff700 1 --2- 172.21.15.67:0/3130649089 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f634406af00 0x7f634406d3c0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:38:16.797 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.792+0000 7f63682ff700 1 --2- 172.21.15.67:0/3130649089 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f636015eab0 0x7f63600c68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:38:16.797 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.792+0000 7f63682ff700 1 --2- 172.21.15.67:0/3130649089 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63600cf6d0 0x7f6360160ce0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:38:16.797 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.792+0000 7f63682ff700 1 --2- 172.21.15.67:0/3130649089 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f63600d1d90 0x7f6360156960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:38:16.798 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.792+0000 7f63682ff700 1 -- 172.21.15.67:0/3130649089 >> 172.21.15.67:0/3130649089 conn(0x7f63600665a0 msgr2=0x7f636006d240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:38:16.798 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.793+0000 7f63682ff700 1 -- 172.21.15.67:0/3130649089 shutdown_connections 2023-10-14T10:38:16.798 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.793+0000 7f63682ff700 1 -- 172.21.15.67:0/3130649089 wait complete. 2023-10-14T10:38:16.807 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-14T10:38:16.807 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-14T10:38:16.933 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.931+0000 7f4573544700 1 Processor -- start 2023-10-14T10:38:16.934 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.931+0000 7f4573544700 1 -- start start 2023-10-14T10:38:16.939 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.932+0000 7f4573544700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f456c0cd950 0x7f456c15e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:38:16.940 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.932+0000 7f4573544700 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f456c0cf290 con 0x7f456c0cede0 2023-10-14T10:38:16.940 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.932+0000 7f4573544700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f456c0b2d60 con 0x7f456c06a840 2023-10-14T10:38:16.940 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.932+0000 7f4573544700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f456c1619a0 con 0x7f456c0cd950 2023-10-14T10:38:16.940 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.932+0000 7f4570adf700 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f456c0cede0 0x7f456c15f570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:54118/0 (socket says 172.21.15.67:54118) 2023-10-14T10:38:16.941 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.932+0000 7f4570adf700 1 -- 172.21.15.67:0/2016716973 learned_addr learned my addr 172.21.15.67:0/2016716973 (peer_addr_for_me v1:172.21.15.67:0/0) 2023-10-14T10:38:16.941 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.932+0000 7f45712e0700 1 --2- 172.21.15.67:0/2016716973 >> v2:172.21.15.72:3300/0 conn(0x7f456c0cd950 0x7f456c15e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:38:16.941 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.932+0000 7f4563fff700 1 -- 172.21.15.67:0/2016716973 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2171515762 0 0) 0x7f456c0cf290 con 0x7f456c0cede0 2023-10-14T10:38:16.942 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.932+0000 7f4563fff700 1 -- 172.21.15.67:0/2016716973 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45580035b0 con 0x7f456c0cede0 2023-10-14T10:38:16.942 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.933+0000 7f4563fff700 1 -- 172.21.15.67:0/2016716973 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 899171380 0 0) 0x7f45580035b0 con 0x7f456c0cede0 2023-10-14T10:38:16.942 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.933+0000 7f4563fff700 1 -- 172.21.15.67:0/2016716973 >> v2:172.21.15.72:3300/0 conn(0x7f456c0cd950 msgr2=0x7f456c15e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:38:16.942 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.933+0000 7f4563fff700 1 --2- 172.21.15.67:0/2016716973 >> v2:172.21.15.72:3300/0 conn(0x7f456c0cd950 0x7f456c15e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:38:16.943 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.933+0000 7f4563fff700 1 -- 172.21.15.67:0/2016716973 >> v1:172.21.15.72:6789/0 conn(0x7f456c06a840 legacy=0x7f456c161df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:38:16.943 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.933+0000 7f4563fff700 1 -- 172.21.15.67:0/2016716973 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f456c0cdd30 con 0x7f456c0cede0 2023-10-14T10:38:16.943 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.933+0000 7f4563fff700 1 -- 172.21.15.67:0/2016716973 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3180043755 0 0) 0x7f45680017f0 con 0x7f456c0cede0 2023-10-14T10:38:16.943 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.933+0000 7f4563fff700 1 -- 172.21.15.67:0/2016716973 >> v1:172.21.15.67:6789/0 conn(0x7f456c0cede0 legacy=0x7f456c15f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:38:16.944 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.933+0000 7f4563fff700 1 --2- 172.21.15.67:0/2016716973 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f4558003a00 0x7f4558003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:38:16.944 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.933+0000 7f4563fff700 1 --2- 172.21.15.67:0/2016716973 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4558004400 0x7f45580048d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:38:16.944 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.933+0000 7f4563fff700 1 --2- 172.21.15.67:0/2016716973 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f45580050d0 0x7f455800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:38:16.945 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.933+0000 7f4563fff700 1 -- 172.21.15.67:0/2016716973 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f456c1619a0 con 0x7f45580050d0 2023-10-14T10:38:16.945 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.933+0000 7f4563fff700 1 -- 172.21.15.67:0/2016716973 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f455800c030 con 0x7f4558004400 2023-10-14T10:38:16.945 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.933+0000 7f4563fff700 1 -- 172.21.15.67:0/2016716973 --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f455800c1f0 con 0x7f4558003a00 2023-10-14T10:38:16.945 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.933+0000 7f4570adf700 1 --2- 172.21.15.67:0/2016716973 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4558004400 0x7f45580048d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:38:16.946 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.933+0000 7f4571ae1700 1 --2- 172.21.15.67:0/2016716973 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f45580050d0 0x7f455800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:38:16.946 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.933+0000 7f45712e0700 1 --2- 172.21.15.67:0/2016716973 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f4558003a00 0x7f4558003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:38:16.946 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.933+0000 7f4571ae1700 1 -- 172.21.15.67:0/2016716973 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f4558003a00 msgr2=0x7f4558003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:38:16.946 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.933+0000 7f4571ae1700 1 --2- 172.21.15.67:0/2016716973 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f4558003a00 0x7f4558003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:38:16.947 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.933+0000 7f4571ae1700 1 -- 172.21.15.67:0/2016716973 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4558004400 msgr2=0x7f45580048d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-14T10:38:16.947 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.933+0000 7f4571ae1700 1 --2- 172.21.15.67:0/2016716973 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4558004400 0x7f45580048d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:38:16.947 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.933+0000 7f4571ae1700 1 -- 172.21.15.67:0/2016716973 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45580035b0 con 0x7f45580050d0 2023-10-14T10:38:16.948 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.933+0000 7f45712e0700 1 --2- 172.21.15.67:0/2016716973 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f4558003a00 0x7f4558003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-14T10:38:16.948 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.933+0000 7f4570adf700 1 --2- 172.21.15.67:0/2016716973 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4558004400 0x7f45580048d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-14T10:38:16.948 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.933+0000 7f4571ae1700 1 -- 172.21.15.67:0/2016716973 reap_dead start 2023-10-14T10:38:16.948 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.933+0000 7f4571ae1700 1 --2- 172.21.15.67:0/2016716973 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f45580050d0 0x7f455800ba10 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f456400c290 tx=0x7f4558005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c6b05d31aeb58da server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:38:16.949 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.933+0000 7f4563fff700 1 -- 172.21.15.67:0/2016716973 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4564001e70 con 0x7f45580050d0 2023-10-14T10:38:16.949 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.934+0000 7f4563fff700 1 -- 172.21.15.67:0/2016716973 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f456400ce70 con 0x7f45580050d0 2023-10-14T10:38:16.949 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.934+0000 7f4563fff700 1 -- 172.21.15.67:0/2016716973 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4564003c40 con 0x7f45580050d0 2023-10-14T10:38:16.949 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.934+0000 7f4573544700 1 -- 172.21.15.67:0/2016716973 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f45580050d0 msgr2=0x7f455800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:38:16.950 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.934+0000 7f4573544700 1 --2- 172.21.15.67:0/2016716973 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f45580050d0 0x7f455800ba10 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f456400c290 tx=0x7f4558005880 comp rx=0 tx=0).stop 2023-10-14T10:38:16.950 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.934+0000 7f4573544700 1 -- 172.21.15.67:0/2016716973 shutdown_connections 2023-10-14T10:38:16.950 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.934+0000 7f4573544700 1 --2- 172.21.15.67:0/2016716973 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f45580050d0 0x7f455800ba10 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:38:16.951 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.934+0000 7f4573544700 1 -- 172.21.15.67:0/2016716973 >> 172.21.15.67:0/2016716973 conn(0x7f456c06dae0 msgr2=0x7f456c06def0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:38:16.951 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.934+0000 7f4573544700 1 -- 172.21.15.67:0/2016716973 shutdown_connections 2023-10-14T10:38:16.951 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.934+0000 7f4573544700 1 -- 172.21.15.67:0/2016716973 wait complete. 2023-10-14T10:38:16.951 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.935+0000 7f4573544700 1 Processor -- start 2023-10-14T10:38:16.952 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.935+0000 7f4573544700 1 -- start start 2023-10-14T10:38:16.952 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.936+0000 7f4573544700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f45580050d0 0x7f456c0b1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:38:16.952 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.936+0000 7f4573544700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f456c0ac8b0 0x7f456c0acd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:38:16.952 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.936+0000 7f4573544700 1 --2- >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f456c0c9920 0x7f456c0c9da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:38:16.953 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.936+0000 7f4573544700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f456c0af340 con 0x7f456c0ac8b0 2023-10-14T10:38:16.953 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.936+0000 7f4573544700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f456c0ae6c0 con 0x7f45580050d0 2023-10-14T10:38:16.953 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.936+0000 7f4573544700 1 -- --> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f456c0ae9a0 con 0x7f456c0c9920 2023-10-14T10:38:16.953 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.936+0000 7f4570adf700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f456c0ac8b0 0x7f456c0acd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:38:16.954 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.936+0000 7f4570adf700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f456c0ac8b0 0x7f456c0acd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46364/0 (socket says 172.21.15.67:46364) 2023-10-14T10:38:16.954 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.936+0000 7f4570adf700 1 -- 172.21.15.67:0/1392957722 learned_addr learned my addr 172.21.15.67:0/1392957722 (peer_addr_for_me v2:172.21.15.67:0/0) 2023-10-14T10:38:16.954 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.936+0000 7f45712e0700 1 --2- 172.21.15.67:0/1392957722 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f45580050d0 0x7f456c0b1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:38:16.955 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.936+0000 7f4571ae1700 1 --2- 172.21.15.67:0/1392957722 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f456c0c9920 0x7f456c0c9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:38:16.955 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.937+0000 7f4570adf700 1 -- 172.21.15.67:0/1392957722 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f456c0c9920 msgr2=0x7f456c0c9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:38:16.955 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.937+0000 7f4570adf700 1 --2- 172.21.15.67:0/1392957722 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f456c0c9920 0x7f456c0c9da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:38:16.955 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.937+0000 7f4570adf700 1 -- 172.21.15.67:0/1392957722 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f45580050d0 msgr2=0x7f456c0b1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:38:16.956 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.937+0000 7f4570adf700 1 --2- 172.21.15.67:0/1392957722 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f45580050d0 0x7f456c0b1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:38:16.956 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.937+0000 7f4570adf700 1 -- 172.21.15.67:0/1392957722 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f456c0aec80 con 0x7f456c0ac8b0 2023-10-14T10:38:16.956 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.937+0000 7f4571ae1700 1 --2- 172.21.15.67:0/1392957722 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f456c0c9920 0x7f456c0c9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-14T10:38:16.956 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.937+0000 7f4570adf700 1 --2- 172.21.15.67:0/1392957722 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f456c0ac8b0 0x7f456c0acd30 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f45680017e0 tx=0x7f456800d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:38:16.957 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.937+0000 7f4561ffb700 1 -- 172.21.15.67:0/1392957722 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4568000f70 con 0x7f456c0ac8b0 2023-10-14T10:38:16.957 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.937+0000 7f4561ffb700 1 -- 172.21.15.67:0/1392957722 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f456800da40 con 0x7f456c0ac8b0 2023-10-14T10:38:16.957 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.937+0000 7f4573544700 1 -- 172.21.15.67:0/1392957722 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f456c0aef60 con 0x7f456c0ac8b0 2023-10-14T10:38:16.957 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.937+0000 7f4561ffb700 1 -- 172.21.15.67:0/1392957722 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f456800ddb0 con 0x7f456c0ac8b0 2023-10-14T10:38:16.958 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.937+0000 7f4573544700 1 -- 172.21.15.67:0/1392957722 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f456c0b1c90 con 0x7f456c0ac8b0 2023-10-14T10:38:16.959 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.938+0000 7f4573544700 1 -- 172.21.15.67:0/1392957722 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f456c0af480 con 0x7f456c0ac8b0 2023-10-14T10:38:16.959 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.939+0000 7f4561ffb700 1 -- 172.21.15.67:0/1392957722 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f45680070d0 con 0x7f456c0ac8b0 2023-10-14T10:38:16.960 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.940+0000 7f4561ffb700 1 --2- 172.21.15.67:0/1392957722 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f455006af00 0x7f455006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-14T10:38:16.960 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.940+0000 7f4561ffb700 1 -- 172.21.15.67:0/1392957722 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5729+0+0 (secure 0 0 0) 0x7f4568089f40 con 0x7f456c0ac8b0 2023-10-14T10:38:16.960 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.940+0000 7f45712e0700 1 --2- 172.21.15.67:0/1392957722 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f455006af00 0x7f455006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-14T10:38:16.960 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.940+0000 7f45712e0700 1 --2- 172.21.15.67:0/1392957722 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f455006af00 0x7f455006d3c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f455c009cc0 tx=0x7f455c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-14T10:38:16.961 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:16.944+0000 7f4561ffb700 1 -- 172.21.15.67:0/1392957722 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4568055020 con 0x7f456c0ac8b0 2023-10-14T10:38:17.189 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:17.187+0000 7f4573544700 1 -- 172.21.15.67:0/1392957722 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f456c0af480 con 0x7f456c0ac8b0 2023-10-14T10:38:17.190 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:17.188+0000 7f4561ffb700 1 -- 172.21.15.67:0/1392957722 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4918 (secure 0 0 0) 0x7f456c0af480 con 0x7f456c0ac8b0 2023-10-14T10:38:17.191 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-10-14T10:38:17.191 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6836/3371338659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":3371338659},{"type":"v1","addr":"172.21.15.72:6836","nonce":3371338659}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4688,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/2815272197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6835","nonce":2815272197},{"type":"v1","addr":"172.21.15.72:6837","nonce":2815272197}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5094,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6834/3494180260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":3494180260},{"type":"v1","addr":"172.21.15.67:6834","nonce":3494180260}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-14T10:36:59.462859+0000","modified":"2023-10-14T10:37:02.493817+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5097},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5097":{"gid":5097,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.67:6835/3065963596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6833","nonce":3065963596},{"type":"v1","addr":"172.21.15.67:6835","nonce":3065963596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"testcephfs1","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-14T10:38:17.192 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:17.190+0000 7f4573544700 1 -- 172.21.15.67:0/1392957722 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f455006af00 msgr2=0x7f455006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:38:17.192 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:17.190+0000 7f4573544700 1 --2- 172.21.15.67:0/1392957722 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f455006af00 0x7f455006d3c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f455c009cc0 tx=0x7f455c008040 comp rx=0 tx=0).stop 2023-10-14T10:38:17.193 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:17.190+0000 7f4573544700 1 -- 172.21.15.67:0/1392957722 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f456c0ac8b0 msgr2=0x7f456c0acd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-14T10:38:17.193 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:17.190+0000 7f4573544700 1 --2- 172.21.15.67:0/1392957722 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f456c0ac8b0 0x7f456c0acd30 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f45680017e0 tx=0x7f456800d430 comp rx=0 tx=0).stop 2023-10-14T10:38:17.194 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:17.190+0000 7f4573544700 1 -- 172.21.15.67:0/1392957722 shutdown_connections 2023-10-14T10:38:17.194 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:17.190+0000 7f4573544700 1 --2- 172.21.15.67:0/1392957722 >> [v2:172.21.15.72:6832/703488382,v1:172.21.15.72:6833/703488382] conn(0x7f455006af00 0x7f455006d3c0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:38:17.194 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:17.190+0000 7f4573544700 1 --2- 172.21.15.67:0/1392957722 >> [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] conn(0x7f456c0c9920 0x7f456c0c9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:38:17.194 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:17.190+0000 7f4573544700 1 --2- 172.21.15.67:0/1392957722 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f45580050d0 0x7f456c0b1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:38:17.195 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:17.190+0000 7f4573544700 1 --2- 172.21.15.67:0/1392957722 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f456c0ac8b0 0x7f456c0acd30 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-14T10:38:17.195 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:17.190+0000 7f4573544700 1 -- 172.21.15.67:0/1392957722 >> 172.21.15.67:0/1392957722 conn(0x7f456c06dae0 msgr2=0x7f456c15f500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-14T10:38:17.195 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:17.190+0000 7f4573544700 1 -- 172.21.15.67:0/1392957722 shutdown_connections 2023-10-14T10:38:17.196 INFO:teuthology.orchestra.run.smithi067.stderr:2023-10-14T10:38:17.190+0000 7f4573544700 1 -- 172.21.15.67:0/1392957722 wait complete. 2023-10-14T10:38:17.196 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 16 2023-10-14T10:38:17.204 DEBUG:tasks.cephfs.filesystem:already dead... 2023-10-14T10:38:17.205 INFO:teuthology.misc:Shutting down mds daemons... 2023-10-14T10:38:17.205 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-14T10:38:17.205 INFO:teuthology.orchestra.run:waiting for 300 2023-10-14T10:38:17.213 INFO:tasks.ceph.mds.a:Stopped 2023-10-14T10:38:17.214 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-14T10:38:17.214 INFO:teuthology.orchestra.run:waiting for 300 2023-10-14T10:38:17.263 INFO:tasks.ceph.mds.c:Stopped 2023-10-14T10:38:17.264 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-14T10:38:17.265 INFO:teuthology.orchestra.run:waiting for 300 2023-10-14T10:38:17.275 INFO:tasks.ceph.mds.b:Stopped 2023-10-14T10:38:17.276 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-14T10:38:17.276 INFO:teuthology.orchestra.run:waiting for 300 2023-10-14T10:38:17.327 INFO:tasks.ceph.mds.d:Stopped 2023-10-14T10:38:17.328 INFO:teuthology.misc:Shutting down osd daemons... 2023-10-14T10:38:17.328 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-10-14T10:38:17.329 INFO:teuthology.orchestra.run:waiting for 300 2023-10-14T10:38:17.371 INFO:tasks.ceph.osd.0:Stopped 2023-10-14T10:38:17.371 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-10-14T10:38:17.372 INFO:teuthology.orchestra.run:waiting for 300 2023-10-14T10:38:17.407 INFO:tasks.ceph.osd.1:Stopped 2023-10-14T10:38:17.408 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-10-14T10:38:17.408 INFO:teuthology.orchestra.run:waiting for 300 2023-10-14T10:38:17.484 INFO:tasks.ceph.osd.2:Stopped 2023-10-14T10:38:17.485 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-10-14T10:38:17.485 INFO:teuthology.orchestra.run:waiting for 300 2023-10-14T10:38:17.529 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2023-10-14T10:38:17.529 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2023-10-14T10:38:17.529 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2023-10-14T10:38:17.530 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2023-10-14T10:38:17.553 INFO:tasks.ceph.osd.3:Stopped 2023-10-14T10:38:17.554 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-10-14T10:38:17.554 INFO:teuthology.orchestra.run:waiting for 300 2023-10-14T10:38:17.594 INFO:tasks.ceph.osd.4:Stopped 2023-10-14T10:38:17.595 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-10-14T10:38:17.595 INFO:teuthology.orchestra.run:waiting for 300 2023-10-14T10:38:17.667 INFO:tasks.ceph.osd.5:Stopped 2023-10-14T10:38:17.668 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-10-14T10:38:17.668 INFO:teuthology.orchestra.run:waiting for 300 2023-10-14T10:38:17.737 INFO:tasks.ceph.osd.6:Stopped 2023-10-14T10:38:17.738 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-10-14T10:38:17.739 INFO:teuthology.orchestra.run:waiting for 300 2023-10-14T10:38:17.816 INFO:tasks.ceph.osd.7:Stopped 2023-10-14T10:38:17.816 INFO:teuthology.misc:Shutting down mgr daemons... 2023-10-14T10:38:17.817 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-10-14T10:38:17.818 INFO:teuthology.orchestra.run:waiting for 300 2023-10-14T10:38:17.840 INFO:tasks.ceph.mgr.y:Stopped 2023-10-14T10:38:17.841 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-10-14T10:38:17.841 INFO:teuthology.orchestra.run:waiting for 300 2023-10-14T10:38:17.884 INFO:tasks.ceph.mgr.x:Stopped 2023-10-14T10:38:17.884 INFO:teuthology.misc:Shutting down mon daemons... 2023-10-14T10:38:17.885 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-10-14T10:38:17.885 INFO:teuthology.orchestra.run:waiting for 300 2023-10-14T10:38:17.896 INFO:tasks.ceph.mon.a:Stopped 2023-10-14T10:38:17.896 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-10-14T10:38:17.896 INFO:teuthology.orchestra.run:waiting for 300 2023-10-14T10:38:17.936 INFO:tasks.ceph.mon.b:Stopped 2023-10-14T10:38:17.937 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-10-14T10:38:17.937 INFO:teuthology.orchestra.run:waiting for 300 2023-10-14T10:38:17.988 INFO:tasks.ceph.mon.c:Stopped 2023-10-14T10:38:17.989 INFO:tasks.ceph:Checking cluster log for badness... 2023-10-14T10:38:17.989 DEBUG:teuthology.orchestra.run.smithi067:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | head -n 1 2023-10-14T10:38:18.027 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi067.front.sepia.ceph.com 2023-10-14T10:38:18.028 DEBUG:teuthology.orchestra.run.smithi067:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-10-14T10:38:18.321 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi067.front.sepia.ceph.com 2023-10-14T10:38:18.322 DEBUG:teuthology.orchestra.run.smithi067:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-10-14T10:38:18.418 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi067.front.sepia.ceph.com 2023-10-14T10:38:18.419 DEBUG:teuthology.orchestra.run.smithi067:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-10-14T10:38:18.500 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi067.front.sepia.ceph.com 2023-10-14T10:38:18.648 DEBUG:teuthology.orchestra.run.smithi067:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-10-14T10:38:18.722 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi072.front.sepia.ceph.com 2023-10-14T10:38:18.723 DEBUG:teuthology.orchestra.run.smithi072:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-10-14T10:38:18.960 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi072.front.sepia.ceph.com 2023-10-14T10:38:18.960 DEBUG:teuthology.orchestra.run.smithi072:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-10-14T10:38:19.168 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi072.front.sepia.ceph.com 2023-10-14T10:38:19.169 DEBUG:teuthology.orchestra.run.smithi072:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-10-14T10:38:19.265 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi072.front.sepia.ceph.com 2023-10-14T10:38:19.266 DEBUG:teuthology.orchestra.run.smithi072:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-10-14T10:38:19.362 INFO:tasks.ceph:Archiving mon data... 2023-10-14T10:38:19.363 DEBUG:teuthology.misc:Transferring archived files from smithi067:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/rishabh-2023-10-13_18:04:14-fs-rishabh-2023oct13-testing-default-smithi/7425219/data/mon.a.tgz 2023-10-14T10:38:19.363 DEBUG:teuthology.orchestra.run.smithi067:> mktemp 2023-10-14T10:38:19.382 INFO:teuthology.orchestra.run.smithi067.stdout:/tmp/tmp.nO6EnLnmGe 2023-10-14T10:38:19.383 DEBUG:teuthology.orchestra.run.smithi067:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.nO6EnLnmGe 2023-10-14T10:38:19.519 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod 0666 /tmp/tmp.nO6EnLnmGe 2023-10-14T10:38:19.607 DEBUG:teuthology.orchestra.remote:smithi067:/tmp/tmp.nO6EnLnmGe is 280KB 2023-10-14T10:38:19.702 DEBUG:teuthology.orchestra.run.smithi067:> rm -fr /tmp/tmp.nO6EnLnmGe 2023-10-14T10:38:19.719 DEBUG:teuthology.misc:Transferring archived files from smithi072:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/rishabh-2023-10-13_18:04:14-fs-rishabh-2023oct13-testing-default-smithi/7425219/data/mon.b.tgz 2023-10-14T10:38:19.719 DEBUG:teuthology.orchestra.run.smithi072:> mktemp 2023-10-14T10:38:19.738 INFO:teuthology.orchestra.run.smithi072.stdout:/tmp/tmp.LhYbqbbuWX 2023-10-14T10:38:19.738 DEBUG:teuthology.orchestra.run.smithi072:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.LhYbqbbuWX 2023-10-14T10:38:19.884 DEBUG:teuthology.orchestra.run.smithi072:> sudo chmod 0666 /tmp/tmp.LhYbqbbuWX 2023-10-14T10:38:19.970 DEBUG:teuthology.orchestra.remote:smithi072:/tmp/tmp.LhYbqbbuWX is 375KB 2023-10-14T10:38:20.068 DEBUG:teuthology.orchestra.run.smithi072:> rm -fr /tmp/tmp.LhYbqbbuWX 2023-10-14T10:38:20.084 DEBUG:teuthology.misc:Transferring archived files from smithi072:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/rishabh-2023-10-13_18:04:14-fs-rishabh-2023oct13-testing-default-smithi/7425219/data/mon.c.tgz 2023-10-14T10:38:20.085 DEBUG:teuthology.orchestra.run.smithi072:> mktemp 2023-10-14T10:38:20.141 INFO:teuthology.orchestra.run.smithi072.stdout:/tmp/tmp.r9BDy61GFX 2023-10-14T10:38:20.142 DEBUG:teuthology.orchestra.run.smithi072:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.r9BDy61GFX 2023-10-14T10:38:20.275 DEBUG:teuthology.orchestra.run.smithi072:> sudo chmod 0666 /tmp/tmp.r9BDy61GFX 2023-10-14T10:38:20.362 DEBUG:teuthology.orchestra.remote:smithi072:/tmp/tmp.r9BDy61GFX is 375KB 2023-10-14T10:38:20.460 DEBUG:teuthology.orchestra.run.smithi072:> rm -fr /tmp/tmp.r9BDy61GFX 2023-10-14T10:38:20.477 INFO:tasks.ceph:Cleaning ceph cluster... 2023-10-14T10:38:20.477 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-14T10:38:20.480 DEBUG:teuthology.orchestra.run.smithi072:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-14T10:38:20.543 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-10-14T10:38:20.544 DEBUG:teuthology.orchestra.run.smithi067:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-14T10:38:20.551 DEBUG:teuthology.orchestra.run.smithi072:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-14T10:38:20.589 INFO:teuthology.orchestra.run.smithi067.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-10-14T10:38:20.620 INFO:teuthology.orchestra.run.smithi072.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-10-14T10:38:20.626 INFO:tasks.ceph:Archiving crash dumps... 2023-10-14T10:38:20.628 DEBUG:teuthology.misc:Transferring archived files from smithi067:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2023-10-13_18:04:14-fs-rishabh-2023oct13-testing-default-smithi/7425219/remote/smithi067/crash 2023-10-14T10:38:20.629 DEBUG:teuthology.orchestra.run.smithi067:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-14T10:38:20.665 DEBUG:teuthology.misc:Transferring archived files from smithi072:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2023-10-13_18:04:14-fs-rishabh-2023oct13-testing-default-smithi/7425219/remote/smithi072/crash 2023-10-14T10:38:20.666 DEBUG:teuthology.orchestra.run.smithi072:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-14T10:38:20.692 INFO:tasks.ceph:Shutting down logrotate 2023-10-14T10:38:20.693 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-14T10:38:20.710 DEBUG:teuthology.orchestra.run.smithi072:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-14T10:38:20.779 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-10-14T10:38:20.807 DEBUG:teuthology.orchestra.run.smithi072:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-10-14T10:38:20.833 INFO:tasks.ceph:Compressing logs... 2023-10-14T10:38:20.834 DEBUG:teuthology.orchestra.run.smithi067:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-10-14T10:38:20.849 DEBUG:teuthology.orchestra.run.smithi072:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-10-14T10:38:20.874 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118016.log 2023-10-14T10:38:20.875 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117878.log 2023-10-14T10:38:20.875 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose /var/log/ceph/ceph-client.admin.118016.log: -- /var/log/ceph/ceph-client.admin.103900.log 2023-10-14T10:38:20.911 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2023-10-14T10:38:20.911 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.110703.log 2023-10-14T10:38:20.911 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104279.log 2023-10-14T10:38:20.912 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.1.110703.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-10-14T10:38:20.912 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.110703.log.gz 2023-10-14T10:38:20.912 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2023-10-14T10:38:20.912 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.104279.log: /var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.111318.log 2023-10-14T10:38:20.913 INFO:teuthology.orchestra.run.smithi072.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.104279.log.gz 2023-10-14T10:38:20.913 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104234.log 2023-10-14T10:38:20.913 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.3.111318.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104144.log 2023-10-14T10:38:20.914 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.3.111318.log.gz 2023-10-14T10:38:20.914 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.104234.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104332.log 2023-10-14T10:38:20.914 INFO:teuthology.orchestra.run.smithi072.stderr: 91.4% -- replaced with /var/log/ceph/ceph.log.gz 2023-10-14T10:38:20.914 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.104144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104052.log 2023-10-14T10:38:20.915 INFO:teuthology.orchestra.run.smithi072.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.104234.log.gz 2023-10-14T10:38:20.915 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2023-10-14T10:38:20.915 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.104332.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.104144.log.gz 2023-10-14T10:38:20.916 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.104052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104189.log 2023-10-14T10:38:20.916 INFO:teuthology.orchestra.run.smithi072.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.104332.log.gz 2023-10-14T10:38:20.916 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-mds.d.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.104052.log.gz 2023-10-14T10:38:20.916 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-10-14T10:38:20.917 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.104189.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2023-10-14T10:38:20.917 INFO:teuthology.orchestra.run.smithi072.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.104189.log.gz 2023-10-14T10:38:20.917 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph.audit.log: /var/log/ceph/ceph-osd.4.log: 91.9% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2023-10-14T10:38:20.917 INFO:teuthology.orchestra.run.smithi072.stderr: 93.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-10-14T10:38:20.918 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.110711.log 2023-10-14T10:38:20.918 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2023-10-14T10:38:20.918 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.1.110711.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.111322.log 2023-10-14T10:38:20.919 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.110711.log.gz 2023-10-14T10:38:20.919 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2023-10-14T10:38:20.919 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.3.111322.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.111322.log.gz 2023-10-14T10:38:20.921 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104006.log 2023-10-14T10:38:20.921 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2023-10-14T10:38:20.922 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.104006.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.104006.log.gz 2023-10-14T10:38:20.927 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104097.log 2023-10-14T10:38:20.927 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.104097.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.104097.log.gz 2023-10-14T10:38:20.927 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2023-10-14T10:38:20.932 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2023-10-14T10:38:20.946 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.117878.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.118016.log.gz 2023-10-14T10:38:20.947 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110875.log 2023-10-14T10:38:20.947 INFO:teuthology.orchestra.run.smithi067.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.117878.log.gz 2023-10-14T10:38:20.947 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111236.log 2023-10-14T10:38:20.948 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.103900.log: /var/log/ceph/ceph-client.admin.110875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110878.log 2023-10-14T10:38:20.948 INFO:teuthology.orchestra.run.smithi067.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.103900.log.gz 2023-10-14T10:38:20.948 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.111236.log: 87.8%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.110875.log.gz--verbose 2023-10-14T10:38:20.949 INFO:teuthology.orchestra.run.smithi067.stderr:-- /var/log/ceph/ceph-client.admin.112423.log 2023-10-14T10:38:20.949 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.110878.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.111236.log.gz 2023-10-14T10:38:20.949 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.118461.log 2023-10-14T10:38:20.950 INFO:teuthology.orchestra.run.smithi067.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.110878.log.gz 2023-10-14T10:38:20.950 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114406.log 2023-10-14T10:38:20.950 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.112423.log: /var/log/ceph/ceph-client.0.118461.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110877.log 2023-10-14T10:38:20.951 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.118461.log.gz 2023-10-14T10:38:20.951 INFO:teuthology.orchestra.run.smithi067.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.112423.log.gz 2023-10-14T10:38:20.951 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 /var/log/ceph/ceph-client.admin.114406.log: --verbose -- /var/log/ceph/ceph-client.admin.112147.log 2023-10-14T10:38:20.951 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.110877.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.114406.log.gz 2023-10-14T10:38:20.952 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119803.log 2023-10-14T10:38:20.952 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.112147.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.110877.log.gz 2023-10-14T10:38:20.952 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114038.log 2023-10-14T10:38:20.953 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.119803.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.112147.log.gz 2023-10-14T10:38:20.953 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117970.log 2023-10-14T10:38:20.953 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.114038.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.119803.log.gz 2023-10-14T10:38:20.954 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117494.log 2023-10-14T10:38:20.954 INFO:teuthology.orchestra.run.smithi067.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.114038.log.gz 2023-10-14T10:38:20.954 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114959.log 2023-10-14T10:38:20.954 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.117970.log: gzip /var/log/ceph/ceph-client.admin.117494.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.115942.log 2023-10-14T10:38:20.955 INFO:teuthology.orchestra.run.smithi067.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.117970.log.gz 2023-10-14T10:38:20.955 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.114959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111917.log 2023-10-14T10:38:20.955 INFO:teuthology.orchestra.run.smithi067.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.117494.log.gz 2023-10-14T10:38:20.956 INFO:teuthology.orchestra.run.smithi067.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.114959.log.gz/var/log/ceph/ceph-client.admin.115942.log: 2023-10-14T10:38:20.956 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118062.log 2023-10-14T10:38:20.956 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.111917.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.115942.log.gz 2023-10-14T10:38:20.957 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116444.log 2023-10-14T10:38:20.957 INFO:teuthology.orchestra.run.smithi067.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.111917.log.gz 2023-10-14T10:38:20.957 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.118062.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2023-10-14T10:38:20.957 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.116444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111616.log 88.0% 2023-10-14T10:38:20.958 INFO:teuthology.orchestra.run.smithi067.stderr: -- replaced with /var/log/ceph/ceph-client.admin.118062.log.gz 2023-10-14T10:38:20.958 INFO:teuthology.orchestra.run.smithi067.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.116444.log.gz 2023-10-14T10:38:20.958 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116398.log 2023-10-14T10:38:20.959 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.111616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116536.log 2023-10-14T10:38:20.959 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.116398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104208.log 2023-10-14T10:38:20.959 INFO:teuthology.orchestra.run.smithi067.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.111616.log.gz 2023-10-14T10:38:20.960 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.116536.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.116398.log.gz 2023-10-14T10:38:20.960 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116582.log 2023-10-14T10:38:20.960 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.104208.log: 87.8%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.116536.log.gz--verbose 2023-10-14T10:38:20.960 INFO:teuthology.orchestra.run.smithi067.stderr:-- /var/log/ceph/ceph-client.admin.116168.log 2023-10-14T10:38:20.961 INFO:teuthology.orchestra.run.smithi067.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.104208.log.gz 2023-10-14T10:38:20.961 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.116582.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-10-14T10:38:20.961 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.116168.log: 89.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.116582.log.gz-5 2023-10-14T10:38:20.962 INFO:teuthology.orchestra.run.smithi067.stderr:--verbose -- /var/log/ceph/ceph-client.admin.114545.log 2023-10-14T10:38:20.962 INFO:teuthology.orchestra.run.smithi067.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.116168.log.gz 2023-10-14T10:38:20.962 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119476.log 2023-10-14T10:38:20.962 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.114545.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119522.log 2023-10-14T10:38:20.965 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.119476.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.114545.log.gz 2023-10-14T10:38:20.965 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119893.log 2023-10-14T10:38:20.966 INFO:teuthology.orchestra.run.smithi067.stderr: 84.4% -- replaced with /var/log/ceph/ceph.log.gz 2023-10-14T10:38:20.966 INFO:teuthology.orchestra.run.smithi067.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.119476.log.gz 2023-10-14T10:38:20.966 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119430.log 2023-10-14T10:38:20.966 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.119522.log: /var/log/ceph/ceph-client.admin.119893.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.119522.log.gz 2023-10-14T10:38:20.967 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111249.log 2023-10-14T10:38:20.967 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.119430.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.119893.log.gz 2023-10-14T10:38:20.967 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119384.log 2023-10-14T10:38:20.968 INFO:teuthology.orchestra.run.smithi067.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.119430.log.gz 2023-10-14T10:38:20.968 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.111249.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110462.log 2023-10-14T10:38:20.968 INFO:teuthology.orchestra.run.smithi067.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.111249.log.gz 2023-10-14T10:38:20.968 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.119384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117218.log 2023-10-14T10:38:20.969 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.110462.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.119384.log.gz 2023-10-14T10:38:20.969 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2023-10-14T10:38:20.969 INFO:teuthology.orchestra.run.smithi067.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.110462.log.gz 2023-10-14T10:38:20.969 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.117218.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110416.log 2023-10-14T10:38:20.970 INFO:teuthology.orchestra.run.smithi067.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.117218.log.gz 2023-10-14T10:38:20.970 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111479.log 2023-10-14T10:38:20.970 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.110416.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115655.log 2023-10-14T10:38:20.971 INFO:teuthology.orchestra.run.smithi067.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.110416.log.gz 2023-10-14T10:38:20.971 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.111479.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112193.log 2023-10-14T10:38:20.971 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115804.log 2023-10-14T10:38:20.971 INFO:teuthology.orchestra.run.smithi067.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.111479.log.gz 2023-10-14T10:38:20.972 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.115655.log: /var/log/ceph/ceph-client.admin.112193.log: gzip 89.5%-5 -- replaced with /var/log/ceph/ceph-client.admin.115655.log.gz--verbose 2023-10-14T10:38:20.972 INFO:teuthology.orchestra.run.smithi067.stderr:-- /var/log/ceph/ceph-client.admin.116214.log 2023-10-14T10:38:20.972 INFO:teuthology.orchestra.run.smithi067.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.112193.log.gz 2023-10-14T10:38:20.973 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.115804.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114867.log 2023-10-14T10:38:20.973 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.116214.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2023-10-14T10:38:20.973 INFO:teuthology.orchestra.run.smithi067.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.115804.log.gz 2023-10-14T10:38:20.973 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.114867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115896.log 2023-10-14T10:38:20.974 INFO:teuthology.orchestra.run.smithi067.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.116214.log.gz 2023-10-14T10:38:20.974 INFO:teuthology.orchestra.run.smithi067.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.114867.log.gz 2023-10-14T10:38:20.974 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114821.log 2023-10-14T10:38:20.975 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110654.log 2023-10-14T10:38:20.975 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.115896.log: /var/log/ceph/ceph-client.admin.114821.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109771.log 2023-10-14T10:38:20.975 INFO:teuthology.orchestra.run.smithi067.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.115896.log.gz 2023-10-14T10:38:20.975 INFO:teuthology.orchestra.run.smithi067.stderr: 92.2% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2023-10-14T10:38:20.976 INFO:teuthology.orchestra.run.smithi067.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.114821.log.gz 2023-10-14T10:38:20.976 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.110654.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119847.log 2023-10-14T10:38:20.976 INFO:teuthology.orchestra.run.smithi067.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.110654.log.gz 2023-10-14T10:38:20.977 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.109771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119246.log 2023-10-14T10:38:20.977 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.113812.log 2023-10-14T10:38:20.977 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.119847.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.109771.log.gz 2023-10-14T10:38:20.978 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.119246.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111244.log 2023-10-14T10:38:20.978 INFO:teuthology.orchestra.run.smithi067.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.119847.log.gz 2023-10-14T10:38:20.978 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.2.113812.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112285.log 87.4% 2023-10-14T10:38:20.978 INFO:teuthology.orchestra.run.smithi067.stderr: -- replaced with /var/log/ceph/ceph-client.admin.119246.log.gz 2023-10-14T10:38:20.979 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.2.113812.log.gz 2023-10-14T10:38:20.979 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.111244.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2023-10-14T10:38:20.979 INFO:teuthology.orchestra.run.smithi067.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.111244.log.gz 2023-10-14T10:38:20.980 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.112285.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115224.log 2023-10-14T10:38:20.980 INFO:teuthology.orchestra.run.smithi067.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.112285.log.gz 2023-10-14T10:38:20.980 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111247.log 2023-10-14T10:38:20.980 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-osd.3.log: /var/log/ceph/ceph-client.admin.115224.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119108.log 2023-10-14T10:38:20.981 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.111247.log: 89.0%gzip -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.115224.log.gz-- 2023-10-14T10:38:20.981 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.116076.log 2023-10-14T10:38:20.981 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.119108.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.111247.log.gz 2023-10-14T10:38:20.982 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117170.log 2023-10-14T10:38:20.982 INFO:teuthology.orchestra.run.smithi067.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.119108.log.gz 2023-10-14T10:38:20.982 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.116076.log: /var/log/ceph/ceph-client.admin.117170.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.116076.log.gz 2023-10-14T10:38:20.982 INFO:teuthology.orchestra.run.smithi067.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.117170.log.gz 2023-10-14T10:38:20.983 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112331.log 2023-10-14T10:38:20.983 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117402.log 2023-10-14T10:38:20.983 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.112331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111618.log 2023-10-14T10:38:20.984 INFO:teuthology.orchestra.run.smithi067.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.112331.log.gz 2023-10-14T10:38:20.984 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.117402.log: /var/log/ceph/ceph-client.admin.111618.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.117402.log.gz 2023-10-14T10:38:20.984 INFO:teuthology.orchestra.run.smithi067.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.111618.log.gz 2023-10-14T10:38:20.984 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118711.log 2023-10-14T10:38:20.985 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110882.log 2023-10-14T10:38:20.985 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.118711.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117924.log 2023-10-14T10:38:20.985 INFO:teuthology.orchestra.run.smithi067.stderr: 89.0%/var/log/ceph/ceph-client.admin.110882.log: -- replaced with /var/log/ceph/ceph-client.admin.118711.log.gz 2023-10-14T10:38:20.986 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.117924.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.110882.log.gz 2023-10-14T10:38:20.986 INFO:teuthology.orchestra.run.smithi067.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.117924.log.gz 2023-10-14T10:38:20.986 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116122.log 2023-10-14T10:38:20.986 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111234.log 2023-10-14T10:38:20.987 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.116122.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2023-10-14T10:38:20.987 INFO:teuthology.orchestra.run.smithi067.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.116122.log.gz 2023-10-14T10:38:20.987 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.111234.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114775.log 2023-10-14T10:38:20.988 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115362.log 2023-10-14T10:38:20.988 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-mds.c.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.111234.log.gz 2023-10-14T10:38:20.988 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.114775.log: /var/log/ceph/ceph-client.admin.115362.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.114775.log.gz 2023-10-14T10:38:20.988 INFO:teuthology.orchestra.run.smithi067.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.115362.log.gz 2023-10-14T10:38:20.989 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111615.log 2023-10-14T10:38:20.989 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119568.log 2023-10-14T10:38:20.989 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.111615.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.111615.log.gz 2023-10-14T10:38:20.990 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.119568.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.119568.log.gz 2023-10-14T10:38:20.990 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119200.log 2023-10-14T10:38:20.990 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114268.log 2023-10-14T10:38:20.991 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.119200.log: /var/log/ceph/ceph-client.admin.114268.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.119200.log.gz 2023-10-14T10:38:20.991 INFO:teuthology.orchestra.run.smithi067.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.114268.log.gz 2023-10-14T10:38:20.991 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115850.log 2023-10-14T10:38:20.992 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114314.log 2023-10-14T10:38:20.992 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.115850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116306.log 2023-10-14T10:38:20.992 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.114314.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.115850.log.gz 2023-10-14T10:38:20.992 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.116306.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.114314.log.gz 2023-10-14T10:38:20.993 INFO:teuthology.orchestra.run.smithi067.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.116306.log.gz 2023-10-14T10:38:20.993 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110554.log 2023-10-14T10:38:20.993 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115270.log 2023-10-14T10:38:20.994 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.110554.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111871.log 2023-10-14T10:38:20.994 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.115270.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.110554.log.gz 2023-10-14T10:38:20.994 INFO:teuthology.orchestra.run.smithi067.stderr: 92.7% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2023-10-14T10:38:20.994 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.111871.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.115270.log.gz 2023-10-14T10:38:20.995 INFO:teuthology.orchestra.run.smithi067.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.111871.log.gz 2023-10-14T10:38:20.995 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114130.log 2023-10-14T10:38:20.995 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117356.log 2023-10-14T10:38:20.996 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.114130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110370.log 2023-10-14T10:38:20.996 INFO:teuthology.orchestra.run.smithi067.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.114130.log.gz 2023-10-14T10:38:20.996 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.117356.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110324.log 2023-10-14T10:38:20.997 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118970.log 2023-10-14T10:38:20.997 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.110370.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.117356.log.gz 2023-10-14T10:38:20.997 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.110324.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.110370.log.gz 2023-10-14T10:38:20.997 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.118970.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.110324.log.gz 2023-10-14T10:38:20.998 INFO:teuthology.orchestra.run.smithi067.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.118970.log.gz 2023-10-14T10:38:20.998 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112669.log 2023-10-14T10:38:20.998 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115097.log 2023-10-14T10:38:20.999 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.112669.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117588.log 2023-10-14T10:38:20.999 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.115097.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-10-14T10:38:20.999 INFO:teuthology.orchestra.run.smithi067.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.112669.log.gz 2023-10-14T10:38:20.999 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.117588.log: 89.1%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.115097.log.gz--verbose 2023-10-14T10:38:21.000 INFO:teuthology.orchestra.run.smithi067.stderr:-- /var/log/ceph/ceph-client.admin.112101.log 2023-10-14T10:38:21.000 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph.audit.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.117588.log.gz 2023-10-14T10:38:21.000 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119154.log 2023-10-14T10:38:21.001 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.112101.log: /var/log/ceph/ceph-client.admin.119154.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.112101.log.gz 2023-10-14T10:38:21.001 INFO:teuthology.orchestra.run.smithi067.stderr: 89.0% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-10-14T10:38:21.001 INFO:teuthology.orchestra.run.smithi067.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.119154.log.gz 2023-10-14T10:38:21.001 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118757.log 2023-10-14T10:38:21.002 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112009.log 2023-10-14T10:38:21.002 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.118757.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115988.log 2023-10-14T10:38:21.002 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.112009.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.118757.log.gz 2023-10-14T10:38:21.003 INFO:teuthology.orchestra.run.smithi067.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.112009.log.gz 2023-10-14T10:38:21.003 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.115988.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.115988.log.gz 2023-10-14T10:38:21.003 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115496.log 2023-10-14T10:38:21.003 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112377.log 2023-10-14T10:38:21.004 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.115496.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110508.log 2023-10-14T10:38:21.004 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.112377.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.115496.log.gz 2023-10-14T10:38:21.004 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.110508.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.112377.log.gz 2023-10-14T10:38:21.005 INFO:teuthology.orchestra.run.smithi067.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.110508.log.gz 2023-10-14T10:38:21.005 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112055.log 2023-10-14T10:38:21.005 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117637.log 2023-10-14T10:38:21.005 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.112055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111963.log 2023-10-14T10:38:21.006 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.117637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111613.log 2023-10-14T10:38:21.006 INFO:teuthology.orchestra.run.smithi067.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.112055.log.gz 2023-10-14T10:38:21.006 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.111963.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.117637.log.gz 2023-10-14T10:38:21.007 INFO:teuthology.orchestra.run.smithi067.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.111963.log.gz 2023-10-14T10:38:21.007 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.111613.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.111613.log.gz 2023-10-14T10:38:21.007 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114084.log 2023-10-14T10:38:21.007 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.113159.log 2023-10-14T10:38:21.008 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.114084.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116812.log 2023-10-14T10:38:21.008 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.0.113159.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116260.log 2023-10-14T10:38:21.008 INFO:teuthology.orchestra.run.smithi067.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.114084.log.gz 2023-10-14T10:38:21.008 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.113159.log.gz 2023-10-14T10:38:21.009 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.116812.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116628.log 2023-10-14T10:38:21.009 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.116260.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.116812.log.gz 2023-10-14T10:38:21.009 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119728.log 2023-10-14T10:38:21.010 INFO:teuthology.orchestra.run.smithi067.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.116260.log.gz 2023-10-14T10:38:21.010 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.116628.log: /var/log/ceph/ceph-client.admin.119728.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.116628.log.gz 2023-10-14T10:38:21.010 INFO:teuthology.orchestra.run.smithi067.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.119728.log.gz 2023-10-14T10:38:21.010 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119614.log 2023-10-14T10:38:21.011 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119661.log 2023-10-14T10:38:21.011 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.119614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104341.log 2023-10-14T10:38:21.011 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.119661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114176.log 2023-10-14T10:38:21.012 INFO:teuthology.orchestra.run.smithi067.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.119614.log.gz 2023-10-14T10:38:21.012 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.104341.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.119661.log.gz 2023-10-14T10:38:21.012 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118878.log 2023-10-14T10:38:21.012 INFO:teuthology.orchestra.run.smithi067.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.104341.log.gz 2023-10-14T10:38:21.013 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.114176.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.97616.log 2023-10-14T10:38:21.013 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.118878.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110278.log 2023-10-14T10:38:21.013 INFO:teuthology.orchestra.run.smithi067.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.114176.log.gz 2023-10-14T10:38:21.014 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph.tmp-client.admin.97616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.113816.log 2023-10-14T10:38:21.014 INFO:teuthology.orchestra.run.smithi067.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.118878.log.gz 2023-10-14T10:38:21.014 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.97616.log.gz 2023-10-14T10:38:21.015 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.110278.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119062.log 2023-10-14T10:38:21.015 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.2.113816.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113381.log 2023-10-14T10:38:21.015 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.2.113816.log.gz 2023-10-14T10:38:21.015 INFO:teuthology.orchestra.run.smithi067.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.110278.log.gz 2023-10-14T10:38:21.016 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.119062.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118924.log 2023-10-14T10:38:21.016 INFO:teuthology.orchestra.run.smithi067.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.119062.log.gz 2023-10-14T10:38:21.016 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.113381.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117738.log 2023-10-14T10:38:21.017 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109998.log 2023-10-14T10:38:21.017 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.118924.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.113381.log.gz 2023-10-14T10:38:21.017 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.117738.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.118924.log.gz 2023-10-14T10:38:21.017 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.109998.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.117738.log.gz 2023-10-14T10:38:21.018 INFO:teuthology.orchestra.run.smithi067.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.109998.log.gz 2023-10-14T10:38:21.018 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118154.log 2023-10-14T10:38:21.018 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115408.log 2023-10-14T10:38:21.019 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.118154.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110876.log 2023-10-14T10:38:21.019 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.115408.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.118154.log.gz 2023-10-14T10:38:21.019 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.110876.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.115408.log.gz 2023-10-14T10:38:21.019 INFO:teuthology.orchestra.run.smithi067.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.110876.log.gz 2023-10-14T10:38:21.020 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109577.log 2023-10-14T10:38:21.020 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116994.log 2023-10-14T10:38:21.020 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.109577.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114683.log 2023-10-14T10:38:21.021 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.116994.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2023-10-14T10:38:21.021 INFO:teuthology.orchestra.run.smithi067.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.109577.log.gz 2023-10-14T10:38:21.021 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.114683.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.116994.log.gz 2023-10-14T10:38:21.021 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-osd.1.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.114683.log.gz 2023-10-14T10:38:21.022 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110186.log 2023-10-14T10:38:21.022 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110872.log 2023-10-14T10:38:21.022 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.110186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.118457.log 2023-10-14T10:38:21.022 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.110872.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.110186.log.gz 2023-10-14T10:38:21.023 INFO:teuthology.orchestra.run.smithi067.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.110872.log.gz 2023-10-14T10:38:21.023 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.0.118457.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.118457.log.gz 2023-10-14T10:38:21.023 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119338.log 2023-10-14T10:38:21.024 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110600.log 2023-10-14T10:38:21.024 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.119338.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114453.log 2023-10-14T10:38:21.024 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.110600.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.119338.log.gz 2023-10-14T10:38:21.024 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.114453.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.110600.log.gz 2023-10-14T10:38:21.025 INFO:teuthology.orchestra.run.smithi067.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.114453.log.gz 2023-10-14T10:38:21.025 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115588.log 2023-10-14T10:38:21.025 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117540.log 2023-10-14T10:38:21.026 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.115588.log: /var/log/ceph/ceph-client.admin.117540.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.115588.log.gz 2023-10-14T10:38:21.026 INFO:teuthology.orchestra.run.smithi067.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.117540.log.gz 2023-10-14T10:38:21.026 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115051.log 2023-10-14T10:38:21.026 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116720.log 2023-10-14T10:38:21.027 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.115051.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110232.log 2023-10-14T10:38:21.027 INFO:teuthology.orchestra.run.smithi067.stderr: 88.1%/var/log/ceph/ceph-client.admin.116720.log: -- replaced with /var/log/ceph/ceph-client.admin.115051.log.gz 2023-10-14T10:38:21.027 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.110232.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.116720.log.gz 2023-10-14T10:38:21.028 INFO:teuthology.orchestra.run.smithi067.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.110232.log.gz 2023-10-14T10:38:21.028 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112239.log 2023-10-14T10:38:21.028 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111612.log 2023-10-14T10:38:21.028 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.112239.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.113151.log 2023-10-14T10:38:21.029 INFO:teuthology.orchestra.run.smithi067.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.112239.log.gz 2023-10-14T10:38:21.029 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.111612.log: /var/log/ceph/ceph-client.0.113151.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.113151.log.gz 2023-10-14T10:38:21.029 INFO:teuthology.orchestra.run.smithi067.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.111612.log.gz 2023-10-14T10:38:21.030 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111246.log 2023-10-14T10:38:21.030 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117078.log 2023-10-14T10:38:21.030 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.111246.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111606.log 2023-10-14T10:38:21.031 INFO:teuthology.orchestra.run.smithi067.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.111246.log.gz 2023-10-14T10:38:21.032 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.117078.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.117078.log.gz 2023-10-14T10:38:21.032 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114637.log 2023-10-14T10:38:21.032 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.111606.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111248.log 2023-10-14T10:38:21.032 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.114637.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.111606.log.gz 2023-10-14T10:38:21.033 INFO:teuthology.orchestra.run.smithi067.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.114637.log.gz 2023-10-14T10:38:21.033 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.111248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116352.log 2023-10-14T10:38:21.033 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118108.log 2023-10-14T10:38:21.033 INFO:teuthology.orchestra.run.smithi067.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.111248.log.gz 2023-10-14T10:38:21.034 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.116352.log: /var/log/ceph/ceph-client.admin.118108.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.116352.log.gz 2023-10-14T10:38:21.034 INFO:teuthology.orchestra.run.smithi067.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.118108.log.gz 2023-10-14T10:38:21.034 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114499.log 2023-10-14T10:38:21.035 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114913.log 2023-10-14T10:38:21.035 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.114499.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110881.log 2023-10-14T10:38:21.035 INFO:teuthology.orchestra.run.smithi067.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.114499.log.gz 2023-10-14T10:38:21.036 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.114913.log: /var/log/ceph/ceph-client.admin.110881.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.114913.log.gz 2023-10-14T10:38:21.036 INFO:teuthology.orchestra.run.smithi067.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.110881.log.gz 2023-10-14T10:38:21.036 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116904.log 2023-10-14T10:38:21.036 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119292.log 2023-10-14T10:38:21.037 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.116904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117786.log 2023-10-14T10:38:21.037 INFO:teuthology.orchestra.run.smithi067.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.116904.log.gz 2023-10-14T10:38:21.037 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.119292.log: /var/log/ceph/ceph-client.admin.117786.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.119292.log.gz 2023-10-14T10:38:21.037 INFO:teuthology.orchestra.run.smithi067.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.117786.log.gz 2023-10-14T10:38:21.038 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110044.log 2023-10-14T10:38:21.038 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115316.log 2023-10-14T10:38:21.038 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 /var/log/ceph/ceph-client.admin.110044.log: --verbose -- /var/log/ceph/ceph-client.admin.116674.log 2023-10-14T10:38:21.039 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.115316.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.110044.log.gz 2023-10-14T10:38:21.039 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.116674.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.115316.log.gz 2023-10-14T10:38:21.039 INFO:teuthology.orchestra.run.smithi067.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.116674.log.gz 2023-10-14T10:38:21.039 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109892.log 2023-10-14T10:38:21.040 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112469.log 2023-10-14T10:38:21.040 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.109892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109686.log 2023-10-14T10:38:21.040 INFO:teuthology.orchestra.run.smithi067.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.109892.log.gz 2023-10-14T10:38:21.040 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.112469.log: /var/log/ceph/ceph-client.admin.109686.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.112469.log.gz 2023-10-14T10:38:21.041 INFO:teuthology.orchestra.run.smithi067.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.109686.log.gz 2023-10-14T10:38:21.041 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118811.log 2023-10-14T10:38:21.041 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117448.log 2023-10-14T10:38:21.042 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.118811.log: /var/log/ceph/ceph-client.admin.117448.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.118811.log.gz 2023-10-14T10:38:21.042 INFO:teuthology.orchestra.run.smithi067.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.117448.log.gz 2023-10-14T10:38:21.042 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114591.log 2023-10-14T10:38:21.042 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111480.log 2023-10-14T10:38:21.043 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.114591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117310.log 2023-10-14T10:38:21.043 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.111480.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.114591.log.gz 2023-10-14T10:38:21.043 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.117310.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.111480.log.gz 2023-10-14T10:38:21.043 INFO:teuthology.orchestra.run.smithi067.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.117310.log.gz 2023-10-14T10:38:21.044 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110870.log 2023-10-14T10:38:21.044 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119016.log 2023-10-14T10:38:21.044 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.110870.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115701.log 2023-10-14T10:38:21.045 INFO:teuthology.orchestra.run.smithi067.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.110870.log.gz 2023-10-14T10:38:21.045 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.119016.log: /var/log/ceph/ceph-client.admin.115701.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.119016.log.gz 2023-10-14T10:38:21.045 INFO:teuthology.orchestra.run.smithi067.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.115701.log.gz 2023-10-14T10:38:21.045 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116490.log 2023-10-14T10:38:21.046 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115005.log 2023-10-14T10:38:21.046 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.116490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114222.log 2023-10-14T10:38:21.046 INFO:teuthology.orchestra.run.smithi067.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.116490.log.gz 2023-10-14T10:38:21.046 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.115005.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.115005.log.gz 2023-10-14T10:38:21.048 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-mds.b.log: 91.5% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2023-10-14T10:38:21.049 INFO:teuthology.orchestra.run.smithi072.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2023-10-14T10:38:21.049 INFO:teuthology.orchestra.run.smithi072.stderr: 95.6% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2023-10-14T10:38:21.049 INFO:teuthology.orchestra.run.smithi072.stderr: 93.2% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2023-10-14T10:38:21.050 INFO:teuthology.orchestra.run.smithi072.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2023-10-14T10:38:21.050 INFO:teuthology.orchestra.run.smithi072.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2023-10-14T10:38:21.050 INFO:teuthology.orchestra.run.smithi072.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2023-10-14T10:38:21.055 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.114222.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.114222.log.gz 2023-10-14T10:38:21.056 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117692.log 2023-10-14T10:38:21.056 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117264.log 2023-10-14T10:38:21.056 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.117692.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.117692.log.gz 2023-10-14T10:38:21.056 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.117264.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.117264.log.gz 2023-10-14T10:38:21.057 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114360.log 2023-10-14T10:38:21.057 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118224.log 2023-10-14T10:38:21.057 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.114360.log: /var/log/ceph/ceph-client.admin.118224.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.114360.log.gz 2023-10-14T10:38:21.058 INFO:teuthology.orchestra.run.smithi067.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.118224.log.gz 2023-10-14T10:38:21.058 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109820.log 2023-10-14T10:38:21.058 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111825.log 2023-10-14T10:38:21.058 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.109820.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110092.log 2023-10-14T10:38:21.059 INFO:teuthology.orchestra.run.smithi067.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.109820.log.gz 2023-10-14T10:38:21.059 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.111825.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.111825.log.gz 2023-10-14T10:38:21.059 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114729.log 2023-10-14T10:38:21.060 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.110092.log: /var/log/ceph/ceph-client.admin.114729.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.110092.log.gz 2023-10-14T10:38:21.060 INFO:teuthology.orchestra.run.smithi067.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.114729.log.gz 2023-10-14T10:38:21.060 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2023-10-14T10:38:21.060 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111250.log 2023-10-14T10:38:21.061 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110140.log 2023-10-14T10:38:21.061 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.111250.log: /var/log/ceph/ceph-client.admin.110140.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.111250.log.gz 2023-10-14T10:38:21.061 INFO:teuthology.orchestra.run.smithi067.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.110140.log.gz 2023-10-14T10:38:21.062 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118665.log 2023-10-14T10:38:21.062 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116858.log 2023-10-14T10:38:21.062 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.118665.log: /var/log/ceph/ceph-client.admin.116858.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.118665.log.gz 2023-10-14T10:38:21.062 INFO:teuthology.orchestra.run.smithi067.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.116858.log.gz 2023-10-14T10:38:21.063 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2023-10-14T10:38:21.063 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115542.log 2023-10-14T10:38:21.063 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-mgr.y.log: /var/log/ceph/ceph-client.admin.115542.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.115542.log.gz 2023-10-14T10:38:21.064 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116766.log 2023-10-14T10:38:21.064 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.116766.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.116766.log.gz 2023-10-14T10:38:21.064 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117124.log 2023-10-14T10:38:21.064 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.117124.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.117124.log.gz 2023-10-14T10:38:21.065 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117832.log 2023-10-14T10:38:21.065 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.117832.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.117832.log.gz 2023-10-14T10:38:21.065 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112515.log 2023-10-14T10:38:21.066 INFO:teuthology.orchestra.run.smithi067.stderr: 92.7% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2023-10-14T10:38:21.066 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109944.log 2023-10-14T10:38:21.066 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.112515.log: /var/log/ceph/ceph-client.admin.109944.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.109944.log.gz 2023-10-14T10:38:21.066 INFO:teuthology.orchestra.run.smithi067.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.112515.log.gz 2023-10-14T10:38:21.067 INFO:teuthology.orchestra.run.smithi067.stderr: 93.1% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2023-10-14T10:38:21.067 INFO:teuthology.orchestra.run.smithi067.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2023-10-14T10:38:21.067 INFO:teuthology.orchestra.run.smithi067.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2023-10-14T10:38:21.067 INFO:teuthology.orchestra.run.smithi067.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2023-10-14T10:38:21.081 INFO:teuthology.orchestra.run.smithi072.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2023-10-14T10:38:21.083 INFO:teuthology.orchestra.run.smithi072.stderr: 2023-10-14T10:38:21.084 INFO:teuthology.orchestra.run.smithi072.stderr:real 0m0.192s 2023-10-14T10:38:21.084 INFO:teuthology.orchestra.run.smithi072.stderr:user 0m0.820s 2023-10-14T10:38:21.084 INFO:teuthology.orchestra.run.smithi072.stderr:sys 0m0.058s 2023-10-14T10:38:21.124 INFO:teuthology.orchestra.run.smithi067.stderr: 90.9% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2023-10-14T10:38:21.125 INFO:teuthology.orchestra.run.smithi067.stderr: 2023-10-14T10:38:21.126 INFO:teuthology.orchestra.run.smithi067.stderr:real 0m0.263s 2023-10-14T10:38:21.126 INFO:teuthology.orchestra.run.smithi067.stderr:user 0m0.741s 2023-10-14T10:38:21.127 INFO:teuthology.orchestra.run.smithi067.stderr:sys 0m0.109s 2023-10-14T10:38:21.127 INFO:tasks.ceph:Archiving logs... 2023-10-14T10:38:21.128 DEBUG:teuthology.misc:Transferring archived files from smithi067:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-10-13_18:04:14-fs-rishabh-2023oct13-testing-default-smithi/7425219/remote/smithi067/log 2023-10-14T10:38:21.129 DEBUG:teuthology.orchestra.run.smithi067:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-14T10:38:21.434 DEBUG:teuthology.misc:Transferring archived files from smithi072:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-10-13_18:04:14-fs-rishabh-2023oct13-testing-default-smithi/7425219/remote/smithi072/log 2023-10-14T10:38:21.436 DEBUG:teuthology.orchestra.run.smithi072:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-14T10:38:21.576 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-10-14T10:38:21.599 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-10-14T10:38:21.599 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-14T10:38:21.603 DEBUG:teuthology.orchestra.run.smithi072:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-14T10:38:21.684 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-10-14T10:38:21.694 INFO:teuthology.task.clock:Checking final clock skew... 2023-10-14T10:38:21.694 DEBUG:teuthology.orchestra.run.smithi067:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-14T10:38:21.697 DEBUG:teuthology.orchestra.run.smithi072:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-14T10:38:21.710 INFO:teuthology.orchestra.run.smithi067.stderr:bash: ntpq: command not found 2023-10-14T10:38:21.713 INFO:teuthology.orchestra.run.smithi072.stderr:bash: ntpq: command not found 2023-10-14T10:38:21.715 INFO:teuthology.orchestra.run.smithi067.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-14T10:38:21.715 INFO:teuthology.orchestra.run.smithi067.stdout:=============================================================================== 2023-10-14T10:38:21.715 INFO:teuthology.orchestra.run.smithi067.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 3 +4292us[+4300us] +/- 90ms 2023-10-14T10:38:21.716 INFO:teuthology.orchestra.run.smithi067.stdout:^* hv02.front.sepia.ceph.com 3 6 377 3 -1680us[-1672us] +/- 59ms 2023-10-14T10:38:21.716 INFO:teuthology.orchestra.run.smithi067.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 2 -1358us[-1358us] +/- 75ms 2023-10-14T10:38:21.716 INFO:teuthology.orchestra.run.smithi067.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-10-14T10:38:21.718 INFO:teuthology.orchestra.run.smithi072.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-14T10:38:21.718 INFO:teuthology.orchestra.run.smithi072.stdout:=============================================================================== 2023-10-14T10:38:21.719 INFO:teuthology.orchestra.run.smithi072.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 4 +4289us[+4296us] +/- 90ms 2023-10-14T10:38:21.719 INFO:teuthology.orchestra.run.smithi072.stdout:^* hv02.front.sepia.ceph.com 3 6 377 2 -1674us[-1666us] +/- 59ms 2023-10-14T10:38:21.719 INFO:teuthology.orchestra.run.smithi072.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 2 -1350us[-1350us] +/- 75ms 2023-10-14T10:38:21.719 INFO:teuthology.orchestra.run.smithi072.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-10-14T10:38:21.720 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-10-14T10:38:21.729 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-10-14T10:38:21.730 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-10-14T10:38:21.740 DEBUG:teuthology.orchestra.run.smithi067:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-14T10:38:21.857 DEBUG:teuthology.orchestra.run.smithi072:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-14T10:38:21.949 DEBUG:teuthology.orchestra.run.smithi067:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-14T10:38:21.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-14T10:38:21.983 DEBUG:teuthology.orchestra.run.smithi072:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-14T10:38:22.014 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-14T10:38:22.015 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-10-14T10:38:22.026 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-10-14T10:38:22.045 INFO:teuthology.task.internal:Duration was 869.801762 seconds 2023-10-14T10:38:22.047 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-10-14T10:38:22.064 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-10-14T10:38:22.064 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-14T10:38:22.067 DEBUG:teuthology.orchestra.run.smithi072:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-14T10:38:22.112 INFO:teuthology.orchestra.run.smithi067.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-14T10:38:22.112 INFO:teuthology.orchestra.run.smithi072.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-14T10:38:22.442 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-10-14T10:38:22.443 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi067.front.sepia.ceph.com 2023-10-14T10:38:22.443 DEBUG:teuthology.orchestra.run.smithi067:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-14T10:38:22.466 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi072.front.sepia.ceph.com 2023-10-14T10:38:22.467 DEBUG:teuthology.orchestra.run.smithi072:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-14T10:38:22.488 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-10-14T10:38:22.489 DEBUG:teuthology.orchestra.run.smithi067:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-14T10:38:22.510 DEBUG:teuthology.orchestra.run.smithi072:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-14T10:38:22.531 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2023-10-14T10:38:22.531 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c has been restored 2023-10-14T10:38:22.532 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2023-10-14T10:38:22.532 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2023-10-14T10:38:22.568 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-10-14T10:38:22.569 DEBUG:teuthology.orchestra.run.smithi067:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-14T10:38:22.592 DEBUG:teuthology.orchestra.run.smithi072:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-14T10:38:22.681 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-10-14T10:38:22.692 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-10-14T10:38:22.692 DEBUG:teuthology.orchestra.run.smithi067:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-14T10:38:22.719 DEBUG:teuthology.orchestra.run.smithi072:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-14T10:38:22.747 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-10-14T10:38:22.757 DEBUG:teuthology.orchestra.run.smithi067:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-14T10:38:22.762 DEBUG:teuthology.orchestra.run.smithi072:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-14T10:38:22.786 INFO:teuthology.orchestra.run.smithi067.stdout:kernel.core_pattern = core 2023-10-14T10:38:22.812 INFO:teuthology.orchestra.run.smithi072.stdout:kernel.core_pattern = core 2023-10-14T10:38:22.826 DEBUG:teuthology.orchestra.run.smithi067:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-14T10:38:22.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-14T10:38:22.857 DEBUG:teuthology.orchestra.run.smithi072:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-14T10:38:22.881 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-14T10:38:22.883 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-10-14T10:38:22.893 INFO:teuthology.task.internal:Transferring archived files... 2023-10-14T10:38:22.894 DEBUG:teuthology.misc:Transferring archived files from smithi067:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-10-13_18:04:14-fs-rishabh-2023oct13-testing-default-smithi/7425219/remote/smithi067 2023-10-14T10:38:22.895 DEBUG:teuthology.orchestra.run.smithi067:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-14T10:38:22.936 DEBUG:teuthology.misc:Transferring archived files from smithi072:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-10-13_18:04:14-fs-rishabh-2023oct13-testing-default-smithi/7425219/remote/smithi072 2023-10-14T10:38:22.937 DEBUG:teuthology.orchestra.run.smithi072:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-14T10:38:22.975 INFO:teuthology.task.internal:Removing archive directory... 2023-10-14T10:38:22.976 DEBUG:teuthology.orchestra.run.smithi067:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-14T10:38:22.979 DEBUG:teuthology.orchestra.run.smithi072:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-14T10:38:23.026 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-10-14T10:38:23.038 INFO:teuthology.task.internal:Not uploading archives. 2023-10-14T10:38:23.039 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-10-14T10:38:23.049 INFO:teuthology.task.internal:Tidying up after the test... 2023-10-14T10:38:23.050 DEBUG:teuthology.orchestra.run.smithi067:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-14T10:38:23.052 DEBUG:teuthology.orchestra.run.smithi072:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-14T10:38:23.067 INFO:teuthology.orchestra.run.smithi067.stdout: 262307 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Oct 14 10:38 /home/ubuntu/cephtest 2023-10-14T10:38:23.067 INFO:teuthology.orchestra.run.smithi067.stdout: 393242 4 d--------- 2 ubuntu ubuntu 4096 Oct 14 10:37 /home/ubuntu/cephtest/mnt.0 2023-10-14T10:38:23.069 INFO:teuthology.orchestra.run.smithi067.stderr:find: ‘/home/ubuntu/cephtest/mnt.0’: Permission denied 2023-10-14T10:38:23.069 INFO:teuthology.orchestra.run.smithi067.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2023-10-14T10:38:23.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-14T10:38:23.072 ERROR:teuthology.run_tasks:Manager failed: internal.base Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/task/internal/__init__.py", line 53, in base run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi067 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2023-10-14T10:38:23.080 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-10-14T10:38:23.091 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-10-14T10:38:23.102 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: CommandFailedError: Command failed on smithi067 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2023-10-14T10:38:23.103 INFO:teuthology.nuke:Checking targets against current locks 2023-10-14T10:38:23.111 INFO:teuthology.orchestra.run.smithi072.stdout: 262320 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 14 10:38 /home/ubuntu/cephtest 2023-10-14T10:38:23.144 DEBUG:teuthology.nuke:shortname: smithi067 2023-10-14T10:38:23.144 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-14T10:38:23.169 DEBUG:teuthology.nuke:shortname: smithi072 2023-10-14T10:38:23.170 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-14T10:38:23.174 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi067.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-10-13_18:04:14-fs-rishabh-2023oct13-testing-default-smithi/7425219', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-14 10:17:40.368743', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-10-14T10:38:23.198 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi072.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-10-13_18:04:14-fs-rishabh-2023oct13-testing-default-smithi/7425219', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-14 10:17:40.367152', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-10-14T10:38:23.227 INFO:teuthology.orchestra.console.smithi067:Power off 2023-10-14T10:38:23.228 DEBUG:teuthology.orchestra.console.smithi067:pexpect command: ipmitool -H smithi067.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-14T10:38:23.243 INFO:teuthology.orchestra.console.smithi072:Power off 2023-10-14T10:38:23.244 DEBUG:teuthology.orchestra.console.smithi072:pexpect command: ipmitool -H smithi072.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-14T10:38:23.252 DEBUG:teuthology.orchestra.console.smithi067:power off output: Chassis Power Control: Down/Off 2023-10-14T10:38:23.253 DEBUG:teuthology.orchestra.console.smithi067:pexpect command: ipmitool -H smithi067.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-14T10:38:23.265 DEBUG:teuthology.orchestra.console.smithi072:power off output: Chassis Power Control: Down/Off 2023-10-14T10:38:23.266 DEBUG:teuthology.orchestra.console.smithi072:pexpect command: ipmitool -H smithi072.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-14T10:38:23.276 DEBUG:teuthology.orchestra.console.smithi067:check power output: Chassis Power is on 2023-10-14T10:38:23.291 DEBUG:teuthology.orchestra.console.smithi072:check power output: Chassis Power is on 2023-10-14T10:38:27.278 DEBUG:teuthology.orchestra.console.smithi067:pexpect command: ipmitool -H smithi067.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-14T10:38:27.292 DEBUG:teuthology.orchestra.console.smithi072:pexpect command: ipmitool -H smithi072.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-14T10:38:27.390 DEBUG:teuthology.orchestra.console.smithi067:check power output: Chassis Power is on 2023-10-14T10:38:27.404 DEBUG:teuthology.orchestra.console.smithi072:check power output: Chassis Power is on 2023-10-14T10:38:31.394 DEBUG:teuthology.orchestra.console.smithi067:pexpect command: ipmitool -H smithi067.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-14T10:38:31.405 DEBUG:teuthology.orchestra.console.smithi072:pexpect command: ipmitool -H smithi072.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-14T10:38:31.505 DEBUG:teuthology.orchestra.console.smithi067:check power output: Chassis Power is on 2023-10-14T10:38:31.519 DEBUG:teuthology.orchestra.console.smithi072:check power output: Chassis Power is off 2023-10-14T10:38:31.620 INFO:teuthology.orchestra.console.smithi072:Power off completed 2023-10-14T10:38:35.509 DEBUG:teuthology.orchestra.console.smithi067:pexpect command: ipmitool -H smithi067.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-14T10:38:35.620 DEBUG:teuthology.orchestra.console.smithi067:check power output: Chassis Power is off 2023-10-14T10:38:35.721 INFO:teuthology.orchestra.console.smithi067:Power off completed 2023-10-14T10:38:35.827 INFO:teuthology.run:Summary data: description: fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/basic}} duration: 869.8017616271973 failure_reason: 'Test failure: test_volume_create (tasks.cephfs.test_volumes.TestVolumes)' flavor: default owner: scheduled_rishabh@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=8f19c712116b4ded8e6968d8f7ed2de7 status: fail success: false 2023-10-14T10:38:35.827 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-14T10:38:35.912 INFO:teuthology.run:FAIL